# EZPsych JSON Experiment Spec

This document describes the JSON format accepted by the EZPsych v1 API. It is the
programmatic equivalent of the Excel workbook described in
`docs/workbook_creation_prompt.md` — the platform converts a JSON spec into that
exact 4-sheet workbook before running anything, so the semantics are identical,
**with one exception:** `experiment_config.model` (see
["Ignored keys"](#ignored-keys-warning) below) — the JSON path always runs the
platform default model, while the workbook path lets you pick from a whitelist.

Written for AI agents: every field name, type, enum and constraint is listed
explicitly, together with the mistakes that most often produce validation errors.

---

## Top-level shape

```json
{
  "version": 1,
  "questionnaire": [ ... ],
  "codebook": [ ... ],
  "personas": [ ... ],
  "experiment_config": { ... }
}
```

| Key | Type | Required | Maps to sheet |
|-----|------|----------|---------------|
| `version` | integer | No | — (metadata, ignored) |
| `name` / `title` / `description` / `notes` | string | No | — (metadata, ignored) |
| `questionnaire` | array of objects | **Yes** | `questionnaire` |
| `codebook` | array of objects | **Yes** | `codebook` |
| `personas` | array of objects | **Yes** | `personas` |
| `experiment_config` | object | **Yes** | `experiment_config` |

Any other top-level key produces a warning and is ignored.

### Size limits

Specs are checked against these caps **before** anything else runs, so a spec
that breaks one gets a fast validation error and no cost estimate. They sit far
above any realistic questionnaire:

| Limit | Maximum |
|-------|---------|
| Request body | 512 KB |
| `questionnaire` questions | 400 |
| `options` per question | 26 |
| `personas` dimensions | 40 |
| `values` per persona dimension | 60 |
| `experiment_config.sample_size` | 5000 |
| `experiment_config.repeats` | 10 |
| `experiment_config.item_effect_sample_size` | 5000 |

An oversized request body is rejected with `413` and
`{"error": "spec_too_large"}`; the structural caps come back as ordinary
`{"path": ..., "message": ...}` validation errors.

---

## `questionnaire` — the items respondents see

One object per **question** (not per option — options are nested, unlike the
Excel layout).

| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `question_id` | string | **Yes** | Stable unique id, e.g. `"EXT1"`. Must be unique across the array and is the join key with `codebook`. |
| `item_type` | string enum | **Yes** | `"single_choice"` \| `"multiple_choice"` \| `"text"` |
| `question_text` | string | **Yes** | The question stem. Must be non-empty. |
| `display_order` | integer ≥ 1 | No | Must be unique. If **no** item in the array declares it, order is auto-assigned from array position. If **any** item declares it, **all** items must. |
| `block_id` | string | No | Group label (e.g. `"EXT"`). Defaults to `question_id` with a warning. |
| `instrument` | string | No | Scale name (e.g. `"BFI"`). Defaults to `"custom"` with a warning. |
| `required` | boolean | No | Default `true`. Metadata only. |
| `allow_multiple` | boolean | No | Default: `true` for `multiple_choice`, `false` otherwise. Must match `item_type`. |
| `min_selections` | integer ≥ 0 | No | Default `1`. |
| `max_selections` | integer ≥ 1 | No | Choice items only. Forced to `1` for `single_choice`. For `multiple_choice` must be ≥ `min_selections`; omit for "no limit". |
| `options` | array of objects | Choice items only | See below. Forbidden on `text` items. |
| `score_min` | number | **Text items only** | Lowest possible score for the item. |
| `score_max` | number | **Text items only** | Highest possible score. Must be `> score_min`. |
| `response_max_tokens` | integer ≥ 1 | Text items only | Output token cap for the free-text answer. |

### `options` entries (choice items)

| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `option_id` | string | **Yes** | Short id, e.g. `"A"`. Unique within the question. This is the key used in `codebook.score_map`. |
| `option_text` | string | **Yes** | Label shown to the respondent. |
| `option_order` | integer ≥ 1 | No | Unique within the question. Auto-assigned from array position if no option in that question declares it. |

Constraints:

- `single_choice` requires ≥ 1 option, `multiple_choice` requires ≥ 2.
- `text` items must not define `options`, `allow_multiple: true`, or `max_selections`.
- Only `text` items may define `score_min`, `score_max`, `response_max_tokens`.

---

## `codebook` — scoring rules

Exactly **one object per `question_id`**. Every question in `questionnaire` must
have an entry, and every entry must reference an existing question.

| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `question_id` | string | **Yes** | Must match a `questionnaire` entry. No duplicates. |
| `scoring_type` | string enum | **Yes** | `"choice_score"` \| `"text_score"` \| `"none"` |
| `dimension` | string | Yes when scored | Construct name, e.g. `"Extraversion"`. Items sharing a dimension are aggregated together. |
| `score_aggregation` | string enum | No | `"mean"` \| `"sum"` \| `"max"` \| `"min"`. Default `"mean"`. Forced to `"none"` when `scoring_type` is `"none"`. |
| `reverse_scored` | boolean | No | Default `false`. **Metadata only** — you must reverse the numbers in `score_map` yourself. |
| `score_map` | object | Yes when scored | Maps a response label to a numeric score. |
| `score_descriptions` | object | Required for `text_score` | Maps the same labels to rubric criteria text. |
| `scoring_instructions` | string | `text_score` only | Instructions handed to the LLM rubric scorer. |

### `score_map` semantics by `scoring_type`

- **`choice_score`** — keys are `option_id` values. The key set must be
  **exactly** the question's option ids: no missing options, no unknown keys.
  Values are numbers (`{"A": 1, "B": 2, "C": 3, "D": 4, "E": 5}`).
- **`text_score`** — keys are free-form rubric labels (e.g. `"LEVEL_1"`). Every
  label needs a non-empty `score_descriptions` entry, and every value must fall
  inside the item's `[score_min, score_max]` range. Only valid on `text` items.
- **`none`** — `score_map`, `score_descriptions` and `scoring_instructions` must
  all be absent. The question is asked but not scored.

Hard rules enforced by the validator:

- `item_type: "text"` ⇔ `scoring_type: "text_score"` (both directions).
- Choice items may only use `"choice_score"` or `"none"`.
- Scored questions cannot use `score_aggregation: "none"`.

### Understanding dimensions

`dimension` groups items into a measured construct. Items sharing a dimension are
combined into a single per-persona score using `score_aggregation`, and the
analysis report computes Cronbach's alpha, inter-item correlations and item
discrimination per dimension.

- A dimension needs **at least 2 scored items** for reliability analysis — a
  single-item dimension produces a warning.
- Different `item_type` / `scoring_type` values may share a dimension. When their
  score ranges differ (e.g. 0/1 vs 1–5), the platform normalizes each item to
  `[0, 1]` before aggregating.
- Use one `score_aggregation` per dimension (mixing them produces a warning).

Reverse scoring example: a normal item is `A=1 … E=5`; the reverse-keyed item is
`A=5 … E=1`. Flip the numbers in `score_map` — setting `reverse_scored: true`
alone changes nothing.

---

## `personas` — virtual respondent dimensions

An array of dimension definitions. The platform draws non-repeating combinations
via Latin Hypercube Sampling.

| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `name` | string | **Yes** | Dimension name, e.g. `"Age"`. Unique across the array. |
| `values` | array of strings | **Yes** | Candidate values. Non-empty, no duplicates within the dimension. |

```json
"personas": [
  {"name": "Age",    "values": ["18-24", "25-34", "35-44", "45-54"]},
  {"name": "Gender", "values": ["Male", "Female"]}
]
```

Constraints:

- `name` may not be one of the reserved column names: `persona_id`, `id`, `text`,
  `description`, `persona_text`, `dimension`, `field`, `name`, `placeholder`,
  `field_key`, `template_key`, `value`, `category`, `level`, `option`, `values`,
  `categories`, `levels`, `options`, `weight`, `weights`.
- The product of all `values` lengths must be **≥ the number of personas needed**
  (`max(sample_size, item_effect_sample_size)`), otherwise LHS cannot produce
  enough unique respondents.
- Each dimension gets a `persona_template` placeholder derived from `name`:
  lowercased, with every run of non-alphanumeric characters replaced by `_`
  (`"Education level"` → `{education_level}`).

---

## `experiment_config` — runtime settings

A **flat** JSON object (no nesting). Keys are case-insensitive and normalized to
lowercase.

### Required

| Key | Type | Notes |
|-----|------|-------|
| `sample_size` | integer ≥ 1 | Number of personas in the main flow. |

### Common optional keys

| Key | Type | Default | Notes |
|-----|------|---------|-------|
| `repeats` | integer ≥ 1 | `1` | How many times each persona answers each question. |
| `persona_seed` | integer | `42` | Seed for reproducible persona generation. |
| `persona_oversample_mult` | number > 0 | `2.0` | LHS oversampling factor. Raise it when the dimension space is small. |
| `persona_template` | string | auto | Python format string, e.g. `"A {gender} aged {age}, working as a {occupation}."` Placeholders must exist (see above). |
| `persona_count` | integer ≥ 1 | auto | Overrides the derived persona count. Normally omit it. |
| `item_effect_enabled` | boolean | `false` | Run question-order monitoring after the main flow. |
| `item_effect_sample_size` | integer ≥ 2 | `sample_size` | Only when `item_effect_enabled` is true; must be ≤ `sample_size`. |
| `text_score_enabled` | boolean | `true` | LLM rubric scoring for `text_score` items. |
| `report_enabled` | boolean | `true` | Generate the analysis report. |
| `report_narrative_enabled` | boolean | `true` | LLM-written narrative sections in the report. |
| `persona_narrative_enabled` | boolean | `true` | LLM-written persona biographies. |

### Reserved keys — accepted, but not usable yet

These four are recognized (type-checked, no "unknown key" warning) but exist for
an in-development "contextual answering" mode that batches multiple questions
into one running conversation per respondent instead of one call per question.
**`answering_mode` other than `"independent"` is currently rejected** —
"contextual answering mode is not yet available" — so do not set it. The other
three only take effect inside that mode's code path and are inert under the
`independent` mode this API currently runs:

| Key | Type | Default | Notes |
|-----|------|---------|-------|
| `answering_mode` | string enum | `"independent"` | Only `"independent"` is currently accepted; any other value is a validation error. |
| `questions_per_call` | integer ≥ 1 | `1` | No effect while `answering_mode` is `independent`. |
| `context_max_turns` | integer ≥ 0 | `0` | No effect while `answering_mode` is `independent`. |
| `question_order` | string enum | `"fixed"` | `"shuffle"` only randomizes order inside the contextual code path. Under `independent` mode, questions always run in `display_order` regardless of this value. |

`answering_mode="contextual"` is also permanently incompatible with
`item_effect_enabled=true`, even once contextual mode ships: question-order
monitoring assumes each answer is independent of question order, which
contextual mode removes. The documented alternative once available will be
`question_order="shuffle"`.

### Respondent-call options (opt-in, cost impact)

These three keys change how every simulated respondent answers. They apply to the
main flow **and** to question-order monitoring; persona-narrative and text-scoring
calls are unaffected.

| Key | Type | Range | Default | Cost impact |
|-----|------|-------|---------|-------------|
| `temperature` | number | `0.0`–`2.0` | `0.2` | None. Higher values give more varied answers. |
| `enable_thinking` | boolean | `true` / `false` | `false` | **≈ ×8 per-call cost.** Reasoning tokens are billed as output tokens. |
| `enable_websearch` | boolean | `true` / `false` | `false` | **≈ ×40 per-call cost** — a flat **$0.02 per respondent call** on top of tokens. |

Notes:

- `enable_thinking` requests `reasoning: {"enabled": true, "effort": "medium"}`
  from OpenRouter. The reasoning trace is excluded from the response, so answers
  parse exactly as before, but the tokens are still billed. The runner also raises
  `max_tokens` ×8 so reasoning cannot truncate the answer.
- `enable_websearch` attaches OpenRouter's web plugin (`plugins: [{"id": "web"}]`).
  The pinned model id is never changed. The plugin fee is charged per request, not
  per token, which is why it dwarfs the token cost of a short questionnaire answer.
- Both flags are estimated conservatively up front. Unused frozen credits are
  released when the job finishes, so an over-estimate costs you nothing.
- A temperature outside `0.0`–`2.0`, or a non-boolean flag value, is a validation
  error at `experiment_config.<key>`.

### Rejected keys (validation error)

Credentials and runtime paths are configured server-side and may never appear in
a spec: `api_key_env`, `api_key_value`, `base_url`, `text_score_api_key_env`,
`text_score_api_key_value`, `text_score_base_url`, `outdir`, `plan_csv`,
`plan_csv_encoding`, `retry_from_outdir`, `survey_workbook_file`,
`workbook_questionnaire_sheet`, `workbook_codebook_sheet`,
`workbook_personas_sheet`, `workbook_config_sheet`.

### Ignored keys (warning)

`model` and `text_score_model` are pinned for this JSON API intake path
specifically: supplying either key produces a warning and the value is
discarded, and every run submitted through `POST /api/v1/experiments` uses
`budget.DEFAULT_MODEL` (currently `google/gemini-3.1-flash-lite-preview`).

This is different from the Excel-workbook intake path (see
[`workbook_creation_prompt.md`](/docs/agent/workbook_creation_prompt.md)),
where `model` **is** honored and can be set to any id in the platform's
supported-model whitelist (`budget.SUPPORTED_MODELS`) — that path validates
the value at upload instead of discarding it. If your integration needs a
non-default model, use the workbook path; the JSON API does not currently
expose model selection.

Unknown keys produce a warning and are forwarded to the runner as-is.

---

## API endpoints

### `POST /api/v1/experiments/validate`

Public, no authentication, rate limited (5/min, 30/hour), max body 512 KB, plus
the structural caps listed under [Size limits](#size-limits). Runs **zero** LLM
calls.

Request body: the spec object. Response `200`:

```json
{
  "valid": true,
  "errors": [],
  "warnings": [{"path": "questionnaire[0].instrument", "message": "missing, defaulted to 'custom'"}],
  "estimate": {
    "question_count": 6,
    "persona_count": 20,
    "total_calls": 448,
    "estimated_price_usd": 0.2355,
    "estimated_credits": 2355,
    "feature_costs": {
      "temperature": 0.2,
      "enable_thinking": false,
      "thinking_multiplier": 1,
      "thinking_extra_usd": 0.0,
      "enable_websearch": false,
      "websearch_per_call_usd": 0.0,
      "websearch_calls": 0,
      "websearch_usd": 0.0
    }
  }
}
```

`feature_costs` breaks out what `enable_thinking` and `enable_websearch` added to
the price, so you can show the surcharge separately. `*_usd` fields are raw
provider cost; the matching `priced_*_usd` fields are those amounts after markup
and safety margin.

`estimate` is `null` when `valid` is `false`. Errors and warnings are always
`{"path": ..., "message": ...}` objects, where `path` is a JSON pointer-ish
expression such as `"codebook[3].score_map.A"` or
`"questionnaire[1].options[2].option_id"`. Validation accumulates as many
problems as possible in one pass — fix them all, then re-submit.

### `POST /api/v1/experiments`

API-key authenticated (`Authorization: Bearer <key>`), max body 512 KB, same
structural caps as `validate`. Validates the spec, converts it to a workbook,
freezes credits and enqueues the run.

Response `201`:

```json
{"job_id": "20260727T061200Z-1a2b3c4d", "status": "queued", "estimated_credits": 2355, "warnings": []}
```

Errors:

| Status | `error` | Meaning |
|--------|---------|---------|
| `400` | `invalid_spec` | Validation failed; `errors` lists the problems. |
| `400` | `invalid_json` | Body is not valid JSON. |
| `401` | — | Missing or invalid API key. |
| `402` | `insufficient_credits` | Includes `needed`, `have` and the full `estimate`. |
| `402` | `monthly_cap_exceeded` | The API key's monthly cap would be exceeded; includes `monthly_cap_credits`, `used_this_month` and `needed`. |
| `403` | `email_not_verified` | Verify the account's email first. |
| `413` | `spec_too_large` | Body exceeds 2 MB. |
| `503` | `platform_maintenance` | Provider balance too low to accept new runs. |

---

## Common mistakes

1. **Flattening options** — the JSON format nests `options` inside each question.
   Do not repeat a question object once per option (that is the Excel layout).
2. **`score_map` keys that are not `option_id`s** — for `choice_score`, the keys
   must exactly match the question's option ids, all of them.
3. **Reverse scoring the flag only** — flip the numbers in `score_map`;
   `reverse_scored` is metadata.
4. **Text items with options** or choice items with `score_min`/`score_max`.
5. **`text_score` without `score_descriptions`** — every rubric label needs
   criteria text, and every score must be inside `[score_min, score_max]`.
6. **Missing codebook entries** — every `question_id` needs exactly one entry.
7. **Duplicate `display_order`** — or declaring it on some questions but not all.
8. **Too few persona combinations** — `sample_size` above the product of the
   dimension value counts.
9. **`persona_template` placeholders that do not exist** — use the normalized,
   lowercased dimension names.
10. **Putting API keys or paths in `experiment_config`.**

---

## Full runnable example

A small Big-Five-style spec with two dimensions, one reverse-keyed item, one
multiple-choice item and one LLM-scored free-text item.

```json
{
  "version": 1,
  "name": "Mini Big Five pilot",
  "questionnaire": [
    {
      "question_id": "EXT1",
      "display_order": 1,
      "block_id": "EXT",
      "instrument": "MiniBFI",
      "item_type": "single_choice",
      "question_text": "I am the life of the party.",
      "options": [
        {"option_id": "A", "option_text": "Strongly disagree"},
        {"option_id": "B", "option_text": "Disagree"},
        {"option_id": "C", "option_text": "Neither agree nor disagree"},
        {"option_id": "D", "option_text": "Agree"},
        {"option_id": "E", "option_text": "Strongly agree"}
      ]
    },
    {
      "question_id": "EXT2",
      "display_order": 2,
      "block_id": "EXT",
      "instrument": "MiniBFI",
      "item_type": "single_choice",
      "question_text": "I don't talk a lot.",
      "options": [
        {"option_id": "A", "option_text": "Strongly disagree"},
        {"option_id": "B", "option_text": "Disagree"},
        {"option_id": "C", "option_text": "Neither agree nor disagree"},
        {"option_id": "D", "option_text": "Agree"},
        {"option_id": "E", "option_text": "Strongly agree"}
      ]
    },
    {
      "question_id": "CON1",
      "display_order": 3,
      "block_id": "CON",
      "instrument": "MiniBFI",
      "item_type": "single_choice",
      "question_text": "I am always prepared.",
      "options": [
        {"option_id": "A", "option_text": "Strongly disagree"},
        {"option_id": "B", "option_text": "Disagree"},
        {"option_id": "C", "option_text": "Neither agree nor disagree"},
        {"option_id": "D", "option_text": "Agree"},
        {"option_id": "E", "option_text": "Strongly agree"}
      ]
    },
    {
      "question_id": "CON2",
      "display_order": 4,
      "block_id": "CON",
      "instrument": "MiniBFI",
      "item_type": "single_choice",
      "question_text": "I leave my belongings around.",
      "options": [
        {"option_id": "A", "option_text": "Strongly disagree"},
        {"option_id": "B", "option_text": "Disagree"},
        {"option_id": "C", "option_text": "Neither agree nor disagree"},
        {"option_id": "D", "option_text": "Agree"},
        {"option_id": "E", "option_text": "Strongly agree"}
      ]
    },
    {
      "question_id": "HAB1",
      "display_order": 5,
      "block_id": "HAB",
      "instrument": "MiniBFI",
      "item_type": "multiple_choice",
      "question_text": "Which of the following do you do in a typical week? (Select all that apply)",
      "min_selections": 1,
      "max_selections": 4,
      "options": [
        {"option_id": "A", "option_text": "Plan the week in advance"},
        {"option_id": "B", "option_text": "Keep a written to-do list"},
        {"option_id": "C", "option_text": "Tidy my workspace"},
        {"option_id": "D", "option_text": "Review what I finished"}
      ]
    },
    {
      "question_id": "REF1",
      "display_order": 6,
      "block_id": "REF",
      "instrument": "MiniBFI",
      "item_type": "text",
      "question_text": "Describe how you organised your last busy week. What did you plan, and what actually happened?",
      "score_min": 1,
      "score_max": 5,
      "response_max_tokens": 200
    }
  ],
  "codebook": [
    {
      "question_id": "EXT1",
      "dimension": "Extraversion",
      "scoring_type": "choice_score",
      "score_aggregation": "mean",
      "reverse_scored": false,
      "score_map": {"A": 1, "B": 2, "C": 3, "D": 4, "E": 5}
    },
    {
      "question_id": "EXT2",
      "dimension": "Extraversion",
      "scoring_type": "choice_score",
      "score_aggregation": "mean",
      "reverse_scored": true,
      "score_map": {"A": 5, "B": 4, "C": 3, "D": 2, "E": 1}
    },
    {
      "question_id": "CON1",
      "dimension": "Conscientiousness",
      "scoring_type": "choice_score",
      "score_aggregation": "mean",
      "reverse_scored": false,
      "score_map": {"A": 1, "B": 2, "C": 3, "D": 4, "E": 5}
    },
    {
      "question_id": "CON2",
      "dimension": "Conscientiousness",
      "scoring_type": "choice_score",
      "score_aggregation": "mean",
      "reverse_scored": true,
      "score_map": {"A": 5, "B": 4, "C": 3, "D": 2, "E": 1}
    },
    {
      "question_id": "HAB1",
      "dimension": "Conscientiousness",
      "scoring_type": "choice_score",
      "score_aggregation": "mean",
      "reverse_scored": false,
      "score_map": {"A": 1, "B": 1, "C": 1, "D": 1}
    },
    {
      "question_id": "REF1",
      "dimension": "Conscientiousness",
      "scoring_type": "text_score",
      "score_aggregation": "mean",
      "reverse_scored": false,
      "score_map": {"LEVEL_1": 1, "LEVEL_2": 2, "LEVEL_3": 3, "LEVEL_4": 4, "LEVEL_5": 5},
      "score_descriptions": {
        "LEVEL_1": "No planning described; the answer is vague or avoidant.",
        "LEVEL_2": "Mentions intentions but no concrete plan or follow-through.",
        "LEVEL_3": "Describes a basic plan with limited detail on execution.",
        "LEVEL_4": "Clear plan, concrete actions, and some reflection on results.",
        "LEVEL_5": "Detailed plan, multiple strategies, adjustments, and reflection on outcomes."
      },
      "scoring_instructions": "Score the answer on a 1 to 5 scale for organisational behaviour. Judge whether the response shows advance planning, concrete follow-through, and reflection on the outcome. Return only the best matching rubric label."
    }
  ],
  "personas": [
    {"name": "Age", "values": ["18-24", "25-34", "35-44", "45-54", "55-64"]},
    {"name": "Gender", "values": ["Male", "Female"]},
    {"name": "Occupation", "values": ["Student", "Software engineer", "Teacher", "Healthcare worker", "Retired"]},
    {"name": "Education", "values": ["High school", "Bachelor's degree", "Master's degree"]}
  ],
  "experiment_config": {
    "sample_size": 20,
    "repeats": 1,
    "persona_seed": 42,
    "persona_template": "A {gender} aged {age}, working as a {occupation}. Education: {education}.",
    "item_effect_enabled": true,
    "item_effect_sample_size": 10,
    "text_score_enabled": true
  }
}
```

Validate it with:

```bash
curl -X POST https://ezpsy.ai/api/v1/experiments/validate \
  -H "Content-Type: application/json" \
  --data @spec.json
```

Then submit it with:

```bash
curl -X POST https://ezpsy.ai/api/v1/experiments \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $EZPSYCH_API_KEY" \
  --data @spec.json
```