# EZPsych Workbook Guide

## Overview

EZPsych runs experiments from a single Excel workbook (`.xlsx`). The workbook contains four required sheets that together define your questionnaire, scoring rules, virtual respondent dimensions, and runtime settings.

Download `survey_workbook_example.xlsx` as a starting template and modify it for your own study.

---

## Sheet 1: `questionnaire`

Defines what each respondent sees — questions, options, and display settings.

### Columns

| Column | Required | Description |
|--------|----------|-------------|
| `question_id` | Yes | Stable unique identifier (e.g., `EXT1`, `Q3`). Shared with `codebook`. |
| `display_order` | Yes | Integer controlling question sequence. Must be unique across questions. |
| `block_id` | No | Group label for related items (e.g., `EXT` for Extraversion items). |
| `instrument` | No | Scale name (e.g., `BFI`, `PHQ-9`). |
| `item_type` | Yes | One of: `single_choice`, `multiple_choice`, `text`. |
| `question_text` | Yes | The question stem shown to the respondent. |
| `option_id` | Choice only | Short ID for this option (e.g., `A`, `B`, `C`). |
| `option_text` | Choice only | Full label for this option (e.g., "Strongly agree"). |
| `option_order` | Choice only | Integer controlling option display order. |
| `required` | No | `TRUE` or `FALSE`. Metadata only. |
| `allow_multiple` | Choice only | `TRUE` for multiple_choice, `FALSE` for single_choice. |
| `min_selections` | No | Minimum number of options to select. |
| `max_selections` | Choice only | `1` for single_choice. |
| `score_min (text_item_only)` | Text only | Minimum possible score for text items. |
| `score_max (text_item_only)` | Text only | Maximum possible score for text items. |
| `response_max_tokens (text_item_only)` | Text only | Max output tokens for text responses. |

### Row structure

- **Choice questions**: one row per option. All rows for the same `question_id` share the same `question_text`, `display_order`, etc.
- **Text questions**: exactly one row. Leave `option_id`, `option_text`, `option_order` blank.

---

## Sheet 2: `codebook`

Defines how responses are scored and grouped into dimensions.

### Columns

| Column | Required | Description |
|--------|----------|-------------|
| `question_id` | Yes | Must match a `question_id` in `questionnaire`. |
| `dimension` | Yes | Score dimension name. Multiple questions can share the same dimension — their scores will be aggregated together. See "Understanding dimensions" below. |
| `scoring_type` | Yes | One of: `choice_score`, `text_score`, `none`. |
| `score_aggregation` | No | `mean` or `sum`. Controls how multiple items in the same dimension are combined in the summary output. |
| `reverse_scored` | No | `TRUE` / `FALSE`. Metadata only — you must reverse the `score_value` mapping yourself. |
| `response_value` | Choice only | Must match an `option_id` from `questionnaire`. |
| `score_value` | Choice only | The numeric score for this option. |
| `score_description` | No | Human-readable label for this score. |
| `scoring_instructions` | Text only | Instructions for LLM-based rubric scoring. |

### Understanding dimensions

The `dimension` column is how you tell the platform which questions belong together. In psychology, a "dimension" is a construct being measured — for example, the Big Five Inventory has 5 dimensions: Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness, each measured by multiple items.

**How dimensions are used:**

1. **Score aggregation** — items sharing the same `dimension` are combined (via `mean` or `sum`) into a single dimension score per persona in `summary_scores.csv`.
2. **Analysis report** — the platform computes Cronbach's alpha (internal consistency), inter-item correlation matrices, and item discrimination for each dimension. **A dimension needs at least 2 scored items** for these analyses to work.
3. **Item effect** — order-effect monitoring compares scores by dimension.

**Example — BFI Extraversion (4 items in one dimension):**

| question_id | dimension | scoring_type | score_aggregation | reverse_scored | response_value | score_value |
| --- | --- | --- | --- | --- | --- | --- |
| EXT1 | Extraversion | choice_score | mean | FALSE | A | 1 |
| EXT1 | Extraversion | choice_score | mean | FALSE | B | 2 |
| ... | ... | ... | ... | ... | ... | ... |
| EXT2 | Extraversion | choice_score | mean | TRUE | A | 5 |
| EXT2 | Extraversion | choice_score | mean | TRUE | B | 4 |
| ... | ... | ... | ... | ... | ... | ... |

All four EXT items share `dimension=Extraversion`. Their scores are averaged (`score_aggregation=mean`) into a single Extraversion score per persona.

**Common patterns:**

- **Multi-item scale** (BFI, PHQ-9): many items → same dimension, `score_aggregation=mean`
- **Multiple choice count**: each selection = 1 point → own dimension, `score_aggregation=sum`
- **Single standalone question**: one item → its own dimension name
- **Mixed types in one dimension**: different `item_type` / `scoring_type` can share the same dimension. See below.

**Can different question types share a dimension?**

Yes. The `dimension` name is entirely user-defined — you can use any string that represents the construct you are measuring (e.g., `Extraversion`, `Depression`, `生活满意度`, `Team_Performance`). The platform aggregates scores purely by matching dimension names and does not care about `item_type` or `scoring_type`.

For example, the example workbook has a `Resilience` dimension containing two different question types:

| question_id | item_type | scoring_type | dimension | score range |
| --- | --- | --- | --- | --- |
| COPE2 | single_choice | choice_score | Resilience | 1–5 |
| REF1 | text | text_score | Resilience | 1–5 |

When items in the same dimension have different score ranges (e.g., one is 0/1 binary and another is 1–5 Likert), the platform **automatically normalizes** each item's score to [0, 1] before aggregating. This means you can freely mix question types without worrying about scale alignment — just group items by the construct they measure.

### Scoring examples

Normal 5-point item: `A=1, B=2, C=3, D=4, E=5`

Reverse-scored item: `A=5, B=4, C=3, D=2, E=1` — flip the values in `score_value`, not just the `reverse_scored` flag.

For `text_score` items: provide a rubric in the `scoring_instructions` column with labels and criteria. The LLM will assign a rubric label, which maps to a score via `score_value`.

---

## Sheet 3: `personas`

Defines the dimensions for generating virtual respondents via Latin Hypercube Sampling (LHS).

### Format

Wide-format table:
- **Row 1** (header): dimension names (e.g., `age`, `gender`, `region`, `occupation`)
- **Rows 2+**: candidate values for each dimension, listed downward

Example:

| age | gender | region | occupation |
|-----|--------|--------|------------|
| 18-23 | Male | Urban | Student |
| 24-29 | Female | Suburban | Engineer |
| 30-35 | | Rural | Teacher |
| 36-41 | | | Medical staff |

The platform samples from these dimensions using LHS to create diverse, non-repeating persona combinations. Empty cells are skipped for that dimension.

---

## Sheet 4: `experiment_config`

Runtime settings in a two-column `key` / `value` format.

### Required fields

| Key | Description | Example |
|-----|-------------|---------|
| `sample_size` | Number of personas to run in the main flow | `20` |
| `repeats` | Times each persona answers each question | `1` |

### Model selection

| Key | Default | Description |
|-----|---------|-------------|
| `model` | `google/gemini-3.1-flash-lite-preview` | LLM used for the main flow and question-order monitoring. **Optional.** If set, it must be one of the platform's supported OpenRouter model ids (see below) — anything else is rejected at upload time with a clear error. If omitted, the platform uses its default model. |

Supported values (`budget.SUPPORTED_MODELS`, 13 total):

```
google/gemini-3.1-flash-lite-preview   (default)
openai/gpt-4o
openai/gpt-4o-mini
openai/gpt-4.1-mini
openai/gpt-4.1-nano
openai/o4-mini
google/gemini-2.5-flash
google/gemini-2.5-pro
x-ai/grok-3-mini
x-ai/grok-4-fast
deepseek/deepseek-chat-v3-0324
meta-llama/llama-4-scout
meta-llama/llama-4-maverick
```

A workbook written for an older version of this doc may set `model` to an
arbitrary OpenRouter id (e.g. a value not on this list) — that now fails
validation on upload. Pick one of the ids above, or delete the `model` row
entirely to use the default.

The cost estimate, the actual respondent calls, and billing all resolve
`model` through the same whitelist check, so the price you're quoted and the
model that actually runs cannot disagree.

> Note: `persona_narrative_model` and `text_score_model` (below) are separate
> settings and are **not** checked against this whitelist — any value you put
> there is used as-is.

### Persona generation fields

| Key | Default | Description |
|-----|---------|-------------|
| `persona_seed` | `42` | Random seed for reproducible persona generation |
| `persona_oversample_mult` | `2.0` | LHS oversampling factor. A value of `2.0` means each generation round draws 2x the remaining target count as candidates, then de-duplicates. Increase this if your dimension space has limited unique combinations. |
| `persona_template` | auto | Python format string for persona text. Use dimension names as `{placeholders}`. Example: `A {gender} aged {age}, from {region}, working as a {occupation}.` |

> **Note:** `persona_count` is no longer needed. The platform automatically generates `max(sample_size, item_effect_sample_size)` personas.

### Question-order monitoring fields

| Key | Default | Description |
|-----|---------|-------------|
| `item_effect_enabled` | `false` | Set to `true` to run question-order monitoring after the main flow |
| `item_effect_sample_size` | = sample_size | Number of personas for monitoring (must be <= sample_size) |

### Text scoring fields

| Key | Default | Description |
|-----|---------|-------------|
| `text_score_enabled` | `true` | Enable LLM-based rubric scoring for `text_score` items |

> **Note:** The text scoring model defaults to the same model specified in `model`. You do not need to configure it separately.

### Respondent-call options (opt-in — these change your cost)

| Key | Type | Range | Default | Cost impact |
|-----|------|-------|---------|-------------|
| `temperature` | number | `0.0`–`2.0` | `0.2` | None. Higher = more varied, less deterministic answers. |
| `enable_thinking` | boolean | `true` / `false` | `false` | **≈ x8 per-call cost.** The model reasons before answering; reasoning tokens are billed as output tokens. |
| `enable_websearch` | boolean | `true` / `false` | `false` | **≈ x40 per-call cost.** Adds a flat **$0.02 per respondent call** — a per-request plugin fee, not a token charge. |

Booleans accept the usual spellings: `true`/`false`, `TRUE`/`FALSE`, `yes`/`no`,
`1`/`0`. Anything else is rejected with a validation error.

These options apply to the main flow **and** to question-order monitoring. They do
not affect persona biographies or text scoring, which keep their own fixed
settings. Leave all three out and the run behaves and prices exactly as before.

> **Cost warning:** `enable_websearch` is the expensive one. A questionnaire answer
> is only a handful of tokens, so a flat $0.02 search fee per call dominates
> everything else — 20 personas x 6 questions is already $2.40 of search fees
> before any tokens are counted. Turn it on only when the answers genuinely need
> current, real-world information. The cost estimate shown before you start the run
> includes these surcharges; unused frozen credits are released when the run ends.

### Reserved fields — not usable yet

| Key | Default | Status |
|-----|---------|--------|
| `answering_mode` | `independent` | Reserved for an in-development "contextual" mode where a respondent answers the whole questionnaire in one running conversation instead of one call per question. **Any value other than `independent` is currently rejected at upload** ("contextual answering mode is not yet available"). Leave this out. |
| `questions_per_call` | `1` | Reserved for contextual mode (how many questions go into one turn). Has no effect while `answering_mode` is (necessarily) `independent`. |
| `context_max_turns` | `0` | Reserved for contextual mode (how much conversation history to keep). Has no effect in `independent` mode. |
| `question_order` | `fixed` | Reserved for contextual mode. `shuffle` gives each respondent their own randomized question order, but that logic only runs inside the contextual code path — in the current `independent`-only mode, questions always run in `display_order` regardless of this setting. |

Do not set these fields in a production workbook. They exist in the schema so the
upcoming contextual mode has somewhere to read its config from, not because they
do anything today.

### Fields you should NOT put in the workbook

The following are configured via environment variables (`.env` file), not the workbook:

- `base_url` — defaults to `https://openrouter.ai/api/v1`
- `api_key_env` — defaults to `OPENROUTER_API_KEY`

---

## Common Mistakes

1. **Renaming a sheet** — keep exact names: `questionnaire`, `codebook`, `personas`, `experiment_config`
2. **Changing header columns** — headers must match exactly
3. **Forgetting codebook rows** — every `question_id` in `questionnaire` must appear in `codebook`
4. **Reverse scoring** — setting `reverse_scored=true` without reversing the actual `score_value` numbers
5. **Text questions with option rows** — text items should have exactly one row with no option columns
6. **Duplicate display_order** — each question must have a unique `display_order` value
7. **Putting API keys in the workbook** — use the `.env` file instead

---

## Quick Start

1. Download `survey_workbook_example.xlsx`
2. Copy it and rename for your study
3. Replace `questionnaire` and `codebook` content with your items
4. Adjust `personas` dimensions for your target population
5. Set `sample_size` and `repeats` in `experiment_config`; optionally set `model` to one of the supported ids above (otherwise the default is used)
6. Upload to EZPsych and click "Start Run"

---

## AI-Assisted Workbook Creation

If you want an AI assistant to help create a workbook, paste this prompt:

```
Help me create an EZPsych workbook (.xlsx) with these four sheets:

1. questionnaire — my items with question_id, display_order, item_type, question_text, and options
2. codebook — scoring rules matching each question_id, with dimension, scoring_type, and score_value
3. personas — wide-format LHS dimensions (one column per dimension, values downward)
4. experiment_config — key/value pairs: sample_size, repeats, and any optional settings (model is optional too — only set it if you need something other than the platform default, and only from the supported model list)

Rules:
- Every question_id in questionnaire must have matching rows in codebook
- For choice items, response_value must match option_id
- Reverse-scored items need reversed score_value numbers (not just the flag)
- Text items get one row with no option columns
- If you set `model`, it must be one of the platform's supported OpenRouter ids — do not invent one
- Do NOT include api_key_env or base_url in experiment_config
- Do NOT set answering_mode to anything other than independent (or leave it out) — contextual mode is not available yet

My questionnaire: [describe your items here]
```