EZPsych for AI Agents
Machine-readable entry point: /llms.txt
EZPsych (http://ezpsy.ai) lets researchers pilot-test questionnaires with LLM-simulated respondents: you upload (or submit as JSON) a questionnaire, virtual personas answer it, and the platform returns scored, psychometrically-analyzed results — no human participants required.
If you are an autonomous agent building an experiment on behalf of a human operator, this page is your
starting point. The documents below are also served as raw text/plain markdown at
the URLs shown, so you can fetch them directly without parsing HTML.
Contents
- Workbook / experiment creation guide — How to define a questionnaire, scoring rules, personas, and run settings. (raw: workbook_creation_prompt.md)
- Experiment JSON spec — The JSON schema accepted by POST /api/v1/experiments/validate and POST /api/v1/experiments. (raw: experiment_json_spec.md)
- API reference — Full REST API: auth, endpoints, request/response examples, error semantics, rate limits. (raw: api_reference.md)
- Market research guide — Using EZPsych for concept tests, ad/copy comparisons, Van Westendorp pricing research, and questionnaire QA before fieldwork. (raw: market_research_guide.md)
Quick start
- Read the workbook / spec guide below to learn how a questionnaire, scoring rules, personas, and run settings are represented.
- Build an experiment JSON spec per the experiment JSON spec.
- Call
POST /api/v1/experiments/validate(public, free, no LLM calls) and fix anyerrorsuntil"valid": true. See the API reference. - Ask your human operator to register at
http://ezpsy.ai/registerand create an API key at/settings/api-keys, then hand you the key. - Submit with
POST /api/v1/experimentsusing that key, poll status, and download results.
Workbook / experiment creation guide
How to define a questionnaire, scoring rules, personas, and run settings. — raw markdown: workbook_creation_prompt.md
EZPsych Workbook Guide
Overview
EZPsych runs experiments from a single Excel workbook (.xlsx). The workbook contains four required sheets that together define your questionnaire, scoring rules, virtual respondent dimensions, and runtime settings.
Download survey_workbook_example.xlsx as a starting template and modify it for your own study.
Sheet 1: questionnaire
Defines what each respondent sees — questions, options, and display settings.
Columns
| Column | Required | Description |
|---|---|---|
question_id |
Yes | Stable unique identifier (e.g., EXT1, Q3). Shared with codebook. |
display_order |
Yes | Integer controlling question sequence. Must be unique across questions. |
block_id |
No | Group label for related items (e.g., EXT for Extraversion items). |
instrument |
No | Scale name (e.g., BFI, PHQ-9). |
item_type |
Yes | One of: single_choice, multiple_choice, text. |
question_text |
Yes | The question stem shown to the respondent. |
option_id |
Choice only | Short ID for this option (e.g., A, B, C). |
option_text |
Choice only | Full label for this option (e.g., "Strongly agree"). |
option_order |
Choice only | Integer controlling option display order. |
required |
No | TRUE or FALSE. Metadata only. |
allow_multiple |
Choice only | TRUE for multiple_choice, FALSE for single_choice. |
min_selections |
No | Minimum number of options to select. |
max_selections |
Choice only | 1 for single_choice. |
score_min (text_item_only) |
Text only | Minimum possible score for text items. |
score_max (text_item_only) |
Text only | Maximum possible score for text items. |
response_max_tokens (text_item_only) |
Text only | Max output tokens for text responses. |
Row structure
- Choice questions: one row per option. All rows for the same
question_idshare the samequestion_text,display_order, etc. - Text questions: exactly one row. Leave
option_id,option_text,option_orderblank.
Sheet 2: codebook
Defines how responses are scored and grouped into dimensions.
Columns
| Column | Required | Description |
|---|---|---|
question_id |
Yes | Must match a question_id in questionnaire. |
dimension |
Yes | Score dimension name. Multiple questions can share the same dimension — their scores will be aggregated together. See "Understanding dimensions" below. |
scoring_type |
Yes | One of: choice_score, text_score, none. |
score_aggregation |
No | mean or sum. Controls how multiple items in the same dimension are combined in the summary output. |
reverse_scored |
No | TRUE / FALSE. Metadata only — you must reverse the score_value mapping yourself. |
response_value |
Choice only | Must match an option_id from questionnaire. |
score_value |
Choice only | The numeric score for this option. |
score_description |
No | Human-readable label for this score. |
scoring_instructions |
Text only | Instructions for LLM-based rubric scoring. |
Understanding dimensions
The dimension column is how you tell the platform which questions belong together. In psychology, a "dimension" is a construct being measured — for example, the Big Five Inventory has 5 dimensions: Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness, each measured by multiple items.
How dimensions are used:
- Score aggregation — items sharing the same
dimensionare combined (viameanorsum) into a single dimension score per persona insummary_scores.csv. - Analysis report — the platform computes Cronbach's alpha (internal consistency), inter-item correlation matrices, and item discrimination for each dimension. A dimension needs at least 2 scored items for these analyses to work.
- Item effect — order-effect monitoring compares scores by dimension.
Example — BFI Extraversion (4 items in one dimension):
| question_id | dimension | scoring_type | score_aggregation | reverse_scored | response_value | score_value |
|---|---|---|---|---|---|---|
| EXT1 | Extraversion | choice_score | mean | FALSE | A | 1 |
| EXT1 | Extraversion | choice_score | mean | FALSE | B | 2 |
| ... | ... | ... | ... | ... | ... | ... |
| EXT2 | Extraversion | choice_score | mean | TRUE | A | 5 |
| EXT2 | Extraversion | choice_score | mean | TRUE | B | 4 |
| ... | ... | ... | ... | ... | ... | ... |
All four EXT items share dimension=Extraversion. Their scores are averaged (score_aggregation=mean) into a single Extraversion score per persona.
Common patterns:
- Multi-item scale (BFI, PHQ-9): many items → same dimension,
score_aggregation=mean - Multiple choice count: each selection = 1 point → own dimension,
score_aggregation=sum - Single standalone question: one item → its own dimension name
- Mixed types in one dimension: different
item_type/scoring_typecan share the same dimension. See below.
Can different question types share a dimension?
Yes. The dimension name is entirely user-defined — you can use any string that represents the construct you are measuring (e.g., Extraversion, Depression, 生活满意度, Team_Performance). The platform aggregates scores purely by matching dimension names and does not care about item_type or scoring_type.
For example, the example workbook has a Resilience dimension containing two different question types:
| question_id | item_type | scoring_type | dimension | score range |
|---|---|---|---|---|
| COPE2 | single_choice | choice_score | Resilience | 1–5 |
| REF1 | text | text_score | Resilience | 1–5 |
When items in the same dimension have different score ranges (e.g., one is 0/1 binary and another is 1–5 Likert), the platform automatically normalizes each item's score to [0, 1] before aggregating. This means you can freely mix question types without worrying about scale alignment — just group items by the construct they measure.
Scoring examples
Normal 5-point item: A=1, B=2, C=3, D=4, E=5
Reverse-scored item: A=5, B=4, C=3, D=2, E=1 — flip the values in score_value, not just the reverse_scored flag.
For text_score items: provide a rubric in the scoring_instructions column with labels and criteria. The LLM will assign a rubric label, which maps to a score via score_value.
Sheet 3: personas
Defines the dimensions for generating virtual respondents via Latin Hypercube Sampling (LHS).
Format
Wide-format table:
- Row 1 (header): dimension names (e.g., age, gender, region, occupation)
- Rows 2+: candidate values for each dimension, listed downward
Example:
| age | gender | region | occupation |
|---|---|---|---|
| 18-23 | Male | Urban | Student |
| 24-29 | Female | Suburban | Engineer |
| 30-35 | Rural | Teacher | |
| 36-41 | Medical staff |
The platform samples from these dimensions using LHS to create diverse, non-repeating persona combinations. Empty cells are skipped for that dimension.
Sheet 4: experiment_config
Runtime settings in a two-column key / value format.
Required fields
| Key | Description | Example |
|---|---|---|
sample_size |
Number of personas to run in the main flow | 20 |
repeats |
Times each persona answers each question | 1 |
Model selection
| Key | Default | Description |
|---|---|---|
model |
google/gemini-3.1-flash-lite-preview |
LLM used for the main flow and question-order monitoring. Optional. If set, it must be one of the platform's supported OpenRouter model ids (see below) — anything else is rejected at upload time with a clear error. If omitted, the platform uses its default model. |
Supported values (budget.SUPPORTED_MODELS, 13 total):
google/gemini-3.1-flash-lite-preview (default)
openai/gpt-4o
openai/gpt-4o-mini
openai/gpt-4.1-mini
openai/gpt-4.1-nano
openai/o4-mini
google/gemini-2.5-flash
google/gemini-2.5-pro
x-ai/grok-3-mini
x-ai/grok-4-fast
deepseek/deepseek-chat-v3-0324
meta-llama/llama-4-scout
meta-llama/llama-4-maverick
A workbook written for an older version of this doc may set model to an
arbitrary OpenRouter id (e.g. a value not on this list) — that now fails
validation on upload. Pick one of the ids above, or delete the model row
entirely to use the default.
The cost estimate, the actual respondent calls, and billing all resolve
model through the same whitelist check, so the price you're quoted and the
model that actually runs cannot disagree.
Note:
persona_narrative_modelandtext_score_model(below) are separate settings and are not checked against this whitelist — any value you put there is used as-is.
Persona generation fields
| Key | Default | Description |
|---|---|---|
persona_seed |
42 |
Random seed for reproducible persona generation |
persona_oversample_mult |
2.0 |
LHS oversampling factor. A value of 2.0 means each generation round draws 2x the remaining target count as candidates, then de-duplicates. Increase this if your dimension space has limited unique combinations. |
persona_template |
auto | Python format string for persona text. Use dimension names as {placeholders}. Example: A {gender} aged {age}, from {region}, working as a {occupation}. |
Note:
persona_countis no longer needed. The platform automatically generatesmax(sample_size, item_effect_sample_size)personas.
Question-order monitoring fields
| Key | Default | Description |
|---|---|---|
item_effect_enabled |
false |
Set to true to run question-order monitoring after the main flow |
item_effect_sample_size |
= sample_size | Number of personas for monitoring (must be <= sample_size) |
Text scoring fields
| Key | Default | Description |
|---|---|---|
text_score_enabled |
true |
Enable LLM-based rubric scoring for text_score items |
Note: The text scoring model defaults to the same model specified in
model. You do not need to configure it separately.
Respondent-call options (opt-in — these change your cost)
| Key | Type | Range | Default | Cost impact |
|---|---|---|---|---|
temperature |
number | 0.0–2.0 |
0.2 |
None. Higher = more varied, less deterministic answers. |
enable_thinking |
boolean | true / false |
false |
≈ x8 per-call cost. The model reasons before answering; reasoning tokens are billed as output tokens. |
enable_websearch |
boolean | true / false |
false |
≈ x40 per-call cost. Adds a flat $0.02 per respondent call — a per-request plugin fee, not a token charge. |
Booleans accept the usual spellings: true/false, TRUE/FALSE, yes/no,
1/0. Anything else is rejected with a validation error.
These options apply to the main flow and to question-order monitoring. They do not affect persona biographies or text scoring, which keep their own fixed settings. Leave all three out and the run behaves and prices exactly as before.
Cost warning:
enable_websearchis the expensive one. A questionnaire answer is only a handful of tokens, so a flat $0.02 search fee per call dominates everything else — 20 personas x 6 questions is already $2.40 of search fees before any tokens are counted. Turn it on only when the answers genuinely need current, real-world information. The cost estimate shown before you start the run includes these surcharges; unused frozen credits are released when the run ends.
Reserved fields — not usable yet
| Key | Default | Status |
|---|---|---|
answering_mode |
independent |
Reserved for an in-development "contextual" mode where a respondent answers the whole questionnaire in one running conversation instead of one call per question. Any value other than independent is currently rejected at upload ("contextual answering mode is not yet available"). Leave this out. |
questions_per_call |
1 |
Reserved for contextual mode (how many questions go into one turn). Has no effect while answering_mode is (necessarily) independent. |
context_max_turns |
0 |
Reserved for contextual mode (how much conversation history to keep). Has no effect in independent mode. |
question_order |
fixed |
Reserved for contextual mode. shuffle gives each respondent their own randomized question order, but that logic only runs inside the contextual code path — in the current independent-only mode, questions always run in display_order regardless of this setting. |
Do not set these fields in a production workbook. They exist in the schema so the upcoming contextual mode has somewhere to read its config from, not because they do anything today.
Fields you should NOT put in the workbook
The following are configured via environment variables (.env file), not the workbook:
base_url— defaults tohttps://openrouter.ai/api/v1api_key_env— defaults toOPENROUTER_API_KEY
Common Mistakes
- Renaming a sheet — keep exact names:
questionnaire,codebook,personas,experiment_config - Changing header columns — headers must match exactly
- Forgetting codebook rows — every
question_idinquestionnairemust appear incodebook - Reverse scoring — setting
reverse_scored=truewithout reversing the actualscore_valuenumbers - Text questions with option rows — text items should have exactly one row with no option columns
- Duplicate display_order — each question must have a unique
display_ordervalue - Putting API keys in the workbook — use the
.envfile instead
Quick Start
- Download
survey_workbook_example.xlsx - Copy it and rename for your study
- Replace
questionnaireandcodebookcontent with your items - Adjust
personasdimensions for your target population - Set
sample_sizeandrepeatsinexperiment_config; optionally setmodelto one of the supported ids above (otherwise the default is used) - Upload to EZPsych and click "Start Run"
AI-Assisted Workbook Creation
If you want an AI assistant to help create a workbook, paste this prompt:
Help me create an EZPsych workbook (.xlsx) with these four sheets:
1. questionnaire — my items with question_id, display_order, item_type, question_text, and options
2. codebook — scoring rules matching each question_id, with dimension, scoring_type, and score_value
3. personas — wide-format LHS dimensions (one column per dimension, values downward)
4. experiment_config — key/value pairs: sample_size, repeats, and any optional settings (model is optional too — only set it if you need something other than the platform default, and only from the supported model list)
Rules:
- Every question_id in questionnaire must have matching rows in codebook
- For choice items, response_value must match option_id
- Reverse-scored items need reversed score_value numbers (not just the flag)
- Text items get one row with no option columns
- If you set `model`, it must be one of the platform's supported OpenRouter ids — do not invent one
- Do NOT include api_key_env or base_url in experiment_config
- Do NOT set answering_mode to anything other than independent (or leave it out) — contextual mode is not available yet
My questionnaire: [describe your items here]
Experiment JSON spec
The JSON schema accepted by POST /api/v1/experiments/validate and POST /api/v1/experiments. — raw markdown: experiment_json_spec.md
EZPsych JSON Experiment Spec
This document describes the JSON format accepted by the EZPsych v1 API. It is the
programmatic equivalent of the Excel workbook described in
docs/workbook_creation_prompt.md — the platform converts a JSON spec into that
exact 4-sheet workbook before running anything, so the semantics are identical,
with one exception: experiment_config.model (see
"Ignored keys" below) — the JSON path always runs the
platform default model, while the workbook path lets you pick from a whitelist.
Written for AI agents: every field name, type, enum and constraint is listed explicitly, together with the mistakes that most often produce validation errors.
Top-level shape
{
"version": 1,
"questionnaire": [ ... ],
"codebook": [ ... ],
"personas": [ ... ],
"experiment_config": { ... }
}
| Key | Type | Required | Maps to sheet |
|---|---|---|---|
version |
integer | No | — (metadata, ignored) |
name / title / description / notes |
string | No | — (metadata, ignored) |
questionnaire |
array of objects | Yes | questionnaire |
codebook |
array of objects | Yes | codebook |
personas |
array of objects | Yes | personas |
experiment_config |
object | Yes | experiment_config |
Any other top-level key produces a warning and is ignored.
Size limits
Specs are checked against these caps before anything else runs, so a spec that breaks one gets a fast validation error and no cost estimate. They sit far above any realistic questionnaire:
| Limit | Maximum |
|---|---|
| Request body | 512 KB |
questionnaire questions |
400 |
options per question |
26 |
personas dimensions |
40 |
values per persona dimension |
60 |
experiment_config.sample_size |
5000 |
experiment_config.repeats |
10 |
experiment_config.item_effect_sample_size |
5000 |
An oversized request body is rejected with 413 and
{"error": "spec_too_large"}; the structural caps come back as ordinary
{"path": ..., "message": ...} validation errors.
questionnaire — the items respondents see
One object per question (not per option — options are nested, unlike the Excel layout).
| Field | Type | Required | Notes |
|---|---|---|---|
question_id |
string | Yes | Stable unique id, e.g. "EXT1". Must be unique across the array and is the join key with codebook. |
item_type |
string enum | Yes | "single_choice" | "multiple_choice" | "text" |
question_text |
string | Yes | The question stem. Must be non-empty. |
display_order |
integer ≥ 1 | No | Must be unique. If no item in the array declares it, order is auto-assigned from array position. If any item declares it, all items must. |
block_id |
string | No | Group label (e.g. "EXT"). Defaults to question_id with a warning. |
instrument |
string | No | Scale name (e.g. "BFI"). Defaults to "custom" with a warning. |
required |
boolean | No | Default true. Metadata only. |
allow_multiple |
boolean | No | Default: true for multiple_choice, false otherwise. Must match item_type. |
min_selections |
integer ≥ 0 | No | Default 1. |
max_selections |
integer ≥ 1 | No | Choice items only. Forced to 1 for single_choice. For multiple_choice must be ≥ min_selections; omit for "no limit". |
options |
array of objects | Choice items only | See below. Forbidden on text items. |
score_min |
number | Text items only | Lowest possible score for the item. |
score_max |
number | Text items only | Highest possible score. Must be > score_min. |
response_max_tokens |
integer ≥ 1 | Text items only | Output token cap for the free-text answer. |
options entries (choice items)
| Field | Type | Required | Notes |
|---|---|---|---|
option_id |
string | Yes | Short id, e.g. "A". Unique within the question. This is the key used in codebook.score_map. |
option_text |
string | Yes | Label shown to the respondent. |
option_order |
integer ≥ 1 | No | Unique within the question. Auto-assigned from array position if no option in that question declares it. |
Constraints:
single_choicerequires ≥ 1 option,multiple_choicerequires ≥ 2.textitems must not defineoptions,allow_multiple: true, ormax_selections.- Only
textitems may definescore_min,score_max,response_max_tokens.
codebook — scoring rules
Exactly one object per question_id. Every question in questionnaire must
have an entry, and every entry must reference an existing question.
| Field | Type | Required | Notes |
|---|---|---|---|
question_id |
string | Yes | Must match a questionnaire entry. No duplicates. |
scoring_type |
string enum | Yes | "choice_score" | "text_score" | "none" |
dimension |
string | Yes when scored | Construct name, e.g. "Extraversion". Items sharing a dimension are aggregated together. |
score_aggregation |
string enum | No | "mean" | "sum" | "max" | "min". Default "mean". Forced to "none" when scoring_type is "none". |
reverse_scored |
boolean | No | Default false. Metadata only — you must reverse the numbers in score_map yourself. |
score_map |
object | Yes when scored | Maps a response label to a numeric score. |
score_descriptions |
object | Required for text_score |
Maps the same labels to rubric criteria text. |
scoring_instructions |
string | text_score only |
Instructions handed to the LLM rubric scorer. |
score_map semantics by scoring_type
choice_score— keys areoption_idvalues. The key set must be exactly the question's option ids: no missing options, no unknown keys. Values are numbers ({"A": 1, "B": 2, "C": 3, "D": 4, "E": 5}).text_score— keys are free-form rubric labels (e.g."LEVEL_1"). Every label needs a non-emptyscore_descriptionsentry, and every value must fall inside the item's[score_min, score_max]range. Only valid ontextitems.none—score_map,score_descriptionsandscoring_instructionsmust all be absent. The question is asked but not scored.
Hard rules enforced by the validator:
item_type: "text"⇔scoring_type: "text_score"(both directions).- Choice items may only use
"choice_score"or"none". - Scored questions cannot use
score_aggregation: "none".
Understanding dimensions
dimension groups items into a measured construct. Items sharing a dimension are
combined into a single per-persona score using score_aggregation, and the
analysis report computes Cronbach's alpha, inter-item correlations and item
discrimination per dimension.
- A dimension needs at least 2 scored items for reliability analysis — a single-item dimension produces a warning.
- Different
item_type/scoring_typevalues may share a dimension. When their score ranges differ (e.g. 0/1 vs 1–5), the platform normalizes each item to[0, 1]before aggregating. - Use one
score_aggregationper dimension (mixing them produces a warning).
Reverse scoring example: a normal item is A=1 … E=5; the reverse-keyed item is
A=5 … E=1. Flip the numbers in score_map — setting reverse_scored: true
alone changes nothing.
personas — virtual respondent dimensions
An array of dimension definitions. The platform draws non-repeating combinations via Latin Hypercube Sampling.
| Field | Type | Required | Notes |
|---|---|---|---|
name |
string | Yes | Dimension name, e.g. "Age". Unique across the array. |
values |
array of strings | Yes | Candidate values. Non-empty, no duplicates within the dimension. |
"personas": [
{"name": "Age", "values": ["18-24", "25-34", "35-44", "45-54"]},
{"name": "Gender", "values": ["Male", "Female"]}
]
Constraints:
namemay not be one of the reserved column names:persona_id,id,text,description,persona_text,dimension,field,name,placeholder,field_key,template_key,value,category,level,option,values,categories,levels,options,weight,weights.- The product of all
valueslengths must be ≥ the number of personas needed (max(sample_size, item_effect_sample_size)), otherwise LHS cannot produce enough unique respondents. - Each dimension gets a
persona_templateplaceholder derived fromname: lowercased, with every run of non-alphanumeric characters replaced by_("Education level"→{education_level}).
experiment_config — runtime settings
A flat JSON object (no nesting). Keys are case-insensitive and normalized to lowercase.
Required
| Key | Type | Notes |
|---|---|---|
sample_size |
integer ≥ 1 | Number of personas in the main flow. |
Common optional keys
| Key | Type | Default | Notes |
|---|---|---|---|
repeats |
integer ≥ 1 | 1 |
How many times each persona answers each question. |
persona_seed |
integer | 42 |
Seed for reproducible persona generation. |
persona_oversample_mult |
number > 0 | 2.0 |
LHS oversampling factor. Raise it when the dimension space is small. |
persona_template |
string | auto | Python format string, e.g. "A {gender} aged {age}, working as a {occupation}." Placeholders must exist (see above). |
persona_count |
integer ≥ 1 | auto | Overrides the derived persona count. Normally omit it. |
item_effect_enabled |
boolean | false |
Run question-order monitoring after the main flow. |
item_effect_sample_size |
integer ≥ 2 | sample_size |
Only when item_effect_enabled is true; must be ≤ sample_size. |
text_score_enabled |
boolean | true |
LLM rubric scoring for text_score items. |
report_enabled |
boolean | true |
Generate the analysis report. |
report_narrative_enabled |
boolean | true |
LLM-written narrative sections in the report. |
persona_narrative_enabled |
boolean | true |
LLM-written persona biographies. |
Reserved keys — accepted, but not usable yet
These four are recognized (type-checked, no "unknown key" warning) but exist for
an in-development "contextual answering" mode that batches multiple questions
into one running conversation per respondent instead of one call per question.
answering_mode other than "independent" is currently rejected —
"contextual answering mode is not yet available" — so do not set it. The other
three only take effect inside that mode's code path and are inert under the
independent mode this API currently runs:
| Key | Type | Default | Notes |
|---|---|---|---|
answering_mode |
string enum | "independent" |
Only "independent" is currently accepted; any other value is a validation error. |
questions_per_call |
integer ≥ 1 | 1 |
No effect while answering_mode is independent. |
context_max_turns |
integer ≥ 0 | 0 |
No effect while answering_mode is independent. |
question_order |
string enum | "fixed" |
"shuffle" only randomizes order inside the contextual code path. Under independent mode, questions always run in display_order regardless of this value. |
answering_mode="contextual" is also permanently incompatible with
item_effect_enabled=true, even once contextual mode ships: question-order
monitoring assumes each answer is independent of question order, which
contextual mode removes. The documented alternative once available will be
question_order="shuffle".
Respondent-call options (opt-in, cost impact)
These three keys change how every simulated respondent answers. They apply to the main flow and to question-order monitoring; persona-narrative and text-scoring calls are unaffected.
| Key | Type | Range | Default | Cost impact |
|---|---|---|---|---|
temperature |
number | 0.0–2.0 |
0.2 |
None. Higher values give more varied answers. |
enable_thinking |
boolean | true / false |
false |
≈ ×8 per-call cost. Reasoning tokens are billed as output tokens. |
enable_websearch |
boolean | true / false |
false |
≈ ×40 per-call cost — a flat $0.02 per respondent call on top of tokens. |
Notes:
enable_thinkingrequestsreasoning: {"enabled": true, "effort": "medium"}from OpenRouter. The reasoning trace is excluded from the response, so answers parse exactly as before, but the tokens are still billed. The runner also raisesmax_tokens×8 so reasoning cannot truncate the answer.enable_websearchattaches OpenRouter's web plugin (plugins: [{"id": "web"}]). The pinned model id is never changed. The plugin fee is charged per request, not per token, which is why it dwarfs the token cost of a short questionnaire answer.- Both flags are estimated conservatively up front. Unused frozen credits are released when the job finishes, so an over-estimate costs you nothing.
- A temperature outside
0.0–2.0, or a non-boolean flag value, is a validation error atexperiment_config.<key>.
Rejected keys (validation error)
Credentials and runtime paths are configured server-side and may never appear in
a spec: api_key_env, api_key_value, base_url, text_score_api_key_env,
text_score_api_key_value, text_score_base_url, outdir, plan_csv,
plan_csv_encoding, retry_from_outdir, survey_workbook_file,
workbook_questionnaire_sheet, workbook_codebook_sheet,
workbook_personas_sheet, workbook_config_sheet.
Ignored keys (warning)
model and text_score_model are pinned for this JSON API intake path
specifically: supplying either key produces a warning and the value is
discarded, and every run submitted through POST /api/v1/experiments uses
budget.DEFAULT_MODEL (currently google/gemini-3.1-flash-lite-preview).
This is different from the Excel-workbook intake path (see
workbook_creation_prompt.md),
where model is honored and can be set to any id in the platform's
supported-model whitelist (budget.SUPPORTED_MODELS) — that path validates
the value at upload instead of discarding it. If your integration needs a
non-default model, use the workbook path; the JSON API does not currently
expose model selection.
Unknown keys produce a warning and are forwarded to the runner as-is.
API endpoints
POST /api/v1/experiments/validate
Public, no authentication, rate limited (5/min, 30/hour), max body 512 KB, plus the structural caps listed under Size limits. Runs zero LLM calls.
Request body: the spec object. Response 200:
{
"valid": true,
"errors": [],
"warnings": [{"path": "questionnaire[0].instrument", "message": "missing, defaulted to 'custom'"}],
"estimate": {
"question_count": 6,
"persona_count": 20,
"total_calls": 448,
"estimated_price_usd": 0.2355,
"estimated_credits": 2355,
"feature_costs": {
"temperature": 0.2,
"enable_thinking": false,
"thinking_multiplier": 1,
"thinking_extra_usd": 0.0,
"enable_websearch": false,
"websearch_per_call_usd": 0.0,
"websearch_calls": 0,
"websearch_usd": 0.0
}
}
}
feature_costs breaks out what enable_thinking and enable_websearch added to
the price, so you can show the surcharge separately. *_usd fields are raw
provider cost; the matching priced_*_usd fields are those amounts after markup
and safety margin.
estimate is null when valid is false. Errors and warnings are always
{"path": ..., "message": ...} objects, where path is a JSON pointer-ish
expression such as "codebook[3].score_map.A" or
"questionnaire[1].options[2].option_id". Validation accumulates as many
problems as possible in one pass — fix them all, then re-submit.
POST /api/v1/experiments
API-key authenticated (Authorization: Bearer <key>), max body 512 KB, same
structural caps as validate. Validates the spec, converts it to a workbook,
freezes credits and enqueues the run.
Response 201:
{"job_id": "20260727T061200Z-1a2b3c4d", "status": "queued", "estimated_credits": 2355, "warnings": []}
Errors:
| Status | error |
Meaning |
|---|---|---|
400 |
invalid_spec |
Validation failed; errors lists the problems. |
400 |
invalid_json |
Body is not valid JSON. |
401 |
— | Missing or invalid API key. |
402 |
insufficient_credits |
Includes needed, have and the full estimate. |
402 |
monthly_cap_exceeded |
The API key's monthly cap would be exceeded; includes monthly_cap_credits, used_this_month and needed. |
403 |
email_not_verified |
Verify the account's email first. |
413 |
spec_too_large |
Body exceeds 2 MB. |
503 |
platform_maintenance |
Provider balance too low to accept new runs. |
Common mistakes
- Flattening options — the JSON format nests
optionsinside each question. Do not repeat a question object once per option (that is the Excel layout). score_mapkeys that are notoption_ids — forchoice_score, the keys must exactly match the question's option ids, all of them.- Reverse scoring the flag only — flip the numbers in
score_map;reverse_scoredis metadata. - Text items with options or choice items with
score_min/score_max. text_scorewithoutscore_descriptions— every rubric label needs criteria text, and every score must be inside[score_min, score_max].- Missing codebook entries — every
question_idneeds exactly one entry. - Duplicate
display_order— or declaring it on some questions but not all. - Too few persona combinations —
sample_sizeabove the product of the dimension value counts. persona_templateplaceholders that do not exist — use the normalized, lowercased dimension names.- Putting API keys or paths in
experiment_config.
Full runnable example
A small Big-Five-style spec with two dimensions, one reverse-keyed item, one multiple-choice item and one LLM-scored free-text item.
{
"version": 1,
"name": "Mini Big Five pilot",
"questionnaire": [
{
"question_id": "EXT1",
"display_order": 1,
"block_id": "EXT",
"instrument": "MiniBFI",
"item_type": "single_choice",
"question_text": "I am the life of the party.",
"options": [
{"option_id": "A", "option_text": "Strongly disagree"},
{"option_id": "B", "option_text": "Disagree"},
{"option_id": "C", "option_text": "Neither agree nor disagree"},
{"option_id": "D", "option_text": "Agree"},
{"option_id": "E", "option_text": "Strongly agree"}
]
},
{
"question_id": "EXT2",
"display_order": 2,
"block_id": "EXT",
"instrument": "MiniBFI",
"item_type": "single_choice",
"question_text": "I don't talk a lot.",
"options": [
{"option_id": "A", "option_text": "Strongly disagree"},
{"option_id": "B", "option_text": "Disagree"},
{"option_id": "C", "option_text": "Neither agree nor disagree"},
{"option_id": "D", "option_text": "Agree"},
{"option_id": "E", "option_text": "Strongly agree"}
]
},
{
"question_id": "CON1",
"display_order": 3,
"block_id": "CON",
"instrument": "MiniBFI",
"item_type": "single_choice",
"question_text": "I am always prepared.",
"options": [
{"option_id": "A", "option_text": "Strongly disagree"},
{"option_id": "B", "option_text": "Disagree"},
{"option_id": "C", "option_text": "Neither agree nor disagree"},
{"option_id": "D", "option_text": "Agree"},
{"option_id": "E", "option_text": "Strongly agree"}
]
},
{
"question_id": "CON2",
"display_order": 4,
"block_id": "CON",
"instrument": "MiniBFI",
"item_type": "single_choice",
"question_text": "I leave my belongings around.",
"options": [
{"option_id": "A", "option_text": "Strongly disagree"},
{"option_id": "B", "option_text": "Disagree"},
{"option_id": "C", "option_text": "Neither agree nor disagree"},
{"option_id": "D", "option_text": "Agree"},
{"option_id": "E", "option_text": "Strongly agree"}
]
},
{
"question_id": "HAB1",
"display_order": 5,
"block_id": "HAB",
"instrument": "MiniBFI",
"item_type": "multiple_choice",
"question_text": "Which of the following do you do in a typical week? (Select all that apply)",
"min_selections": 1,
"max_selections": 4,
"options": [
{"option_id": "A", "option_text": "Plan the week in advance"},
{"option_id": "B", "option_text": "Keep a written to-do list"},
{"option_id": "C", "option_text": "Tidy my workspace"},
{"option_id": "D", "option_text": "Review what I finished"}
]
},
{
"question_id": "REF1",
"display_order": 6,
"block_id": "REF",
"instrument": "MiniBFI",
"item_type": "text",
"question_text": "Describe how you organised your last busy week. What did you plan, and what actually happened?",
"score_min": 1,
"score_max": 5,
"response_max_tokens": 200
}
],
"codebook": [
{
"question_id": "EXT1",
"dimension": "Extraversion",
"scoring_type": "choice_score",
"score_aggregation": "mean",
"reverse_scored": false,
"score_map": {"A": 1, "B": 2, "C": 3, "D": 4, "E": 5}
},
{
"question_id": "EXT2",
"dimension": "Extraversion",
"scoring_type": "choice_score",
"score_aggregation": "mean",
"reverse_scored": true,
"score_map": {"A": 5, "B": 4, "C": 3, "D": 2, "E": 1}
},
{
"question_id": "CON1",
"dimension": "Conscientiousness",
"scoring_type": "choice_score",
"score_aggregation": "mean",
"reverse_scored": false,
"score_map": {"A": 1, "B": 2, "C": 3, "D": 4, "E": 5}
},
{
"question_id": "CON2",
"dimension": "Conscientiousness",
"scoring_type": "choice_score",
"score_aggregation": "mean",
"reverse_scored": true,
"score_map": {"A": 5, "B": 4, "C": 3, "D": 2, "E": 1}
},
{
"question_id": "HAB1",
"dimension": "Conscientiousness",
"scoring_type": "choice_score",
"score_aggregation": "mean",
"reverse_scored": false,
"score_map": {"A": 1, "B": 1, "C": 1, "D": 1}
},
{
"question_id": "REF1",
"dimension": "Conscientiousness",
"scoring_type": "text_score",
"score_aggregation": "mean",
"reverse_scored": false,
"score_map": {"LEVEL_1": 1, "LEVEL_2": 2, "LEVEL_3": 3, "LEVEL_4": 4, "LEVEL_5": 5},
"score_descriptions": {
"LEVEL_1": "No planning described; the answer is vague or avoidant.",
"LEVEL_2": "Mentions intentions but no concrete plan or follow-through.",
"LEVEL_3": "Describes a basic plan with limited detail on execution.",
"LEVEL_4": "Clear plan, concrete actions, and some reflection on results.",
"LEVEL_5": "Detailed plan, multiple strategies, adjustments, and reflection on outcomes."
},
"scoring_instructions": "Score the answer on a 1 to 5 scale for organisational behaviour. Judge whether the response shows advance planning, concrete follow-through, and reflection on the outcome. Return only the best matching rubric label."
}
],
"personas": [
{"name": "Age", "values": ["18-24", "25-34", "35-44", "45-54", "55-64"]},
{"name": "Gender", "values": ["Male", "Female"]},
{"name": "Occupation", "values": ["Student", "Software engineer", "Teacher", "Healthcare worker", "Retired"]},
{"name": "Education", "values": ["High school", "Bachelor's degree", "Master's degree"]}
],
"experiment_config": {
"sample_size": 20,
"repeats": 1,
"persona_seed": 42,
"persona_template": "A {gender} aged {age}, working as a {occupation}. Education: {education}.",
"item_effect_enabled": true,
"item_effect_sample_size": 10,
"text_score_enabled": true
}
}
Validate it with:
curl -X POST https://ezpsy.ai/api/v1/experiments/validate \
-H "Content-Type: application/json" \
--data @spec.json
Then submit it with:
curl -X POST https://ezpsy.ai/api/v1/experiments \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $EZPSYCH_API_KEY" \
--data @spec.json
API reference
Full REST API: auth, endpoints, request/response examples, error semantics, rate limits. — raw markdown: api_reference.md
EZPsych API Reference (v1)
This document is written for AI agents (and the humans supervising them) who want to build and run an EZPsych experiment entirely through the API, without touching the web dashboard.
If you are an agent: start by reading /docs/agent for an overview,
then workbook_creation_prompt.md and
experiment_json_spec.md to learn the shape
of an experiment spec. This page documents the HTTP contract for submitting and
managing runs.
1. Authentication
All endpoints except POST /api/v1/experiments/validate require a Bearer API key:
Authorization: Bearer ezp_<key>
How to get a key: API keys belong to a human account, not to an agent. Ask your human operator to:
- Register an EZPsych account and verify their email at
https://ezpsy.ai/register. - Sign in and create a key at
https://ezpsy.ai/settings/api-keys. - Give you the key (
ezp_...) as a secret — treat it like a password, never log it or send it to a third party.
Keys act on behalf of the owning account: usage spends that account's credits and runs are listed/managed under that account.
Missing or invalid keys return 401 Unauthorized:
{ "error": "unauthorized", "message": "Missing or invalid API key." }
2. Pricing & credits
- EZPsych runs on prepaid credits:
1 USD = 10,000 credits. POST /api/v1/experiments/validateis free and makes no LLM calls — iterate on your spec until it is valid at no cost.POST /api/v1/experimentsreturns anestimated_creditsfigure before any credits are spent. The estimate includes a 2x service markup and a safety margin.- Once running, credits are deducted in real time based on actual LLM usage x 2 (the same service markup applied up front). Any unused frozen credits are released back to the account when the run finishes, is stopped, or fails.
- See
GET /api/v1/mefor the current balance and/pricingfor full pricing details.
Respondent-call options that change the price
Three experiment_config keys change what each simulated respondent call costs.
All three are opt-in and default to the cheap setting, so a spec that omits them
prices exactly as before.
| Key | Type | Range | Default | Cost impact |
|---|---|---|---|---|
temperature |
number | 0.0–2.0 |
0.2 |
None. |
enable_thinking |
boolean | true / false |
false |
≈ x8 per-call cost (reasoning tokens are billed as output tokens). |
enable_websearch |
boolean | true / false |
false |
≈ x40 per-call cost — a flat $0.02 per respondent call, charged per request rather than per token. |
They apply to the main flow and to question-order monitoring; persona-narrative and text-scoring calls are unaffected. Turning both on multiplies the two effects together. Estimates for these features are deliberately conservative — the extra frozen credits are released when the run ends.
The estimate object breaks the surcharges out under feature_costs so you can
show your human operator what the options are adding:
"feature_costs": {
"temperature": 0.2,
"enable_thinking": true,
"thinking_multiplier": 8,
"thinking_extra_usd": 0.3326,
"priced_thinking_extra_usd": 0.7983,
"enable_websearch": true,
"websearch_per_call_usd": 0.02,
"websearch_calls": 360,
"websearch_usd": 7.2,
"priced_websearch_usd": 17.28
}
*_usd fields are raw provider cost; priced_*_usd fields are the same amounts
after markup and safety margin, i.e. their share of estimated_price_usd.
3. Endpoints
POST /api/v1/experiments/validate
Public. No auth required. Rate-limited (see Rate limits) and size-capped (see Size limits).
Validate an experiment spec before spending anything. Call this repeatedly while you fix issues — it is free and safe to call as often as the rate limit allows.
Request body — a JSON experiment spec, documented in
experiment_json_spec.md:
{
"questionnaire": [ ... ],
"codebook": [ ... ],
"personas": { ... },
"experiment_config": { "sample_size": 20, "repeats": 1 }
}
Note: experiment_config.model is accepted but ignored on this JSON API path
— it always runs the platform default model. Model selection is only available
through the Excel-workbook upload path; see
experiment_json_spec.md
for details.
Response — 200 OK whether or not the spec is valid; check the valid field:
{
"valid": false,
"errors": [
{ "path": "codebook[3].response_value", "message": "response_value 'F' does not match any option_id for question_id 'EXT2'." },
{ "path": "experiment_config.sample_size", "message": "sample_size must be a positive integer." }
],
"warnings": [
{ "path": "codebook", "message": "Dimension 'Resilience' has only 1 scored item — Cronbach's alpha requires at least 2." }
],
"estimate": null
}
When valid is true, estimate is populated with the same cost-estimate shape
returned by POST /api/v1/experiments (see below), so you can show your human
operator the cost before they hand you a key to submit:
{
"valid": true,
"errors": [],
"warnings": [],
"estimate": {
"estimated_credits": 4200,
"estimated_price_usd": 0.42,
"model": "google/gemini-3.1-flash-lite-preview",
"sample_size": 20,
"total_calls": 180
}
}
POST /api/v1/experiments
Auth: Bearer. Rate-limited and size-capped (see Size limits).
Submit a validated spec and start the run. This spends the owner's credits (after freezing an estimate up front — see Pricing & credits).
Request body: the same experiment spec shape accepted by validate.
Response — 202 Accepted:
{
"job_id": "20260727T093000Z-a1b2c3d4",
"status": "queued",
"estimated_credits": 4200
}
Error responses:
| Status | Body | Meaning |
|---|---|---|
400 |
{"error": "invalid_spec", "errors": [...]} |
Spec failed validation — call validate first and fix all errors. |
402 |
{"error": "insufficient_credits", "needed": 4200, "have": 1000} |
Owner does not have enough credits. |
402 |
{"error": "monthly_cap_exceeded", "monthly_cap_credits": 50000, "used_this_month": 48000, "needed": 4200} |
This key's monthly cap would be exceeded. |
503 |
{"error": "platform_maintenance", "message": "..."} |
Upstream LLM provider balance too low; retry later. |
GET /api/v1/me
Auth: Bearer.
{ "username": "researcher_jane", "credits": 18500 }
GET /api/v1/experiments
Auth: Bearer. List all runs owned by the key's account, most recent first.
Optional query params: q (search by filename/job_id), status (filter by status).
{
"experiments": [
{
"job_id": "20260727T093000Z-a1b2c3d4",
"status": "running",
"phase": "main_flow",
"created_at": "2026-07-27T09:30:00+00:00",
"overall_progress": { "percent": 42.0 }
}
]
}
GET /api/v1/experiments/<job_id>
Auth: Bearer. Full status for one run, including phase progress.
{
"job_id": "20260727T093000Z-a1b2c3d4",
"status": "running",
"phase": "main_flow",
"message": "Main flow is running.",
"main_flow": { "status": "running", "completed": 76, "total": 180, "percent": 42.2 },
"question_order_monitoring": { "enabled": false, "status": "skipped", "percent": 100.0 },
"overall_progress": { "percent": 42.0 },
"download_ready": false
}
phase progresses through: queued -> validating -> main_flow ->
saving_main_outputs -> question_order_monitoring (if enabled) -> packaging ->
completed (or failed / stopped / paused at any point).
GET /api/v1/experiments/<job_id>/result
Auth: Bearer. Returns 404 if the job has no results yet, 409 if it exists but
isn't finished. On success, streams the result ZIP
(application/zip) containing:
- Raw per-response records (CSV)
summary_scores.csv— per-persona, per-dimension scores- Persona definitions
- An HTML/PDF psychometric report (reliability, item analysis, order-effect results if question-order monitoring was enabled)
POST /api/v1/experiments/<job_id>/pause
Auth: Bearer. Only valid while status == "running". Checkpoints progress so the
run can be resumed later; unused credits stay frozen against this job.
{ "pausing": true, "job_id": "20260727T093000Z-a1b2c3d4" }
POST /api/v1/experiments/<job_id>/stop
Auth: Bearer. Only valid while status is running, pausing, or paused.
Cannot be resumed — unused (frozen) credits are released back to the account.
{ "stopping": true, "job_id": "20260727T093000Z-a1b2c3d4" }
POST /api/v1/experiments/<job_id>/resume
Auth: Bearer. Only valid while status == "paused". Requires the account to still
have (or have frozen) enough credits to continue.
{ "resumed": true, "job_id": "20260727T093000Z-a1b2c3d4" }
4. Full walkthrough (curl)
# 0. Your human operator gives you a key as an environment variable.
export EZP_KEY="ezp_xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
# 1. Validate your draft spec. No auth, no cost — iterate until "valid": true.
curl -s -X POST https://ezpsy.ai/api/v1/experiments/validate \
-H "Content-Type: application/json" \
--data @experiment.json | tee validate_result.json
# -> {"valid": false, "errors": [{"path": "...", "message": "..."}], ...}
# Fix experiment.json based on each error's "path"/"message", then re-run step 1.
# 2. Once "valid": true, review "estimate" with your human operator, then submit.
curl -s -X POST https://ezpsy.ai/api/v1/experiments \
-H "Authorization: Bearer $EZP_KEY" \
-H "Content-Type: application/json" \
--data @experiment.json | tee submit_result.json
# -> {"job_id": "20260727T093000Z-a1b2c3d4", "status": "queued", "estimated_credits": 4200}
JOB_ID=$(python3 -c "import json;print(json.load(open('submit_result.json'))['job_id'])")
# 3. Poll status until phase reaches "completed" (or "failed").
while true; do
STATUS=$(curl -s https://ezpsy.ai/api/v1/experiments/$JOB_ID \
-H "Authorization: Bearer $EZP_KEY" | python3 -c "import json,sys;print(json.load(sys.stdin)['status'])")
echo "status=$STATUS"
[ "$STATUS" = "completed" ] || [ "$STATUS" = "failed" ] && break
sleep 5
done
# 4. Download the result archive.
curl -s -L -o results.zip https://ezpsy.ai/api/v1/experiments/$JOB_ID/result \
-H "Authorization: Bearer $EZP_KEY"
5. Error semantics
All error responses are JSON with at least an error machine-readable code and a
human-readable message:
{ "error": "insufficient_credits", "message": "Insufficient credits. Need 4200, have 1000." }
Common codes:
| HTTP status | error |
Meaning |
|---|---|---|
400 |
invalid_spec |
Spec has validation errors — see errors[], or re-run validate. |
401 |
unauthorized |
Missing/invalid/revoked API key. |
402 |
insufficient_credits |
Not enough credits; see needed/have. |
402 |
monthly_cap_exceeded |
The API key's monthly cap would be exceeded. The cap counts the estimated credits of every run the key started in the current calendar month (UTC); it resets at the start of the next month. |
404 |
not_found |
Unknown job_id, or it does not belong to this key's account. |
409 |
conflict |
Action not valid for the run's current status (e.g. pausing a queued run). |
413 |
spec_too_large |
Request body exceeds the size cap — see Size limits. |
422 |
unprocessable |
Spec parsed but could not be executed (rare — prefer relying on validate). |
429 |
rate_limited |
Too many requests — see Rate limits. |
503 |
platform_maintenance |
Upstream LLM provider temporarily unavailable; your credits are safe, retry later. |
6. Rate limits
Endpoints are rate-limited per IP / per API key (limits may change; if you receive
429 Too Many Requests, back off and retry with exponential backoff — a
Retry-After header, when present, tells you how long to wait). validate is the
endpoint you are expected to call the most while iterating on a spec; it has a
generous but finite limit, so don't poll it in a tight loop.
7. Size limits
Spec-accepting endpoints (validate and POST /api/v1/experiments) enforce these
caps. The structural caps are checked before validation and before any cost
estimate, so breaching one is cheap and fails fast:
| Limit | Maximum |
|---|---|
| Request body | 512 KB |
questionnaire questions |
400 |
options per question |
26 |
personas dimensions |
40 |
values per persona dimension |
60 |
experiment_config.sample_size |
5000 |
experiment_config.repeats |
10 |
experiment_config.item_effect_sample_size |
5000 |
An oversized body returns 413 with {"error": "spec_too_large"}. A breached
structural cap is reported as a normal validation error, so on validate it comes
back as 200 with valid: false, and on POST /api/v1/experiments as 400
invalid_spec.
See also
/docs/agent— agent documentation hubworkbook_creation_prompt.md— how to design the questionnaire/codebook/personas/configexperiment_json_spec.md— the exact JSON schema this API accepts/pricing— credit pricing details/terms— terms of service
Market research guide
Using EZPsych for concept tests, ad/copy comparisons, Van Westendorp pricing research, and questionnaire QA before fieldwork. — raw markdown: market_research_guide.md
EZPsych for Market Research
Overview
EZPsych was built for academic questionnaire pilots — LLM-simulated respondents answer your instrument so you can check reliability, item quality, and dimension structure before recruiting real participants. The same machinery (workbook format, persona sampling, scoring, psychometric report) is directly reusable for brand and market research pre-testing: concept tests, ad/copy comparisons, pricing research, and questionnaire QA for a survey firm's fieldwork instrument.
This guide covers what fits, how to design consumer personas, and a worked example.
It assumes you have already read docs/workbook_creation_prompt.md for the base
workbook format (four sheets: questionnaire, codebook, personas,
experiment_config).
What fits vs. what does not
Fits: pre-testing and directional insight
| Use case | How EZPsych supports it |
|---|---|
| Concept testing | Model purchase intent, uniqueness, believability, brand fit, and overall appeal as 5-point single_choice items, each its own dimension. Embed the concept description directly in every relevant question_text (each item is answered independently — there is no shared chat history between questions), so every rating is grounded in the same stimulus. |
| Pricing research (Van Westendorp) | Model the 4 classic price questions ("too expensive," "too cheap," "getting expensive," "a bargain") as single_choice items with price bands as options, and score_value set to each band's midpoint. This gives you a mean price point per question, which is what the Van Westendorp method needs. |
| Ad / copy A-B comparisons via the item-effect mechanism | Put each variant (e.g., AD_COPY_A, AD_COPY_B) in as its own question item sharing the same rating scale, then set item_effect_enabled=true. The order-monitoring pass re-runs a subsample with item order shuffled and reports whether scores move with presentation order. Use this to sanity-check that a copy-A-vs-copy-B difference you see is a real preference and not just a first-shown / last-shown artifact — counterbalance or discount the comparison if the order-effect check flags one. |
| Questionnaire QA before fieldwork | Run your real, human-bound survey draft through EZPsych first. Cronbach's alpha, inter-item correlations, and item discrimination per dimension surface confusing wording, redundant items, or a scale that will not hold together — cheaply, before you pay a panel provider. |
Does not fit: standing in for real fieldwork
- Claiming real market share, revenue, or adoption numbers. Simulated respondents are not a substitute for your target population; they cannot tell you what percentage of the real market will buy.
- Final go/no-go launch decisions. Use EZPsych to catch bad questions, weak differentiation, or a confusing price frame before you spend fieldwork budget — not to replace the fieldwork itself.
- Anything requiring real behavior (actual purchase, willingness to pay measured in an incentive-compatible way, real brand recall over time). LLM personas answer as language models predicting a plausible response, not as economic agents.
Positioning rule: EZPsych sells pre-testing / pilot / directional insight before real fieldwork. Never present simulated output as if it were real consumer research data, and say so explicitly in any report or deck built from these results.
Designing consumer personas
The personas sheet is a wide-format table: one column per dimension, candidate
values listed downward, sampled via Latin Hypercube Sampling (LHS) into diverse,
non-repeating persona combinations. For market research, pick dimensions that
actually predict how someone reacts to your product/price — demographic dimensions
alone are a weak proxy; usage frequency and price sensitivity usually matter more
than age or gender.
China-market consumer preset (6 dimensions)
Ready to copy into the personas sheet header row:
| city_tier | age_band | income_band | gender | category_usage_frequency | price_sensitivity |
|---|---|---|---|---|---|
| Tier 1 (Beijing / Shanghai / Guangzhou / Shenzhen) | 18-24 | Under RMB 5,000/mo | Male | Daily | Very price-sensitive |
| Tier 2 (provincial capital) | 25-34 | RMB 5,000-10,000/mo | Female | A few times a week | Somewhat price-sensitive |
| Tier 3 (prefecture-level city) | 35-44 | RMB 10,000-20,000/mo | A few times a month | Neutral | |
| Tier 4-5 (county town / rural) | 45-54 | RMB 20,000-40,000/mo | Rarely | Not very price-sensitive | |
| 55+ | Over RMB 40,000/mo | Never tried this category | Price-insensitive |
Suggested persona_template:
A {gender} aged {age_band} living in a {city_tier} city in China, household income
{income_band}. Drinks beverages in this category {category_usage_frequency} and is
{price_sensitivity} about price.
This is exactly the preset used in examples/market_research_workbook_example.xlsx.
Western-market variant
Same 6 dimensions, swapped for a US/UK-style geography and USD income bands:
| region | age_band | income_band | gender | category_usage_frequency | price_sensitivity |
|---|---|---|---|---|---|
| Urban | 18-24 | Under $30,000/yr | Male | Daily | Very price-sensitive |
| Suburban | 25-34 | $30,000-60,000/yr | Female | A few times a week | Somewhat price-sensitive |
| Rural | 35-44 | $60,000-100,000/yr | A few times a month | Neutral | |
| 45-54 | $100,000-150,000/yr | Rarely | Not very price-sensitive | ||
| 55+ | Over $150,000/yr | Never tried this category | Price-insensitive |
Suggested persona_template:
A {gender} aged {age_band} living in a {region} area, household income {income_band}.
Drinks beverages in this category {category_usage_frequency} and is
{price_sensitivity} about price.
Notes
- Column headers become
{placeholder}names inpersona_template— keep them simple identifiers (snake_case, no spaces) as shown above. - Reserved words (
dimension,field,value,weight, etc.) cannot be used as column headers — the loader treats a header containing one of those as a non-wide-format sheet and will reject it. Seedocs/workbook_creation_prompt.mdfor the full reserved-word list. - You do not need equal-length columns; short columns (e.g.,
genderwith 2 values) are fine next to long ones (e.g.,income_bandwith 5) — empty cells are skipped. - Swap in whatever dimensions actually matter for your category (e.g.,
household_size,parental_status,existing_brand_used) — the 6-dimension presets above are a starting point, not a fixed schema.
Worked example: market_research_workbook_example.xlsx
examples/market_research_workbook_example.xlsx is a full concept test for a
fictional beverage, "Qing Studio" — a sparkling, low-sugar oolong tea in a 480ml can.
It has 10 questionnaire items across three groups.
1. Concept-rating items (5 items, single_choice, 5-point scale, choice_score)
Each question repeats the concept description in its question_text (there is no
shared context between items) and asks about one attribute:
| question_id | dimension | What it measures |
|---|---|---|
PI1 |
Purchase_Intent |
"How likely are you to buy this product?" |
UNQ1 |
Uniqueness |
How different it is from what's already available |
BEL1 |
Believability |
Whether the claims (low sugar, real fruit juice) are credible |
FIT1 |
Brand_Fit |
Fit with brands the respondent would normally consider |
APPEAL1 |
Overall_Appeal |
Overall appeal, as a catch-all summary rating |
Each is scored A=1 ... E=5, its own dimension, score_aggregation=mean (single
item per dimension, so aggregation is a no-op but must still be a valid value).
2. Van Westendorp price questions (4 items, single_choice price bands, choice_score)
All four share the same 6 price bands as options (Under ¥4 through Over ¥12), with
score_value set to each band's midpoint (3, 5, 7, 9, 11, 13) so the resulting mean
per question is a usable price point:
| question_id | dimension | Classic Van Westendorp question |
|---|---|---|
VW_TOOEXP |
VW_TooExpensive |
Price at which it's too expensive to consider |
VW_TOOCHEAP |
VW_TooCheap |
Price at which you'd doubt its quality |
VW_EXP |
VW_Expensive |
Price starting to feel expensive but still acceptable |
VW_BARGAIN |
VW_Bargain |
Price that feels like a bargain |
The concept description deliberately omits any suggested price — showing a price before asking these questions anchors respondents and defeats the method.
3. Open-text purchase barrier (1 item, text, text_score)
BARRIER1 asks "what (if anything) would stop you from buying this product?" and is
scored with a simple 3-level rubric in codebook.scoring_instructions:
| Label | Score | Meaning |
|---|---|---|
LEVEL_1 |
1 | A clear deal-breaker (health/safety, price, dislike of the category) |
LEVEL_2 |
2 | A soft concern that likely wouldn't stop the purchase |
LEVEL_3 |
3 | No real barrier mentioned |
Personas and run settings
personas uses the China-market preset above (city_tier, age_band,
income_band, gender, category_usage_frequency, price_sensitivity).
experiment_config sets sample_size=200, repeats=1, report_enabled=true, and
item_effect_enabled=false (this example is a plain concept test, not an A/B copy
comparison — flip that flag to true and add a second variant item if you want to
run the ad/copy A-B pattern described above).
Validating it yourself
from survey_workbook import load_survey_workbook_bundle
from budget import estimate_from_workbook
bundle = load_survey_workbook_bundle("examples/market_research_workbook_example.xlsx")
estimate = estimate_from_workbook("examples/market_research_workbook_example.xlsx")
print(len(bundle.questions), "items;", estimate.persona_count, "personas;",
"$%.2f" % estimate.estimated_price_usd, "estimated price")
tests/test_market_example.py runs this exact round-trip plus shape assertions —
python tests/test_market_example.py should print All checks passed.
Honest-use disclaimer
Results from EZPsych — including everything produced from this workbook — are LLM-simulated directional signal, not real consumer research. Use them to catch weak concepts, confusing price framing, or a shaky questionnaire early and cheaply. Do not present simulated purchase intent, price points, or open-text themes as real market data, and do not use them as the sole basis for a launch, pricing, or budget decision. Validate anything that matters with real respondents before you act on it.