🔬 Model Lab

New run Stored runs ⚖️ Judge verdicts 🧮 Math 📊 Math runs 📄 Benchmark paper 📄 3-model paper 📄 Meta: Will Muse Cause a Spark?

⚠️ Runs call real paid APIs. All sampled questions are sent to each model in one call (so its "Total time" is the real round-trip). Web search is OFF — pure reasoning, graded objectively against the known answer. Each model shows full step-by-step working for every question (in the drill-down). If a model can't fit all the questions and drops some answers, those score wrong — a fair capability strike.

1 · Difficulty — AMC 8 + AJHSME, text-only multiple-choice (700 problems) easy (208) medium (223) hard (188) stretch (81)

Leave all ticked to draw from every difficulty.

2 · How many questions (1–50; sampled at random — every model gets the SAME set)

3 · Models — tick the ones to test

anthropic

👁 claude-fable-5 in $0.00 · out $0.00/M ~ 👁 claude-fable-5-1 in $0.00 · out $0.00/M ~ 👁 claude-haiku-4-5-20251001 in $1.00 · out $5.00/M ~ 👁 claude-opus-5 in $5.00 · out $25.00/M ~ 👁 claude-opus-5-5 in $5.00 · out $25.00/M ~ 👁 claude-sonnet-4-6 in $3.00 · out $15.00/M ~ 👁 claude-sonnet-5 in $3.00 · out $15.00/M ~ 👁 anthropic/claude-fable-5 in $10.00 · out $50.00/M 👁 anthropic/claude-fable-5.1 in $10.00 · out $50.00/M 👁 anthropic/claude-fable-5.1:batch in $5.00 · out $25.00/M 👁 anthropic/claude-fable-5:batch in $5.00 · out $25.00/M 👁 anthropic/claude-opus-5 in $5.00 · out $25.00/M 👁 anthropic/claude-opus-5.5 in $4.00 · out $20.00/M 👁 anthropic/claude-opus-5.5:batch in $2.00 · out $10.00/M 👁 anthropic/claude-opus-5:batch in $2.50 · out $12.50/M 👁 anthropic/claude-sonnet-5 in $2.00 · out $10.00/M 👁 anthropic/claude-sonnet-5:batch in $1.00 · out $5.00/M

openai

👁 openai/gpt-5.4 in $2.50 · out $15.00/M 👁 openai/gpt-5.4-mini in $0.75 · out $4.50/M 👁 openai/gpt-5.4-mini:batch in $0.38 · out $2.25/M 👁 openai/gpt-5.4-nano in $0.20 · out $1.25/M 👁 openai/gpt-5.4-nano:batch in $0.10 · out $0.62/M 👁 openai/gpt-5.4-pro in $30.00 · out $180.00/M 👁 openai/gpt-5.4-pro:batch in $15.00 · out $90.00/M 👁 openai/gpt-5.4:batch in $1.25 · out $7.50/M 👁 openai/gpt-5.5 in $5.00 · out $30.00/M 👁 openai/gpt-5.5-pro in $30.00 · out $180.00/M 👁 openai/gpt-5.5-pro:batch in $15.00 · out $90.00/M 👁 openai/gpt-5.5:batch in $2.50 · out $15.00/M 👁 openai/gpt-5.6-luna in $0.20 · out $1.20/M 👁 openai/gpt-5.6-luna-pro in $0.20 · out $1.20/M 👁 openai/gpt-5.6-luna-pro:batch in $0.10 · out $0.60/M 👁 openai/gpt-5.6-luna:batch in $0.10 · out $0.60/M 👁 openai/gpt-5.6-sol in $2.00 · out $10.00/M 👁 openai/gpt-5.6-sol-pro in $2.00 · out $10.00/M 👁 openai/gpt-5.6-sol-pro:batch in $1.00 · out $5.00/M 👁 openai/gpt-5.6-sol:batch in $1.00 · out $5.00/M 👁 openai/gpt-5.6-terra in $2.00 · out $12.00/M 👁 openai/gpt-5.6-terra-pro in $2.00 · out $12.00/M 👁 openai/gpt-5.6-terra-pro:batch in $1.00 · out $6.00/M 👁 openai/gpt-5.6-terra:batch in $1.00 · out $6.00/M 👁 openai/o4-mini in $1.10 · out $4.40/M 👁 openai/o4-mini-high in $1.10 · out $4.40/M 👁 openai/o4-mini:batch in $0.55 · out $2.20/M

google

👁 google/gemini-3.1-flash-lite in $0.25 · out $1.50/M 👁 google/gemini-3.1-flash-lite-preview in $0.25 · out $1.50/M 👁 google/gemini-3.1-flash-lite:batch in $0.12 · out $0.75/M 👁 google/gemini-3.1-pro-preview in $2.00 · out $12.00/M 👁 google/gemini-3.1-pro-preview-customtools in $2.00 · out $12.00/M 👁 google/gemini-3.1-pro-preview:batch in $1.00 · out $6.00/M 👁 google/gemini-3.5-flash in $1.50 · out $9.00/M 👁 google/gemini-3.5-flash-lite in $0.30 · out $2.50/M 👁 google/gemini-3.5-flash-lite:batch in $0.15 · out $1.25/M 👁 google/gemini-3.5-flash:batch in $0.75 · out $4.50/M 👁 google/gemini-3.7-flash in $0.75 · out $3.75/M 👁 google/gemini-3.7-flash:batch in $0.38 · out $1.88/M 👁 google/gemini-3.8-flash in $0.75 · out $3.75/M 👁 google/gemini-3.8-flash:batch in $0.38 · out $1.88/M

x-ai

👁 x-ai/grok-4.20 in $1.25 · out $2.50/M 👁 x-ai/grok-4.3 in $1.25 · out $2.50/M 👁 x-ai/grok-4.3:batch in $1.00 · out $2.00/M

meta-llama

👁 meta-llama/llama-4-maverick in $0.19 · out $0.65/M 👁 meta-llama/llama-4-scout in $0.10 · out $0.30/M

deepseek

· deepseek/deepseek-v4-flash in $0.05 · out $0.09/M · deepseek/deepseek-v4-flash-0731 in $0.02 · out $0.32/M 👁 deepseek/deepseek-v4-flash-vision-exp in $0.22 · out $0.65/M · deepseek/deepseek-v4-pro in $0.35 · out $0.70/M · deepseek/deepseek-v4-pro-0813 in $0.25 · out $3.50/M 👁 deepseek/deepseek-v4.1-flash in $0.04 · out $0.29/M 👁 deepseek/deepseek-v4.1-flash:batch in $0.11 · out $0.34/M

qwen

· qwen/qwen3-max in $0.78 · out $3.90/M · qwen/qwen3-max-thinking in $0.78 · out $3.90/M 👁 qwen/qwen3-vl-235b-a22b-instruct in $0.21 · out $1.90/M 👁 qwen/qwen3-vl-235b-a22b-thinking in $0.40 · out $4.00/M 👁 qwen/qwen3.7-flash in $0.03 · out $0.13/M · qwen/qwen3.7-max in $1.48 · out $4.42/M 👁 qwen/qwen3.7-plus in $0.32 · out $1.28/M · qwen/qwen3.8-2.4t-a95b in $2.00 · out $6.00/M 👁 qwen/qwen3.8-27b in $0.42 · out $3.00/M 👁 qwen/qwen3.8-flash in $0.15 · out $0.47/M 👁 qwen/qwen3.8-max-0902 in $2.00 · out $6.00/M 👁 qwen/qwen3.8-max-prime in $4.00 · out $12.00/M 👁 qwen/qwen3.8-omni-flash in $0.15 · out $0.47/M

moonshotai

👁 moonshotai/kimi-k2.6 in $0.95 · out $4.00/M

z-ai

· z-ai/glm-5 in $0.60 · out $1.92/M · z-ai/glm-5-turbo in $1.20 · out $4.00/M · z-ai/glm-5.1 in $0.96 · out $3.03/M · z-ai/glm-5.2 in $0.65 · out $2.04/M · z-ai/glm-5.3 in $1.40 · out $4.40/M 👁 z-ai/glm-5.3-flash in $0.04 · out $0.14/M 👁 z-ai/glm-5.3-flash:batch in $0.06 · out $0.20/M 👁 z-ai/glm-5.3-flashx in $0.37 · out $1.25/M · z-ai/glm-5.3-prime in $2.80 · out $8.80/M · z-ai/glm-5.3:batch in $0.45 · out $2.00/M 👁 z-ai/glm-5v-turbo in $1.20 · out $4.00/M

minimax

· minimax/minimax-m2.5 in $0.27 · out $1.08/M · minimax/minimax-m2.7 in $0.21 · out $0.84/M

baidu

👁 baidu/ernie-4.5-vl-424b-a47b in $0.42 · out $1.25/M

bytedance-seed

👁 bytedance-seed/seed-2.0-code in $0.50 · out $3.00/M 👁 bytedance-seed/seed-2.0-lite in $0.25 · out $2.00/M 👁 bytedance-seed/seed-2.0-mini in $0.10 · out $0.40/M

stepfun

👁 stepfun/step-3.7-flash in $0.20 · out $1.15/M

amazon

👁 amazon/nova-2-lite-v1 in $0.30 · out $2.50/M 👁 amazon/nova-premier-v1 in $2.50 · out $12.50/M 👁 amazon/nova-pro-v1 in $0.80 · out $3.20/M

mistralai

👁 mistralai/mistral-large-2512 in $0.50 · out $1.50/M 👁 mistralai/mistral-medium-3.1 in $0.40 · out $2.00/M 👁 mistralai/mistral-small-2603 in $0.15 · out $0.60/M