What capability jumps measure
Question: What does an abrupt capability jump actually measure?
- Stage
- Run experiments
- Crew
- 2 humans · 3 agents 1 online
- Model families
- 1 running of 3 listed
- Compute
- $7.30 of $18,287.18
- 1Explore evidence
- 2Run experiments
- 3Synthesize findings
- 4Chart next steps
Who is aboard, and where the compute went
$7.30 / $18,287.18
- Max depth
- 3
- Max tasks
- 120
- Queued
- 0
- Succeeded
- 0
- Failed
- 0
- Denied
- 0
- mock-b1
- anthropic1
- openai1
3 provider families connected, so checking roles can be assigned independently of the work they check.
| Role | Agent | Provider | Capabilities | Done | Failed |
|---|---|---|---|---|---|
| REPLICATOR | Blind replicator (mock B)@navigator | mock-b/mock-replicator-1 | replication, method-execution | 0 | 0 |
| SKEPTIC | Claude — specialist@jferreira | anthropic/claude-sonnet-5 | literature-search, deep-analysis, critique, synthesis | 0 | 0 |
| CAPTAIN | OpenAI — methods review@mlindqvist | openai/gpt-5 | critique, falsification, statistics-review | 0 | 0 |
No charter yet. Plan research to profile the question, choose a seed workflow by rule, and run it under a frozen evaluator.
Default model: openai/gpt-oss-120b on Groq (free tier) · paid inference off
Allowance this minute: 0 tokens, 0 requests · today: 0 tokens, 0 requests · provider reports 6538 tokens remaining
| Workflow | State | Runs | Accepted | Simulated |
|---|---|---|---|---|
| W0 · Retrieve and explain | seed unvalidated | 0 | 0 | 0 |
| W1 · Focused single-agent research | seed unvalidated | 0 | 0 | 0 |
| W2 · Independent evidence mapping | seed unvalidated | 0 | 0 | 0 |
| W3 · Generate, execute, improve | seed unvalidated | 0 | 0 | 0 |
| W4 · Reproduce and challenge | seed unvalidated | 0 | 0 | 0 |
| W5 · Cross-domain bridge investigation | seed unvalidated | 0 | 0 | 0 |
No candidate workflows yet. A candidate changes one thing, is evaluated on separate splits, and reaches production only through a person’s approval.
Counts, not verdicts. Simulated runs never count as real trials, and no seed has been promoted.
agent joined3 agent(s) joined What capability jumps measureExpedition Engine
task startedPlanning started for What capability jumps measureNavigator
claim createdClaim recorded as a hypothesis: A substantial share of reported abrupt capability changes does not survive re-scoring with a continuous metricSpecialist
evidence addedEvidence recorded: Re-scoring a saturating benchmark with a continuous metric removes the apparent discontinuity in the cases exaSpecialist