For AI labs
Benchmarks are moving to real work. GDPval shows how
OpenAI's GDPval has 1,320 tasks across 44 occupations, written by professionals averaging 14 years of experience and graded blind by their peers. Frontier models were rated as good as or better than experts on just under half. The gap is closing, which makes the tasks the scarce thing.
Prashant Kumar Singh · · 4 min read
In short
- GDPval: 1,320 tasks across 44 occupations in 9 sectors, each based on a real work product, written by professionals averaging 14 years of experience, with about five rounds of review per task.
- Grading is blind pairwise comparison by expert graders from the same occupation: better, as good, or worse than the human deliverable.
- The implication for labs: tasks now need domain experts, multi-round review and expert graders. The pipeline is the product.
In September 2025 OpenAI released GDPval, and it is the clearest public signal of where evaluation is going. Not puzzles, not toy repositories, not multiple-choice questions with a known answer. Real work, written by people who do that work for a living, graded by their peers, with a deliverable at the end that could be handed to a client.
What GDPval is
1,320 tasks across 44 occupations in the nine sectors that contribute most to US GDP. Each task is based on a real work product: a legal brief, an engineering drawing, a financial model, a nursing care plan, a marketing campaign. The task writers are professionals in those occupations with an average of 14 years of experience, screened before writing. Each task went through roughly five rounds of review before it was accepted into the set.
How it is graded
Expert graders from the same occupation receive a model's deliverable and a human professional's deliverable for the same task, blind, and judge which is better, or whether they are equivalent. Pairwise, by people qualified to judge. The paper reports that expert graders agreed with each other on about 71% of comparisons, which is the number every rubric discussion on this site keeps coming back to: even experts disagree, so the grading process matters as much as the grader.
The headline result
The best frontier models were rated as good as or better than the human expert's deliverable on just under half the tasks. That number will be out of date quickly, and the direction is the point: the gap between model and professional on real work is closing. When models are close to experts, the thing that separates good evaluation from bad is not model access. It is the quality of the tasks and the graders.
What this means for anyone who builds tasks
| GDPval requirement | What it takes to supply |
|---|---|
| Tasks based on real work products | Authors who do the work, with the artefacts to hand |
| Writers with years of experience | Vetting that checks experience and skill, not availability |
| About five review rounds per task | A review pipeline with second specialists and calibration |
| Expert graders from the same occupation | Graders vetted the same way as writers, with a rubric |
| Blind pairwise comparison | A harness that hides provenance and randomises order |
Every row is a pipeline, not a hire. The labs that get good real-work data will be the ones with the pipeline, and the vendors who can supply it will be the ones who already run one.
Where it goes next
PaperBench did this for research reproduction. Terminal-Bench 2.0 did it for terminal work. GDPval did it for the economy. The pattern is the same each time: realistic tasks, expert authors, heavy review, graders who could do the work themselves. Expect every serious evaluation in 2027 to look like this, and expect the tasks, not the models, to be the hard part to source.
Questions people ask
- What is GDPval?
- An OpenAI evaluation released in September 2025 with 1,320 tasks across 44 occupations in nine high-GDP sectors, each based on a real work product, written by experienced professionals and graded blind by expert peers against a human deliverable.
- How were GDPval tasks written and reviewed?
- By professionals averaging 14 years of experience in the occupation, screened beforehand, with each task going through about five rounds of review before acceptance.
- How well do AI models do on GDPval?
- The best frontier models were rated as good as or better than the human expert's deliverable on just under half of tasks at release, with expert graders agreeing with each other about 71% of the time.
If you're a founder
If your product's AI does professional work, evaluate it the GDPval way: real deliverables, judged blind by someone who does the job, against a human version.
If you run an AI lab or data team
Need real-work tasks in a technical domain? Send one existing task and the free review will show how it measures against the GDPval bar.
Sources