For AI labs
How a programme with us runs
From one task to full volume, week by week. A free review on day one, a ten-task paid pilot in week one, calibration in week two, full volume with second review on every item from week three. You can stop after any step.
Prashant Kumar Singh · · 3 min read
In short
- Day 1: free written review of one task. Week 1: ten-task paid pilot on your rubric. Week 2: calibration and guideline fixes. Week 3 onward: full volume with second review on every item and a weekly report.
- Every step ends with a decision in your hands; stopping after the pilot costs a week, not a quarter.
- Calibration happens before volume, which is why the acceptance rate rises over a programme rather than decaying.
Labs ask what working together looks like after the pilot, because the pilot is designed to be low-risk and the question is what happens when the risk goes up. Here is the whole programme, step by step, with what you see at each one.
Day 1: free review of one task
You send one task, rubric or test harness, under NDA or sanitised. Within 48 hours you get a written review. Nothing is owed. The review is a sample of the standard every later step will apply, and the fastest way to decide whether a pilot is worth your week.
Week 1: the paid pilot
Ten tasks in your domain, written to your spec, each one carrying a pinned environment, a re-run reference solution, hidden tests checked against null and fake agents, and a second-specialist review. You grade all ten against your rubric. At the end of the week you get the first report: acceptance, rejection causes, rework, guideline changes. Priced by complexity, agreed in writing before the week starts.
Week 2: calibrate
The pilot's rejections are the most valuable thing it produces. Week two is spent on them: every rejection becomes a line in the guideline, a check in the pre-submission script, or a clarification request to you where the rubric was ambiguous. If grading diverged between your reviewers and mine, a calibration batch resolves it. The guideline that comes out of week two is the one volume runs on. Fix the guideline, not the person.
Week 3 onward: full volume
Contributors who passed the scored trial on your rubric join, with their first batch double-reviewed. Every item gets a second-specialist review before submission. Every Friday you get the same report as in week one. When quality drops, and in some week it will, you hear about it from me the same day with the cause and the fix, not from your own numbers a week later.
| Step | What you see | Can you stop? |
|---|---|---|
| Day 1 | Written review of one task | Yes, nothing owed |
| Week 1 | Ten graded tasks, first report | Yes, you keep the tasks and the report |
| Week 2 | Updated guideline, calibration results | Yes |
| Week 3+ | Volume, weekly report, same-day escalation | Yes, at any week end |
What changes at volume, and what does not
What changes: more contributors, more tasks per week, and a guideline that keeps growing from rejections. What does not change: second review on every item, the Friday report, one accountable person, and the NDA and data-handling rules from day one. The gates that produced the pilot's acceptance rate are the same gates at volume. That is the only reason the rate holds.
Questions people ask
- How does an AI training data programme start?
- With a free written review of one task within 48 hours, then a ten-task paid pilot on your rubric in week one, calibration in week two, and full volume with second review on every item from week three.
- Can I stop an AI data programme after the pilot?
- Yes, after any step. Stopping after the pilot costs one week and you keep the ten tasks and the report.
- What happens when quality drops during a programme?
- You hear about it the same day, with the cause grouped by person, guideline or task type, and the fix applied. Daily updates follow until the acceptance rate recovers.
If you're a founder
The product version has the same shape: free teardown, a fixed-price sprint, a decision at the end of each one. Small commitments that each earn the next.
If you run an AI lab or data team
Day one is free and takes 48 hours. Send a task and judge the review.
Sources