Five production bugs that make fair, hard coding tasks

A good coding task is hard for the same reasons real work is hard, not because the author hid a trick in the wording. When I design tasks for data platforms, I start from bugs I have actually seen in production systems. They are rarely dramatic. They are small, plausible mistakes that make a report look right while being wrong, and they test whether an agent reads carefully and verifies its work.

Here are five patterns I return to. None of them are client-specific. All of them have shipped, somewhere, in real code.

1. Refunds counted as revenue

A transactions table has a type column with values like sale and refund, and an amount column that is always positive. The revenue report sums amount. Totals look healthy. They are wrong by twice the refund volume, because refunds should have been subtracted, not added.

Why it is fair: the schema is visible, the data is visible, and a careful reader who inspects a few rows will see the problem. Why it is hard: nothing crashes, the numbers are plausible, and the fix requires understanding what the report is for, not just what the code does.

2. UTC timestamps in a local-time report

Events are stored in UTC. The daily sales report groups by date. For a business in India, every sale after 6:30 pm local time lands on the next day's row. Daily totals are shifted, weekends look busier than they are, and month-end figures are wrong.

The task states the business timezone. The agent has to notice that the grouping ignores it and convert before grouping. Hidden tests use days with late-evening sales, so a solution that happens to pass on the visible sample fails on the real data.

3. Money stored as floats

Prices are stored as floating-point numbers and summed across thousands of rows. The result is off by a few paise or cents, which is enough to fail a reconciliation against the payment provider. The fix is integer minor units or a decimal type, and rounding at the right step.

This one is fair because the mismatch is visible in the task's own reconciliation check, and hard because the error is tiny and easy to dismiss as noise. Agents that "fix" it by rounding the final total pass the visible sample and fail the hidden one.

4. A hidden byte-order mark in a CSV header

An exported CSV begins with a UTF-8 byte-order mark. The first column header is read as \ufeffid instead of id. Every lookup by that column silently returns nothing, and a join drops all rows. The data looks fine in a spreadsheet, because spreadsheets hide the BOM.

The test is whether the agent inspects the raw bytes, or at least the actual header value, rather than trusting what the file looks like. A one-line fix, once you have found it, and a long hunt if you have not.

5. Records out of order

A script computes running balances or detects state changes by iterating over records, assuming they arrive in time order. The export is ordered by ID, and IDs were assigned by a batch job that ran out of sequence. The logic is correct for sorted input and produces nonsense for the real file.

Hidden tests shuffle the input. Solutions that sort explicitly pass; solutions that rely on the sample file's accidental order fail. The lesson generalises: never trust an ordering the spec did not promise.

What makes these fair rather than cruel

• The bug is in the task's own data or code, visible to anyone who looks. Nothing is hidden in the wording.
• The task says what correct looks like: the reconciliation should match, the daily totals should agree with the timezone stated.
• An experienced engineer would find each bug by doing what engineers do: inspect a few rows, check the types, print the header.
• Hidden tests check the outcome on data the agent has not seen, so the fix has to be general.

Where agents slip

Across these patterns, agents fail in a consistent way: they fix the symptom in the visible sample and stop. They round the total instead of the line items. They sort the sample file instead of the input. They strip the BOM from one file and not the next. Tasks built from real bugs catch that because real data does not cooperate, and that is exactly the skill a lab wants its model to learn.