SeeTheWork

Recorded grading example · 18 October 2026

Accepted invalid amounts; one of two checks failed.

A deliberately flawed submission tested during development. This is a bounded grading fixture, not a candidate session or hiring-validity study.

The task

solve receives an object with a events list. Return the sum of amounts from records whose id is a nonempty string and amount is a Python integer but not a boolean. Ignore invalid records. Only the first valid occurrence of each ID counts globally. Empty input yields 0.

Submitted work solution.py

1def solve(payload):
2    events = payload["events"]
3    seen = set()
4    total = 0
5    for event in events:
6        if not isinstance(event, dict):
7            continue
8        key, amount = event.get("id"), event.get("amount")
9        if not isinstance(key, str) or not key or not isinstance(amount, int):
10            continue
11        if key in seen:
12            continue
13        seen.add(key)
14        total += amount
15    return total

Captured reviewer findings

The implementation computes the required result for all stated inputs.

Contradicted

The integer check accepts bool because bool subclasses int. An event with id 'x' and amount True returns 1 rather than the required 0. Line 8 ↗

A boolean amount incorrectly reserves its ID. Events ('x', True) then ('x', 5) produce 1 rather than 5 because the later valid record is skipped. Line 9 ↗

The implementation leaves caller-owned input lists, records and vectors unchanged.

Supported

The function only iterates events and reads record fields. Updates target a newly allocated seen set and integer total; no caller-owned list, record, or nested vector is written. Line 1 ↗

Review limit The reviewer inspected the source. It was not given execution records; the independent checks below were saved separately.

Independent execution checks

Saved Docker checks against the same constructed submission. These two cases describe their own scope.

Failed Check 1
Input
{
  "events": [
    {
      "amount": true,
      "id": "a"
    },
    {
      "amount": 7,
      "id": "a"
    }
  ]
}
Expected result
7
Passed Check 2
Input
{
  "events": [
    {
      "amount": -3,
      "id": "x"
    },
    {
      "amount": 20,
      "id": "x"
    },
    {
      "amount": 8,
      "id": "y"
    }
  ]
}
Expected result
5
Record and provenance

holdout-swe-faulty. Original task, code, checks and finding excerpts are preserved. The downloadable extract omits provider payloads and local environment details.

reportSha256
d6a190d78fb3fdf06a62b1719018b0eafbe15a6b08ae5a35fda11282faf37b25
requestSha256
7e0431a78f3cf3c1315e7abd58a2cbded68c2e686db7bf568ef0cf0a264300d9
sourceSha256
e7fbd639ea049bc66045f24ecdada4b5200277c88b1bcdf961c3d262f11d25fa
Download sanitized record ↓