[W] SeeTheWork

Candidate review

Discuss a pilot ↗

Illustrative scores · independent of your workspace edits

ML engineering · Example candidate

Investigate a suspicious model improvement

Print / save brief ↗

Task outcome

Addressed customer leakage; production performance is untested.

Delivered Evaluation + model report

Evaluation score3.5/5Illustrative score

Score breakdown

Expand to see why
Leakage diagnosis4.0/5

Detects shared customers. Temporal and feature leakage need further inspection.

Evaluation design4.0/5

Separates customers and fits the pipeline only on training records.

Comparison evidence3.0/5

Defines a fair baseline protocol, but supplies no measured comparison in this excerpt.

Deployment planning3.0/5

Names future-window testing; drift thresholds and rollout controls remain unspecified.

How to read these scores

1 = major gaps · 3 = partially meets · 5 = fully meets. This example weights all four criteria equally.

These scores and excerpts are illustrative, not a live evaluation or a hiring recommendation. In the product, evaluations score submitted work against the configured criteria; missing evidence is identified separately.

Full findings and sources
01

Identifies leakage in evaluation

Supported in this example

02

Compares models consistently

Supported in this example

03

Plans for distribution changes

Follow-up needed

Optional follow-up

How would you evaluate this model when new entities and data drift appear?

Illustrative scores and evidenceInspect an actual saved grading example ↗
Source [01] · Illustrative excerpt

Split audit

Look for shared entities.

split_audit.py

overlap = set(train.customer_id) & set(test.customer_id)
assert len(overlap) == 0
# Original split: assertion fails.
What the source supports
The audit exposes an entity shared between training and evaluation data.
What remains open
No entity overlap alone does not rule out temporal or feature leakage.