wtf( )unctionsystem design, drawn
← all problemsML EngineeringHard

A feature that knew the future

The training set is built by joining each labelled event to that customer's feature values. The join is on customer id.

The feature store holds the customer's current values. So a row labelled six months ago is joined to what is known today — including facts that are a consequence of the label. The model learns from the future, and it scores beautifully offline.

Make the join respect the moment each label was recorded.
Components — tap one, then tap a slot on the diagram
?Joining labels to features by entity is a leak unless it is also joined by time.

Outside every boundary: Training set (one row per label; FAILED: knows the future), Feature store (current values; FAILED: no history), Offline score (excellent), Labelled events (with timestamps), an empty slot for the what the join keys on, an empty slot for the what the store must keep Connections: Labelled events calls what the join keys on — each with its time (step 1) Feature store calls what the join keys on (step 2) what the join keys on calls Training set (step 3) what the store must keep controls Feature store (step 4) Training set controls Offline score — on leaked data

Training setone row per labelknows the future
Feature storecurrent valuesno history
Offline scoreexcellent
Labelled eventswith timestamps