Drift checks that people do not learn to ignore

A drift check that fires on every build gets muted in a week. What to check, what to warn on, and what to leave alone.

The failure mode for any automated check is not missing things. It is training the team to click past it. A drift check has to earn its place on every run, which means being selective about what it blocks on.

Block on what is certainly wrong

A documented identifier that no longer exists in the source is a fact, not an opinion. So is a documented command that fails. These can fail the build, because they are never false positives.

Warn on what is probably wrong

A documented default that differs from the code default is usually drift, and sometimes an intentional override. A version number that has fallen behind is usually neglect, and sometimes a deliberate pin. Warn on these and let a human decide.

Leave alone what cannot be decided mechanically

Whether the prose still describes the behaviour correctly is a judgement. Whether the tutorial flows well is a judgement. Tooling that pretends otherwise produces confident noise, which is worse than silence.

PaperDrift separates these tiers rather than collapsing them into one pass/fail, because a single undifferentiated signal is what gets muted.

← Back to blog

Was dit resultaat nuttig?Terugkoppeling