The parts that were genuinely hard — and how they were solved.
The calibration ladder that proves the models work
Every meaningful score change logs a prediction; an hourly job stamps the actual outcome at 24 hours and 7 days. For a long time the whole view was empty — timestamps were written as objects and parsed to NULL on all 555,000 rows. After the fix, the ladder came out monotone on the latest week: contacts scored Hot responded within 24 hours 47.5% of the time, Warm 15.2%, Cool 9.4%, Cold 1.0%, Dormant 0.6%. That ordering, measured against outcomes rather than asserted, is the single most defensible artefact in the system.
A control arm inside the prediction log
The prediction logger is change-gated, which biases the sample toward contacts that moved. So it also writes a small unbiased random arm every run, tagged as a separate source. That lets the calibration view segment on prediction source and measure the main stream's selection bias directly, rather than hoping it's small.
Billing predicates that close the undefined-flag trap
The convention that an unset ownership flag means legacy team-pay meant every naive equality check let legacy seats reach Stripe and buy a seat their team already paid for. Two pure predicates now gate every mutating billing route on both the modern and legacy stacks. The second-order bug was worse than double billing: the resulting webhook would have silently converted a team seat to self-managed. 142 tests across five files hold the line.
Attribution that records ambiguity instead of guessing
Every activity is attributed not just to a person and an agent but to the deal it happened under, by checking which deals were open at that moment. One candidate is a clean attribution; none means it was lead work. When several open deals overlap, the system records that the attribution was ambiguous along with the candidate deal IDs rather than silently picking one — because a confident wrong number is worse than a flagged uncertain one.
A relevance classifier born from one bad score
Teams run side businesses through their CRM. A contact belonging to an entirely different business, sitting in a Do Not Call stage, was carrying a Win Score of 99 and the label Active Buyer. The classifier that fixed it is deliberately tuned for precision over recall — wrongly excluding a real lead removes them from every AI surface, which is far worse than wasting analysis on a lender — and its single verdict is enforced at six separate pipeline gates.
Design consistency enforced by the test suite
The same metric was rendering on five admin screens with two definitions and three different values. The fix was a frozen metric catalogue and one shared data fetch — but the durable part is the contract test that walks every admin component file and fails the build on hardcoded metric labels, so the drift cannot come back. A companion test pins deleted routes, redirects and the navigation row count.
Reverse-engineering a DSL that fails silently
The CRM's Smart List filter language returns HTTP 200 with zero results for a wrong operator, a scalar value, a camel-cased operation or a raw custom-field handle — four ways to be silently wrong. The condition builder converts all four into loud exceptions at construction time, including a banned numeric operator with a documented arithmetic workaround. Thresholds are per-account percentiles, because a fixed cutoff selects nobody on a team whose median score is single digits.
Priority lanes so new customers never wait behind a backfill
Extraction work runs in four explicit lanes — day-one connect, continuation, coverage backfill, and fleet re-pass — and a higher-urgency enqueue supersedes a queued lower-priority job for the same account. A team that connects today jumps the queue ahead of the fleet-wide sweep, so their consolidated scores land hours after signup rather than after the next weekly rebuild.
