Articles

Short-form writing on topics that sit at the intersection of software engineering and data science. New articles appear here as they’re published.

Horizontal interval plot with two rows. The top row, 'Model A (1st)', shows a blue dot at 71.2% with a horizontal bar spanning roughly 66.8% to 75.6%. The bottom row, 'Model B (2nd)', shows an amber dot at 70.4% with a bar spanning roughly 65.9% to 74.9%. The two bars overlap across almost their entire length. A small bracket between the two dots is annotated 'leaderboard gap: 0.8 pts', and a caption notes the interval half-width is about ±4.4 points — far larger than the gap.

The leaderboard has no error bars

inference
engineering

A model leaderboard reads like a build number: one authoritative ranking, settled. But an eval score is a sample statistic, and the gap between first and second is usually well inside the noise. A ranking reported without an interval is a point estimate wearing the costume of a fact.

19 Jun 2026
Matthew Gibbons
 

The crash that didn’t happen

inference
engineering

Reliability work is invisible when it works. That’s not a management problem — it’s a causal one: the value of prevention lives in the branch of history you never got to observe.

12 Jun 2026
Matthew Gibbons
 

A stranger to your own codebase

engineering
collaboration

Agents hand you the code without the understanding that used to come with writing it, so you re-onboard onto your own project every iteration. In modelling you can ship what you can’t explain — but code isn’t a model, and someone has to keep understanding it.

5 Jun 2026
Matthew Gibbons
 

The test that couldn’t have failed

inference
testing

A green tick only means something if a broken version of the code would have turned it red. Pick the wrong input and you can write a test that passes no matter what — proof of nothing, in a reassuring shade of green.

22 May 2026
Matthew Gibbons

Line chart with consecutive passing runs (zero failures observed) on the x-axis from 0 to about 300, and plausible true failure rate as a percentage on the y-axis from 0 to 30. A blue curve labelled 'Exact 95% upper bound' falls steeply from the top left and flattens as it approaches the x-axis; an amber dashed curve labelled 'Rule of three (3/n)' tracks just above it. A red-orange dot at 20 runs sits at about 14%, annotated '20 green runs still allow a ~14% failure rate'. A dotted grey horizontal line at 1% is annotated to show that roughly 300 runs are needed before the bound drops below it.

Your flaky test is a hypothesis test

inference
testing

Quarantining a flaky test assumes its true pass rate is 100%. It isn’t — a test that touches anything real is a Bernoulli trial, and nineteen green runs prove far less than you’d think.

17 Apr 2026
Matthew Gibbons

Line chart showing 30 days of simulated daily visitors. Blue points show observed values scattered around the expected value of 1,000. An amber dashed horizontal line marks the point prediction at 1,000. A shaded amber band extends from roughly 938 to 1,062, representing the 95% prediction interval. Most but not all observations fall within the band.

How much should you trust a prediction?

foundations
thinking

A point prediction without uncertainty is like a deploy without monitoring. Prediction intervals tell you how much to trust a number, and whether the uncertainty is tight enough to act on.

23 Jan 2026
Matthew Gibbons

Line chart showing ten simulated time series of daily website visitors over 30 days. Ten blue lines each follow a different path, fluctuating between roughly 920 and 1,080 visitors per day. A dashed orange horizontal line marks the expected value of 1,000. Although every trace was generated from identical parameters, no two lines are the same, demonstrating that this variation is an intrinsic property of the Poisson model, not a modelling error.

Your model is wrong (and that’s the point)

foundations
thinking

Software engineers treat error as a bug, but in data science error is information. That shift changes everything.

3 Jan 2026
Matthew Gibbons
No matching items