A sewer is the hardest place to tell bad data from real data. When it rains, every level rises fast and comes back, which is exactly what a failing sensor can look like. These are four tests of the checks the console runs, each written down before its data was opened, on real networks other people published. Only one held in full. The others are here because they are what you need to know before you trust the one that did.
Where the data and the results come from
Every test below ran on data its owners published under an open licence, and was written down, with its method, its pass marks and how the verdict would be decided, on 25 October 2026 before any check read that data. Nothing was tuned afterwards; where a test had to be changed, the change was dated before the new data was read, and both results are kept. The data sets are credited on the sources page.
| Test | Data, published by | Question | Verdict |
|---|
| P45 | Bellinge, Odense (Denmark)
Pedersen et al., DTU and VCS Denmark
doi 10.11583/DTU.12509249, CC BY 4.0 | Did the checks find the stretches the utility's engineers removed by hand? | FALSIFIED as registered; PARTIAL under amendment A1 |
| P46 | Blue Heart, Eastbourne (UK)
Sweetapple, Illingworth and Melville-Shreeve, University of Exeter, with Southern Water
doi 10.5281/zenodo.20699635, CC BY 4.0 | Does dropping a jump that 30% of the gauges share make the list quieter? | FALSIFIED |
| P47 | Köln-Weiden (Germany)
Sonnenschein and Ziel, Hochschule Ruhr West and Universität Duisburg-Essen
doi 10.5281/zenodo.6992694, CC BY 4.0
rule chosen on Blue Heart, tested once here | Does reading rain beside the levels make the list quieter without missing faults? | SUPPORTED |
| P48 | Blue Heart, Eastbourne (UK)
Sweetapple, Illingworth and Melville-Shreeve, University of Exeter, with Southern Water
doi 10.5281/zenodo.20699635, CC BY 4.0
road gully gauges, not used before | Does the same rain rule hold on instruments it was not chosen on? | PARTIAL |
How it was measured. Each reading is checked a week at a time against the two weeks before it. Clean weeks are weeks the publisher's own quality flags call good, and on them the share of readings the list names is the false-alarm side (Köln has no quality flags, so there every week counts). Planted faults are copies of a clean week with one known fault put in (frozen, zero, silent, spikes, a scale or bias error, noise), and the share found within a day is the other side. Fixed limits are the plain method a SCADA has already: the last two weeks' lowest and highest values, widened by a tenth of their range, or six missing samples in a row.
What a utility's engineers removed by hand
Bellinge, a suburb of Odense, published by DTU and VCS Denmark: about ten years of SCADA levels, with the stretches the engineers removed by hand marked in the data. As registered, the test could not run: the newer exports held no removal at all (FALSIFIED). Amended before the older exports were read, it came out PARTIAL: the checks found 215 of 357 removals (60%), against 100 (28%) for fixed high and low limits. All 357 were on one meter, so it says more about that meter than about the network. And in clean weeks the list named 38% of readings against 26% for the limits: noisier than the plain method.
An idea that did not hold
We thought a fault is local and a change of process is shared, so a jump that many gauges show in the same week should be dropped. Tested on 398 sewer gauges in Eastbourne (Blue Heart, University of Exeter, with Southern Water's permission): FALSIFIED. The rule never fired once. The noise was spikes, 6,449 of 8,591 listings: the fast rises a storm makes, and storms are local. The list named 49% of clean gauge-weeks against 44% for the limits.
Rain, read beside the levels
So the console now reads a rain gauge beside the levels and flows: a rise within 12 hours of 0.2 mm of rain is the weather, not the instrument. The rule was chosen on Eastbourne, then written down and tested once on a network it had never seen: Köln-Weiden (Sonnenschein and Ziel, Zenodo). SUPPORTED. The reading-weeks flagged fell from 36% to 21%, under the 26% of fixed limits, and counted on every flag the planted faults found went from 66% to 67%. Five readings over two years: a first result, not the last word.
What rain hides
The same rule, unchanged, on 494 clean weeks of road gully gauges in Eastbourne, new instruments in the town it was chosen in: PARTIAL. The list grew quieter, from 37% to 15% of gauge-weeks (fixed limits 39%), but the planted faults found fell from 57% to 46%. Rain kept more than half the time quiet there, and a spike or a new level during rain cannot be told from the weather. Frozen, zero and silent readings were found as often with the rule as without.
What the console does with it
It reads a rain gauge beside the levels wherever a plant has one, says that spikes, levels and noise are not judged while it rains, still judges frozen, zero and silent readings in any weather, and names the readings a wet week left unjudged. It shows the likeliest real faults first and the level jumps last, because on the Bellinge network the level check gave the most flags in clean weeks. And for overflow readings it says, year by year, how much of the year is there at all, before anyone reports it.
Every result, with the criteria fixed before each run.