The live feed that never goes dark

A live sports data platform has one job that isn't negotiable: the score on screen has to match the score in the world, within a few seconds, for as long as the match is running. Most of the time that's easy — a fast feed pushes an update and the UI reflects it almost instantly. The hard part is the rest of the time, when the fast feed goes quiet and nothing downstream is supposed to notice.

This is how we built that resilience into a real-time esports odds and live-score platform we built: a rich live feed preferred whenever it's actually fresh, a slower fallback source running underneath it at all times, and cleanup jobs that stop the system from lying about what "live" means. None of the pieces are exotic. What mattered was how conservatively they were wired together — and the three ways we got that wiring wrong before it held.

§ 01Two sources, one truth

The design settled on two independent paths to the same state. A rich live feed pushed updates over a persistent connection every one to three seconds while a match was active — cheap to consume, since it only sent what changed. Underneath it, a plain polling job hit a slower fallback source every thirty seconds, for every match marked live, whether or not the push feed was behaving.

The poller was never optional — it ran continuously whether or not anything looked wrong. The only decision left for read time was which answer to trust right now, and that came down to one number: how old is the most recent update we have.

// choose a source: prefer the push feed while it's fresh
function currentScore(matchId) {
  const pushed = liveCache.get(matchId);
  const age = Date.now() - (pushed?.receivedAt ?? 0);

  if (pushed && age < FRESHNESS_MS) return pushed.score;
  return pollCache.get(matchId)?.score ?? pushed?.score ?? null;
}

// the fallback poller wakes up for any match the push feed has gone quiet on
setInterval(() => {
  for (const match of liveMatches()) {
    const pushed = liveCache.get(match.id);
    const age = Date.now() - (pushed?.receivedAt ?? 0);
    if (age >= FRESHNESS_MS) pollOne(match.id);
  }
}, POLL_INTERVAL_MS);

Thirty seconds is a compromise between acceptable staleness and how often we're willing to hit a rate-limited source. The exact number matters less than the property it protects: a client watching a match should never be able to tell that the push feed died. It should just look like the platform switched to a slightly slower cadence.

§ 02A feed that looked alive and wasn't

The first real test wasn't a network outage. The upstream provider changed the shape of its push payload — from an object wrapper around the match array to a bare array on its own. One consumer, a data-collection service, had been written defensively enough to handle both shapes and kept working without anyone noticing the change. A second consumer, the relay broadcasting those updates onward, only understood the old wrapped shape. It checked for a status field that no longer existed, found nothing, and returned early. Silently. Every single message.

One of the two consumers had been silently dropping every message from the feed for hours. No connection errors, no failed health checks — the socket was open, authenticated, and receiving data the whole time. It just never recognized any of it as data.

Every signal that was cheap to check said the system was fine: connection up, upstream healthy, no reconnects because nothing had disconnected. The only thing wrong was a shape assumption baked into one code path, downstream of the socket, silently discarding every message that didn't match it. A payload that fails to parse loudly is an incident. One that fails to parse quietly is a slow leak that looks like calm water.

The fix wasn't clever — accept either shape, log the ones you don't recognize, and treat a long run of zero-parsed-messages as its own health signal. "Connected" and "producing usable data" are different claims, and a check that only verifies the first will happily report green through an outage.

§ 03Stale-live: the status a timestamp forgot to update

The second failure mode lived in the database, not on the wire. A routine audit of matches flagged as currently live turned up a lopsided number: over a hundred and forty carried a "live" status, but fewer than five had received an update in the last hour. The rest were, for all practical purposes, finished — but nothing had ever told them to stop being live.

The root cause is familiar once you see it: "live" was set once, at kickoff, and cleared once, on an explicit "finished" event. If that event never arrived — dropped coverage, a connection blip, any mundane reason — the match stayed live indefinitely. Not incorrect logic; a missing case. Nothing existed for "we haven't heard anything in a suspiciously long time," only for "we were explicitly told it's over."

The fix is the same pattern behind heartbeat timeouts in any distributed system — a live entity needs a lease, not just a state. Every match carrying a live status now carries a last-update timestamp checked on its own schedule, independent of the push feed or the poller:

• If a live match hasn't received an update — from either source — in longer than a generous multiple of its expected cadence, a cleanup job transitions it out of live status, regardless of whether an explicit "finished" signal ever shows up.
• The timeout has to survive a normal lull without being loose enough to leave a genuinely dead match sitting in the live list for hours.
• The cleanup job runs on its own schedule, independent of both data paths, so a bug in either the push feed or the poller can't also disable the mechanism meant to catch that bug's symptoms.

"Live" should never be a status you set and forget — it should expire on its own if nothing renews it.

§ 04The threshold that matched nothing

A related mistake surfaced once the fallback path got more scrutiny: a freshness threshold picked without reference to how often the fallback source actually refreshed. Data was "stale" if it hadn't changed in ten minutes — but the source feeding it only ran on an hourly cycle. The arithmetic is unforgiving: a source that updates once an hour looks stale, by a ten-minute yardstick, for roughly fifty of every sixty. The threshold wasn't wrong on its own terms; it was measuring a source that could never satisfy it, so good, current data was treated as expired and quietly replaced with something synthetic.

This is the same class of problem covered in more depth in how we built the adapters that reconcile odds across sources: a staleness threshold is only meaningful relative to the cadence of the thing it's judging. Pick it in isolation, copied from a different source with a different refresh rate, and it stops measuring staleness and starts measuring the gap between two unrelated cadences.

The fix: tie the threshold to the known sync interval of whatever it's judging, with margin comfortably above it, never below. Small change, but it's the difference between a fallback that quietly takes over during genuine gaps and one that's permanently, uselessly, in control.

§ 05Sync frequency has a budget, not a constant

The fallback source wasn't polled at one fixed rate for every match — too slow for matches about to start, wastefully fast for matches days away. Sync frequency was tiered by proximity to kickoff instead: roughly once a minute for a match live or about to be, stretching out as the match receded into the future, down to about once an hour for anything more than a day or two out.

1. Matches live or in their pre-match window get the tightest interval the fallback source's rate limits allow — freshness matters most exactly when the audience is watching.
2. Matches a few hours out get a middle tier — frequent enough to catch odds or scheduling changes, without spending budget on something nobody's watching yet.
3. Everything further out gets the loosest tier. An hourly check is plenty two days out — the difference between a budget that scales with your catalog and one that scales with catalog times a fixed high-frequency interval, which breaks as the catalog grows.

The reasoning is closer to rate-limit budgeting than real-time engineering: every poll costs a request against a source with a ceiling, so the question isn't "how fresh can we make this" but "where does an extra poll buy the most freshness per unit spent." Near kickoff, that's obviously here. Two days out, it isn't — and pretending otherwise just burns budget where it doesn't count.

§ 06Defensive parsing at the boundary

Every failure above shares a boundary: the point where outside data enters code that assumes a shape. The format-change break happened there. The stale-live problem was partly a parsing problem too — an absent update was never treated as information in its own right. The fix, every time, was to stop trusting the shape and start checking it:

• Validate the shape of an external payload before any business logic touches it. A missing or renamed field should be a loud, logged event, not a silent undefined propagating three function calls deep before it causes a wrong answer.
• When a payload doesn't match any recognized shape, log it and move on — don't crash, and don't silently drop it either. Both extremes hide the same problem from the people who'd want to know.
• Treat "connected, but producing nothing usable" as its own health state, distinct from "connected." A relay that's technically alive but has stopped parsing anything is worse than one that's visibly down, because nobody pages on a metric that still says green.
• Don't let a health check answer a narrower question than the one you actually care about. "Is the socket open" and "is fresh data arriving" are both cheap to check, and they are not the same question.

Same instinct behind the backpressure work described in our writeup on a WebSocket relay that drowned in its own sync: a connection healthy on the metrics you happened to watch can still fail at the one job that matters. Stop trusting the easy signal, instrument the one that predicts the failure.

§ 07What we'd check before shipping the next one

The checklist this settled into:

1. Define freshness as an explicit, measured number — derived from the fallback source's actual cadence, with margin above it, never below.
2. Run the fallback path continuously, not conditionally. A poller that only starts once something's detected as broken adds a delay to every failure; one that's always running just quietly takes over.
3. Give every "live" or "active" status a lease, not a one-way switch — an independent, timeout-driven path back to false that doesn't depend on an explicit signal arriving.
4. Accept more than one shape at every external boundary, log what you don't recognize, and never let an unrecognized shape fail silently — silence is what turns a format-change break into hours of bad data instead of minutes.
5. Scale poll frequency to how much freshness is worth at that moment, not to a single constant. Near-term data is worth far more per request than far-out data, and the budget should reflect that.

None of this depends on the specific feed, source, or domain — a sports score, an inventory count, a sensor reading. Prefer the rich source when it's demonstrably fresh, never let the fallback stop running, and build an independent way to notice when "live" has quietly stopped meaning anything.

— End of essay. If your live data has a way of going stale without telling anyone, let's talk before your dashboard finds out first. Start a project →