§ 01Context
The product is a companion app for competitive esports fans and bettors. In one place it shows live scores for in-progress matches, a best-available-odds table that compares coefficients across many bookmakers, and AI match predictions — win probability plus derived markets. It runs on a freemium subscription. In a product like this the interface is the easy part; the data is the product, and the data is the problem.
§ 02Challenge
Every source disagrees with every other source. Bookmaker feeds are wildly inconsistent — one keys outcomes by numeric IDs instead of team codes; another's “is-live” flag is unreliable, so live vs. pre-match has to be inferred from time; two of them have no API at all. The same match appears under slightly different names across providers and has to be merged into one. Live scores must update in near-real-time and never silently go dark. And the prediction models are useless unless thousands of team-name variants map to a single canonical identity.
The hard part of an odds product isn't the UI — it's making eight disagreeing feeds tell one consistent story, in real time.— Engagement scope
§ 03Architecture
We built it as a set of cooperating services, each owning one hard job:
- Ingestion adapters & normalization. A provider-adapter per feed maps every source's quirks into one
aggregated_oddstable. The two API-less bookmakers are synthesized from a primary source using documented, game-specific correction coefficients, with an explicit fallback chain when the primary is missing. - Match de-duplication & entity resolution. Team-name normalization plus time-bucketed start times merge the same match across providers; a dedicated fuzzy-resolution step maps bookmaker names onto canonical team IDs.
- Live-score relay. A rich real-time feed (~1–3s push) is preferred when fresh, with slower polling as an automatic fallback so scores never stall.
- ML predictions. A separate Python / scikit-learn service (ELO pre-draft, draft-synergy post-draft) called over HTTP, its output flowing back through the relay to clients.
A later generation re-architected the coupled services onto an event bus with typed event contracts, an outbox pattern for reliable publishing, a columnar store for odds history, and an in-memory cache for millisecond live state — a clean CQRS/streaming shape for live sports data.
One adapter per feed, one normalized table — including odds synthesized for API-less bookmakers via correction coefficients.
A WebSocket relay carrying odds, live updates and predictions to clients — rich feed preferred, polling as fallback.
Python + scikit-learn: ELO and draft-synergy models, fed by live snapshots, mapped to 99% of teams via an ID dictionary.
Typed events, outbox publishing, a columnar store for odds history and an in-memory cache for live state.
One connection once flooded the backend with 436,000 messages and a 3.4M backlog. The fix was to stop bulk-syncing and diff every five seconds instead.
— Reliability post-mortem
§ 04Real-time & reliability
Real-time systems fail in ways demos never show. This one taught the lessons the hard way, and we fixed each at the root:
- Backpressure meltdown. The initial-sync path pushed ~436K messages per connection, causing disconnect storms and a 3.4M-message backlog that starved fresh odds. Replaced with a 5-second incremental diff — a textbook push-vs-poll and backpressure fix.
- Stale-live matches. Matches marked “live” but no longer updating were cleaned up with proper lifecycle timeouts, so the board reflects reality.
- Upstream format drift. When a provider silently changed its payload shape, one of two parallel consumers broke; we hardened parsing against exactly that class of change.
- Adaptive sync tiers. Per-match cron frequency scales from once-a-minute near kickoff to hourly for distant matches — freshness where it matters, API budget everywhere else.
§ 05Result
- Eight feeds, one truth. Odds from eight sources — including two with no API — normalized into a single comparable table.
- Live data that stays live. Sub-second updates with automatic fallback and stale-state cleanup; 262K snapshots across 454 matches in a month.
- Predictions that actually resolve. Team coverage went from ~1% to 99% after entity resolution — the difference between a model that fires and one that can't.
- A core built to last. A typed, event-driven backbone the product can grow on.
— Have a real-time or data-heavy product in mind? Start a conversation.


