Skip to main content

Fixed.sh

← All posts

Fixed.sh blog

Graph blast radius and hypothesis ranking on service topology

Why dependency graphs beat flat alert lists for ML, and how reachability and fan-out shape which theories rank first.

  • ML
  • research
  • topology

When a synthetic monitor fires on “orders in the west region,” the failing monitor is rarely the failing system. The interesting question is blast radius: which shared dependencies explain correlated failures across services?

That question is naturally a graph problem. Flattening alerts into text for an LLM discards structure that ranking algorithms use for free.

The dependency graph as a prior

Let G be a dependency graph where vertices are services, hosts, or infrastructure components and edges are “depends on” or “runs on.”

When a set of symptom nodes is unhealthy, candidate root causes often lie in:

  • Upstream ancestors (what they share)
  • Recent change targets on paths into (S)
  • High betweenness nodes whose failure fans out widely

A hypothesis h (“redis-cluster saturation”) is attractive when:

  1. h has a short path to many affected symptoms.
  2. Evidence modalities agree on h’s state (latency, errors, memory).
  3. Alternative ancestors explain only a subset of those symptoms.

This is structurally similar to source localization in network tomography, adapted to CMDB-ish graphs that are incomplete and sometimes wrong.

Propagation models (simple but useful)

You do not need a full physics simulation to rank theories. Useful heuristics include:

  • BFS/DFS reachability from candidate h to the affected set, penalize hypotheses that cannot reach observed symptoms.
  • Fan-out count: how many downstream services depend on h.
  • Bipartite matching between alert groups and shared infrastructure (one redis, many APIs).

When the graph is uncertain, maintain multiple topology hypotheses (“orders-api → redis” vs “orders-api → cache → redis”) and let evidence break ties.

Combining graph priors with ML scores

Think of final ranking as:

A useful mental model is a weighted sum:

score(h) = α·graph(h) + β·fusion(h) + γ·memory(h)

where:

  • graph(h): reachability, fan-out, path length
  • fusion(h): multimodal evidence agreement (metrics, logs, changes)
  • memory(h): similarity to resolved past incidents involving h

Weights need not be learned end-to-end on day one. Interpretable weights help humans understand why a theory jumped to 78% confidence.

Signal maps as human-readable graphs

Visualization matters for trust. A signal map is not decoration, it is the same object the ranker uses, rendered for audit:

  • Source alert or ticket on the left
  • Traversal through services
  • Target hypothesis node
  • Optional “resolved” or “RCA ready” terminus

When the science and the UI share a representation, engineers argue about the graph, not about mystery scores.

Limits of topology-only ML

Graphs from CMDB and discovery tools are stale. Ephemeral workloads, mesh routes, and shadow dependencies create false negatives. Pipelines should:

  • Ingest live traces and recent tickets to patch edges
  • Show confidence on edges (“inferred from traces, 2h window”)
  • Allow manual edge add during an incident

Why this is not “GraphRAG hype”

GraphRAG in the abstract often means “retrieve community summaries from a knowledge graph.” In ops, the immediate win is narrower and more valuable: rank hypotheses on the actual failure topology before anyone writes a postmortem paragraph.

Fixed positions investigation as graph-first triage with language layered on top, not language pretending it saw a graph it never had.