Phase 19 - Lesson 19.9

Graph Algorithms: BFS, DFS, Dijkstra, and Minimum Spanning Trees

Modelling a problem as a graph is usually the whole solution; after that, four classical algorithms cover most of what you need.

⏱ 70 min● Intermediate🔗 Prereqs: 19.7 dynamic programming; heaps/queues
↖ Phase 19 hub
Builds on: 19.7’s DP on a DAG is literally shortest paths on an acyclic graph.
Leads to: 19.10 searches implicit graphs (the state graph you never build); the Euler Lab’s Graphs & Paths problems (grid path minimisation, network reduction) are direct applications.

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

Key Vocabulary

Graph
\(G=(V,E)\); directed or not, weighted or not. In this phase the graph is often implicit: vertices are states, edges are moves.
BFS
Breadth-first search from a source; visits vertices in order of edge-count distance, giving shortest paths in UNWEIGHTED graphs. \(O(V+E)\).
DFS
Depth-first search; the natural tool for connectivity, cycle detection, topological order, and articulation points. \(O(V+E)\).
Dijkstra
Shortest paths from a source with NON-NEGATIVE weights: repeatedly settle the nearest unsettled vertex. \(O((V+E)\log V)\) with a binary heap.
Minimum spanning tree
A cycle-free subset of edges connecting all vertices with minimum total weight. Kruskal (sort + union–find) or Prim (grow with a heap).
Cut property
For any partition of \(V\) into two nonempty parts, the cheapest edge crossing the cut belongs to some MST. This one lemma proves both Kruskal and Prim.
Union–find
Disjoint-set structure with union by rank + path compression; near-constant \(\alpha(n)\) amortised per operation.

Intuition & Motivation

Intuition
Most ‘hard’ problems become easy the moment you can say what a vertex is. A grid cell, a configuration of a puzzle, a state \((\text{mask},\text{city})\) - once vertices and edges are named, BFS/Dijkstra/DP are off-the-shelf. Conversely, if you cannot name the vertex, no algorithm will save you. The modelling is the mathematics; the algorithm is the easy part.

BFS and DFS: what each actually computes

BFS from \(s\) processes vertices in non-decreasing order of hop-count, so the first time it reaches \(v\) it has found a shortest (fewest-edges) path. That is only true when all edges have equal weight. DFS makes no distance guarantee at all; its value is the recursion structure - discovery/finish times, back edges (hence cycles), and topological ordering of a DAG.

ProblemAlgorithmCost
Shortest path, unweightedBFS\(O(V+E)\)
Shortest path, weights \(\ge0\)Dijkstra + binary heap\(O((V+E)\log V)\)
Shortest path, negative edgesBellman–Ford (and it detects negative cycles)\(O(VE)\)
Shortest path, DAGRelax in topological order\(O(V+E)\)
All pairs, small \(V\)Floyd–Warshall\(O(V^3)\)
Connectivity / cycles / topo-orderDFS\(O(V+E)\)
Minimum spanning treeKruskal (union–find) or Prim (heap)\(O(E\log E)\)

Dijkstra: the greedy exchange argument

Theorem - Correctness of Dijkstra
If all edge weights are \(\ge0\), then when Dijkstra pops the unsettled vertex \(u\) with the smallest tentative distance \(d[u]\), that \(d[u]\) is the true shortest distance \(\delta(s,u)\).
Proof
Suppose not, and let \(u\) be the first vertex popped with \(d[u]\gt \delta(s,u)\). Take a true shortest path \(s\rightsquigarrow u\) and let \((x,y)\) be its first edge leaving the settled set (\(x\) settled, \(y\) not). Since \(x\) was settled correctly, \(d[y]\le d[x]+w(x,y)=\delta(s,y)\) after \(x\) was relaxed. Because all weights are non-negative, \(\delta(s,y)\le\delta(s,u)\lt d[u]\). So \(d[y]\lt d[u]\), contradicting the choice of \(u\) as the minimum. □ The non-negativity is used exactly once - in \(\delta(s,y)\le\delta(s,u)\) - and that is precisely where negative edges break the algorithm.
The lazy-deletion detail
Python’s heapq has no decrease-key. The standard workaround: push a new \((d,v)\) pair on every improvement, and when popping, skip any pair whose \(d\) exceeds the recorded best. The heap may hold up to \(E\) entries; the complexity stays \(O(E\log E)=O(E\log V)\). Forgetting the skip test is the most common Dijkstra bug - it does not usually give a wrong answer, it just quietly re-expands stale states and blows up the runtime.

Minimum spanning trees

Theorem - Cut property
Let \((S,V\setminus S)\) be any nontrivial cut and let \(e\) be a minimum-weight edge crossing it. Then some MST contains \(e\) (if weights are distinct, every MST does).
Proof
Take an MST \(T\) not containing \(e=(u,v)\). Adding \(e\) to \(T\) creates a unique cycle, which must cross the cut a second time via some edge \(f\). Then \(T'=T-f+e\) is still spanning and connected, with weight \(w(T')=w(T)-w(f)+w(e)\le w(T)\) since \(w(e)\le w(f)\). So \(T'\) is also an MST, and it contains \(e\). □

Kruskal applies the cut property in increasing weight order (union–find tells you whether an edge would close a cycle); Prim applies it to the cut between the grown tree and the rest. Both are the same lemma, differently scheduled.

Worked Example - Dijkstra on a 5-vertex network, by hand
1
Edges (undirected): A-B 4, A-C 2, B-C 1, B-D 5, C-D 8, C-E 10, D-E 2. Source \(A\).
2
Init \(d=(A{:}0,\ B{:}\infty,\ C{:}\infty,\ D{:}\infty,\ E{:}\infty)\). Pop \(A\) (0). Relax: \(d[B]=4,\ d[C]=2\).
3
Pop the smallest unsettled: \(C\) (2). Relax: \(d[B]=\min(4,\ 2+1)=3\) (improved!), \(d[D]=\min(\infty,2+8)=10\), \(d[E]=2+10=12\).
4
Pop \(B\) (3). Relax: \(d[D]=\min(10,\ 3+5)=8\).
5
Pop \(D\) (8). Relax: \(d[E]=\min(12,\ 8+2)=10\). Pop \(E\) (10). Done.
6
Final: \(d=(0,3,2,8,10)\). Note \(B\) was improved AFTER its first tentative value - that is the lazy-deletion case, and the stale \((4,B)\) entry must be skipped when it surfaces.
7
MST of the same graph (Kruskal): sort B-C 1, A-C 2, D-E 2, A-B 4, B-D 5, C-D 8, C-E 10. Take B-C (1), A-C (2), D-E (2), skip A-B (would cycle A-C-B), take B-D (5). Four edges, five vertices ✓. Total weight \(1+2+2+5=10\).

Coding workbench

The Euler Lab’s Graphs & Paths problems are almost all of the form ‘minimise a cost through a grid or a network’ (Dijkstra or DP on a DAG) or ‘find the cheapest connected subnetwork’ (MST). The hard part is never the algorithm - it is noticing that the object in the problem statement is a graph.

Common Mistakes to Avoid
  • Using BFS for shortest paths in a WEIGHTED graph. It computes hop counts, not distances (unless every weight is equal).
  • Running Dijkstra with negative edges. Use Bellman–Ford; Dijkstra’s exchange argument fails at the step \(\delta(s,y)\le\delta(s,u)\).
  • Forgetting the if d > dist[u]: continue guard. The answer often survives, but the runtime degrades badly on dense graphs.
  • Marking BFS vertices as visited at dequeue time instead of enqueue time - leads to duplicate queue entries and a quadratic blow-up.
  • Implementing union–find without path compression or union by rank; the worst case degrades to \(O(n)\) per query and Kruskal becomes quadratic.
Quant Practitioner Tips
  • Write the graph down as (vertex = ?, edge = ?, weight = ?) in words BEFORE you code. Ninety percent of graph bugs are modelling bugs.
  • A grid is a graph with 4 (or 8) implicit edges per cell - do not build an edge list, just generate neighbours on the fly.
  • 0–1 BFS (a deque, push-front for weight-0 edges) gives Dijkstra’s answer in \(O(V+E)\) when weights are only 0 or 1. Worth knowing.
  • If the graph is a DAG (states only ever move ‘forward’), skip Dijkstra: relax in topological order for \(O(V+E)\). Most DP tables are exactly this.

Practice this in the Euler Lab

Computational problems that exercise exactly this technique. Each opens in the Euler Lab with a Python workbench, a progressive hint ladder, and answer checking. Tier A/B run at full scale in the browser.

Warm-up:
#11 Largest Product in a Grid (5%, tier A) #15 Lattice Paths (6%, tier A) #112 Bouncy Numbers (6%, tier A) #18 Maximum Path Sum I (7%, tier A)

Applied:
#504 Square on the Inside (16%, tier B) #577 Counting Hexagons (18%, tier B) #186 Connectedness of a Network (19%, tier B)

Challenge:
#161 Triominoes (41%, tier C) #184 Triangles Containing the Origin (41%, tier C)

160 Project Euler problems in total are mapped to this lesson. Open the Euler Lab to filter them all.

Knowledge Check

Q1 Medium
Why does Dijkstra fail with negative edge weights?
The heap breaks
A vertex settled as ‘nearest’ may later be improved through a negative edge, violating the exchange argument
It becomes too slow
It only fails on directed graphs
Q2 Easy
BFS from \(s\) gives shortest paths when:
Always
All edge weights are equal (e.g. unweighted)
The graph is a tree
Weights are non-negative
Q3 Medium
The cut property says that for any cut, the minimum-weight crossing edge:
Is in every spanning tree
Is in some MST
Creates a cycle
Is the heaviest edge in some cycle

Practical Exercise

(a) You are given an \(n\times n\) grid of positive costs and must move from the top-left to the bottom-right, paying the cost of each cell you enter, moving in all four directions. Which algorithm, and what is the complexity? What changes if you may only move right and down? (b) Prove that in a graph with distinct edge weights the MST is unique. (c) Show that Dijkstra with a heap is \(O(E\log V)\), not \(O(V^2)\), and say when the \(O(V^2)\) array version is actually better.

▶ Show full solution

(a) Four-directional movement means the state graph has cycles, so it is a genuine shortest-path problem: Dijkstra with vertices = cells, edges = the ≤4 neighbours, weight = the cost of the cell being entered. Cost: \(V=n^2\), \(E\le4n^2\), so \(O(n^2\log n)\). If you may only move right and down, the state graph is a DAG (every move increases \(i+j\)), so you can relax in topological order - which is just the DP \(D[i][j]=c[i][j]+\min(D[i-1][j],\ D[i][j-1])\) - in \(O(n^2)\) with no heap at all. Recognising the DAG saves the log factor and a lot of code.

(b) Suppose \(T_1\ne T_2\) are both MSTs and all weights are distinct. Let \(e\) be the minimum-weight edge in exactly one of them, say \(e\in T_1\setminus T_2\). Adding \(e\) to \(T_2\) creates a cycle \(C\); since \(T_1\) is acyclic, some edge \(f\in C\) is not in \(T_1\). Then \(f\) is in exactly one of the trees too, so by the minimality of \(e\), \(w(e)\lt w(f)\) (strict, by distinctness). Swapping gives \(T_2-f+e\), a spanning tree of strictly smaller weight than \(T_2\) - contradicting that \(T_2\) is an MST. Hence \(T_1=T_2\).

(c) Each edge can trigger at most one push, so the heap holds \(O(E)\) entries and each push/pop costs \(O(\log E)=O(\log V^2)=O(2\log V)=O(\log V)\). Total \(O((V+E)\log V)\). The simple array version scans all \(V\) vertices to find the minimum each round: \(O(V^2+E)\). On a DENSE graph with \(E=\Theta(V^2)\), the array version’s \(O(V^2)\) beats the heap’s \(O(V^2\log V)\) - so for dense graphs, drop the heap.

After the reveal, answer for yourself: In (a), noticing the DAG structure removed the need for a shortest-path algorithm entirely. What is the general lesson about looking at your state graph before choosing an algorithm?

Lesson Summary

Model first: name the vertex, the edge, and the weight. Then BFS gives unweighted shortest paths and DFS gives structure (cycles, topological order), both in \(O(V+E)\); Dijkstra gives shortest paths for non-negative weights in \(O(E\log V)\) by a greedy exchange argument that fails the instant an edge goes negative; and the cut property gives minimum spanning trees via Kruskal (union–find) or Prim. If your state graph is a DAG, relax in topological order and skip the heap - that is just DP.

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers
Q: Where exactly does Dijkstra's correctness proof use non-negativity?
A: In the step \(\delta(s,y)\le\delta(s,u)\) for the frontier vertex \(y\) on a shortest path to \(u\): the rest of the path can only add non-negative length. A negative edge invalidates it, so use Bellman–Ford.
Q: State the cut property and which two algorithms it justifies.
A: For any cut, a minimum-weight crossing edge lies in some MST (exchange argument). It justifies Kruskal (apply it in increasing weight order) and Prim (apply it to the tree/rest cut).

Completion Checklist

Confidence / mastery rating
Personal notes

Source References

This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.