Skip to content

Commit 5ca8e24

Browse files
jordanmonttjordanmontt
authored
Merge pull request #7 from Driolar/master
Added Prim's algorithm
2 parents d074fff + 47b4084 commit 5ca8e24

File tree

1 file changed

+192
-69
lines changed

1 file changed

+192
-69
lines changed

‎Chapters/Chapter6/chapter6.md‎

Lines changed: 192 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -1,69 +1,192 @@
1-
## Minimum spanning treesThe minimum spanning tree is a subset of the edges of a undirected weighted graph that connects all the nodes of the graphwithout any cycles and with the total sum of the weights minimized. There are several algorithms for obtaining the minimum spanningtree of a weighted graph, the most famous is the Kruskal's algorithm. In the case of an disconnected graph, the algorithmreturns a minimum spanning forest \(i.e., it will return a list a minimum spanning trees\)We will show you how to implement Kruskal but before that we have to introduce a little data-structure called _Disjoint-Set_.### Motivating scenarioImagine that you have a telecommunication company and you want to build a connection between different neighbourhoods.Some of the connections are more expensive than others. For example, one connection has to pass under the ground or above some mountains.So, you have a graph in which the nodes represent the different neighbourhoods and the edges represent all thepossible cables that can be built to make the connections between the neighbourhoods. The weights represent the cost ofactually building the connection.Imagine that we have the graph shown in Figure *@spanningsmall@*, we would like to get the tree that allows us to get all the nodes of the graph without circle as shown in Figure *@spanningsmall@*.![Connections costs between neighbourhoods.](figures/kruskal.pdf width=35&label=spanningsmall)![Minimum spanning tree with C as root.](figures/minimum_spanning_tree.pdf width=35&label=spanningsmall)### Disjoint-Set data structureA disjoint-set, also called union-find data structure, is a data structure that stores disjoint sets. It provides two operations:- _unite_ that groups two disjoint sets into one and- _find_ that returns two elements belong to the same disjoint set.For example, in Figure *@union_find_set@* we have two sets of elements $\{A, B, C\}$ and $\{D, E\}$. If we call the operation find with A and D nodes,as they do not belong to the same set, the operation will return false. With A and B nodes, the operation find will return true.![Two Union-Find sets.](figures/union_find.pdf width=35&label=union_find_set)But when we invoke the operation unite with A and D, it will join the two sets to have only one set with all the elements, as in Figure *@only_one_set@*.![Union-Find set: result of the unite operation between A and D. ](figures/only_one_set.pdf width=35&label=only_one_set)This data structure is used in Kruskal's algorithm to detect if adding a new edge creates a cycle in the minimum spanning tree that is being built.The time complexity of both of the operations is $O(a(n))$, where $a$ is the amortized time complexity. Each time thatthe `find:` method is invoked an operation called _path compression_.This is due to the path compression operation that this data structure has an amortized linear time complexity.In Pharo, this data structure represents a node in the Kruskal's graph algorithm.```AIDisjointSetNode >> union: aDSNode
2-
3-
| root1 root2 |
4-
root1 := aDSNode find.
5-
root2 := self find.
6-
7-
root1 = root2 ifTrue: [
8-
"The nodes already belong to the same component"
9-
^ self ].
10-
11-
root1 parent: root2``````AIDisjointSetNode >> find
12-
"Return the root of the component but modifying the parent/child structure during the process of finding a root."
13-
14-
| root next node |
15-
node := self.
16-
root := node.
17-
[ root = root parent ] whileFalse: [ root := root parent ].
18-
19-
"Compress the path leading back to the root.
20-
This is the path compression operation that gives the linear amortized time complexity"
21-
[ node = root ] whileFalse: [
22-
next := node parent.
23-
node parent: root.
24-
node := next ].
25-
26-
^ root```### Kruskal's algorithmAs said above, the Kruskal's algorithm calculates the minimum spanning tree \(or forest\) of an undirected weighted graph.The algorithm has a time complexity of $O(V*log(E)) = O(E*log(E))$. This time complexity is achieved thanks to theDisjoint-Set data structure. This algorithm uses the Disjoint-Set data structure to check if adding an edge to the spanningtree creates a cycle.The pseudocode is:```1. Sort edges in ascending weight.
27-
2. Pick the smallest edge.
28-
Check if its two nodes are already unified.
29-
If they are not, unified them and include the edge to the spanning tree.
30-
Else, discard it.
31-
3. Repeat step 2 until there are all nodes are connected.```This is the implementation of the algorithm in Pharo:```AIKruskal >> run
32-
33-
| treeEdges sortedEdges |
34-
sortBlock := [ :e1 :e2 | e1 weight < e2 weight ].
35-
treeEdges := OrderedCollection new.
36-
nodes do: [ :node | node makeSet].
37-
sortedEdges := edges asSortedCollection: sortBlock.
38-
sortedEdges
39-
reject: [ :edge |
40-
"Only join the two nodes if they don't belong to the same component"
41-
edge from find = edge to find ]
42-
thenDo: [ :edge |
43-
edge from union: edge to.
44-
treeEdges add: edge ].
45-
^ treeEdges```### Kruskal's algorithm for maximum spanning treeAs contrary to the minimum spanning tree, the maximum spanning tree of a graph is a subset of edges of a graph that connects all nodes with the **maximum** possible distance.This is exactly the same algorithm except that we have to order the edges in descending weight instead of ascending.```1. Sort edges in descending weight.
46-
2. Pick the biggest edge...```In the implementation we only need to change one line:```sortBlock := [ :e1 :e2 | e1 weight > e2 weight ].```### Case studyWe can now apply our algorithm to the graph shown at the beginning of this chapter in Figure *@spsituation@*.![Connections costs between neighbourhoods.](figures/kruskal.pdf width=55&label=spsituation)So, like in the other graph algorithms we only need to declare the nodes and the edges an then call the method `run`to obtain the result.```nodes := $A to: $J.
47-
edges := #( #( $A $B 25 ) #( $A $D 8 ) #( $A $F 11 ) #( $B $A 25 )
48-
#( $B $E 1 ) #( $B $C 12 ) #( $C $B 12 ) #( $C $D 16 )
49-
#( $C $F 6 ) #( $C $G 9 ) #( $D $A 8 ) #( $D $C 16 )
50-
#( $E $B 1 ) #( $E $G 14 ) #( $F $A 11 ) #( $F $C 6 )
51-
#( $F $G 5 ) #( $F $J 4 ) #( $G $F 5 ) #( $G $C 9 )
52-
#( $G $E 14 ) #( $G $H 7 ) #( $H $G 7 ) #( $I $J 7 )
53-
#( $J $F 4 ) #( $J $I 7 ) ).
54-
kruskal := AIKruskal new.
55-
kruskal nodes: nodes.
56-
kruskal
57-
edges: edges
58-
from: #first
59-
to: #second
60-
weight: #third.
61-
minimumSpanningTree := kruskal run```If we inspect the `minimumSpanningTree` variable, we get a collection the edges of the minimum spanning tree._DSN_ means _DisjointSetNode_.```DSN $B -> DSN $E weight: 1
62-
DSN $J -> DSN $F weight: 4
63-
DSN $F -> DSN $G weight: 5
64-
DSN $F -> DSN $C weight: 6
65-
DSN $I -> DSN $J weight: 7
66-
DSN $H -> DSN $G weight: 7
67-
DSN $A -> DSN $D weight: 8
68-
DSN $A -> DSN $F weight: 11
69-
DSN $C -> DSN $B weight: 12```![Minimum spanning tree.](figures/minimum_spanning_tree.pdf width=55)If we want to obtain the maximum spanning tree, we only need to call the `maxSpanningTree` methodwhen creating the graph algorithm.```kruskal := AIKruskal new maxSpanningTree.```### ConclusionData structures play a powerful role when it comes to algorithms. In this specific case thank to the Disjoint-Set data structure we can detect cycles in amortizedlinear time complexity with few lines of code. Also, the Kruskal algorithm has many real life applications and it is an important algorithm in the context of graph theory.
1+
## Minimum Spanning Tree
2+
3+
The minimum spanning tree is a subset of the edges of an undirected weighted graph that connects all the nodes of the graph without any cycles and with the total sum of the weights minimized. There are several algorithms for obtaining the minimum spanning tree of a weighted graph, the most famous is the Kruskal's algorithm. In the case of a disconnected graph, the algorithm returns a minimum spanning forest \(i.e., it will return a list a minimum spanning trees\). The other most commonly used algorithm is Prim's algorithm.
4+
5+
We will first show you how to implement Kruskal's algorithm but before, we introduce a little data-structure called _disjoint-set_ that decisively contributes to Kruskal's algorithm efficiency.
6+
7+
### Motivating scenario
8+
9+
Imagine that you have a telecommunication company and you want to build a connection between different neighbourhoods. Some of the connections are more expensive than others. For example, one connection has to pass under the ground or above some mountains. So, you have a graph in which the nodes represent the different neighbourhoods and the edges represent all the possible cables that can be built to make the connections between the neighbourhoods. The weights represent the cost of actually building the connection.
10+
11+
Imagine that we have the graph shown in Figure *@inputgraph@* and we would like to get the tree that allows us to get all the nodes of the graph without cycle as shown in Figure *@outputtree@*.
12+
13+
![Connections costs between neighbourhoods.](figures/kruskal.pdf width=35&label=inputgraph)
14+
15+
![Minimum spanning tree with node C as root.](figures/minimum_spanning_tree.pdf width=35&label=outputtree)
16+
17+
### Disjoint-set data structure
18+
19+
A disjoint-set, also called union-find data structure, is a data structure that stores disjoint sets. It provides two methods:
20+
21+
- _union_ groups two disjoint sets into one and
22+
- _find_ answers whether two elements belong to the same set or not.
23+
24+
For example, in Figure *@union_find_set@* we have two sets of elements $\{A, B, C\}$ and $\{D, E\}$. If we call the method *find* with A and D nodes, as they do not belong to the same set, the method will return false. With A and B nodes, the method *find* will return true.
25+
![Two Union-Find sets.](figures/union_find.pdf width=35&label=union_find_set)
26+
27+
Further, when we invoke the method *union* with A and D, it will join the two sets to have only one set with all the elements, as in Figure *@only_one_set@*.
28+
29+
![Union-Find set: result of the union operation between A and D. ](figures/only_one_set.pdf width=35&label=only_one_set)
30+
31+
A particular implementation of this data structure is the so-called *disjoint-set forest*, which executes in the `find` method an operation called **path compression**. Thanks to this operation, a loop with few code, the time complexity of the methods `find` and `union` is nearly constant. Namely, It has been proven that with path compression, the *amortized* time complexity of both methods is $O(\alpha(n))$, where $\alpha$ is the Ackermann function's inverse, which grows very slowly remaining less than 5 for any practical input size.
32+
33+
In our Pharo library, the disjoint-set forest (with path compression) is supported by the node class `AIDisjointSetNode`.
34+
35+
```
36+
AIDisjointSetNode >> union: aDSNode
37+
38+
| root1 root2 |
39+
root1 := aDSNode find.
40+
root2 := self find.
41+
42+
root1 = root2 ifTrue: [
43+
"The nodes already belong to the same component"
44+
^ self ].
45+
46+
root1 parent: root2
47+
```
48+
49+
```
50+
AIDisjointSetNode >> find
51+
"Return the root of the component but modifying the parent/child structure during the process of finding a root."
52+
53+
| root next node |
54+
node := self.
55+
root := node.
56+
[ root = root parent ] whileFalse: [ root := root parent ].
57+
58+
"Compress the path leading back to the root.
59+
This is the path compression operation that gives the linear amortized time complexity"
60+
[ node = root ] whileFalse: [
61+
next := node parent.
62+
node parent: root.
63+
node := next ].
64+
65+
^ root
66+
```
67+
68+
### Kruskal's algorithm
69+
70+
As said above, the Kruskal's algorithm calculates the minimum spanning tree \(or forest\) of an undirected weighted graph. The algorithm uses the disjoint-set data structure to check if adding an edge to the spanning tree creates a cycle.
71+
72+
Kruskal's algorithm pseudocode is:
73+
74+
```
75+
1. Sort edges in ascending weight.
76+
2. Pick the smallest edge.
77+
Check if its two nodes are already unified.
78+
If they are not, unified them and include the edge to the spanning tree.
79+
Else, discard it.
80+
3. Repeat step 2 until there are all nodes are connected.
81+
```
82+
83+
This is the implementation of the algorithm in Pharo:
84+
85+
```
86+
AIKruskal >> run
87+
88+
| treeEdges sortedEdges |
89+
  sortBlock := [ :e1 :e2 | e1 weight < e2 weight ].
90+
treeEdges := OrderedCollection new.
91+
sortedEdges := edges asSortedCollection: sortBlock.
92+
sortedEdges
93+
reject: [ :edge |
94+
"Only join the two nodes if they don't belong to the same component"
95+
edge from find = edge to find ]
96+
thenDo: [ :edge |
97+
edge from union: edge to.
98+
treeEdges add: edge ].
99+
^ treeEdges
100+
```
101+
102+
The algorithm has a time complexity of $O(E * log(E))$ due to the initial edge sort. The rest has linear time complexity thanks to the disjoint-set data structure.
103+
104+
### Kruskal's algorithm for maximum spanning tree
105+
106+
As opposite to the minimum spanning tree, the maximum spanning tree of a graph is a subset of edges of a graph that connects all nodes with the **maximum** possible distance.
107+
108+
This is exactly the same algorithm except that we have to order the edges in descending weight instead of ascending.
109+
110+
```
111+
1. Sort edges in descending weight.
112+
2. Pick the biggest edge...
113+
```
114+
115+
In the implementation we only need to change one line:
116+
117+
```
118+
sortBlock := [ :e1 :e2 | e1 weight > e2 weight ].
119+
```
120+
121+
### Case study
122+
123+
We can now apply our algorithm to the graph in Figure *@spsituation@*, already shown at the beginning of this chapter in Figure *@inputgraph@*.
124+
125+
![Connections costs between neighbourhoods.](figures/kruskal.pdf width=55&label=spsituation)
126+
127+
So, like in the other graph algorithms we only need to declare the nodes and the edges an then call the method `run` to obtain the result.
128+
129+
```
130+
nodes := $A to: $J.
131+
edges := #( #( $A $B 25 ) #( $A $D 8 ) #( $A $F 11 ) #( $B $A 25 )
132+
#( $B $E 1 ) #( $B $C 12 ) #( $C $B 12 ) #( $C $D 16 )
133+
#( $C $F 6 ) #( $C $G 9 ) #( $D $A 8 ) #( $D $C 16 )
134+
#( $E $B 1 ) #( $E $G 14 ) #( $F $A 11 ) #( $F $C 6 )
135+
#( $F $G 5 ) #( $F $J 4 ) #( $G $F 5 ) #( $G $C 9 )
136+
#( $G $E 14 ) #( $G $H 7 ) #( $H $G 7 ) #( $I $J 7 )
137+
#( $J $F 4 ) #( $J $I 7 ) ).
138+
kruskal := AIKruskal new.
139+
kruskal nodes: nodes.
140+
kruskal
141+
edges: edges
142+
from: #first
143+
to: #second
144+
weight: #third.
145+
minimumSpanningTree := kruskal run
146+
```
147+
148+
If we inspect the `minimumSpanningTree` variable, we get a collection the edges of the minimum spanning tree. DSN means *DisjointSetNode*.
149+
150+
```
151+
DSN $B -> DSN $E weight: 1
152+
DSN $J -> DSN $F weight: 4
153+
DSN $F -> DSN $G weight: 5
154+
DSN $F -> DSN $C weight: 6
155+
DSN $I -> DSN $J weight: 7
156+
DSN $H -> DSN $G weight: 7
157+
DSN $A -> DSN $D weight: 8
158+
DSN $A -> DSN $F weight: 11
159+
DSN $C -> DSN $B weight: 12
160+
```
161+
162+
![Minimum spanning tree.](figures/minimum_spanning_tree.pdf width=55)
163+
164+
If we want to obtain the maximum spanning tree, we only need to call the `maxSpanningTree` method when creating the graph algorithm.
165+
166+
```
167+
kruskal := AIKruskal new maxSpanningTree.
168+
```
169+
170+
### Prim's algorithm
171+
172+
Basically, Prim’s algorithm to find a minimum spanning tree is a modified version of Dijkstra’s algorithm for the shortest path problem. The algorithm starts with a single node and moves through several adjacent nodes, in order to explore all of the connected edges along the way.
173+
174+
Prim's algorithm pseudocode is:
175+
176+
```
177+
1. Initialize a priority queue (or a min-heap) to store edges.
178+
2. Start from an arbitrary node and add all its edges to the priority queue.
179+
3. While the priority queue is not empty:
180+
       Extract the edge with the minimum weight.
181+
       If the edge connects to a node not already in the tree, add it to the tree and add its edges to the priority queue.
182+
```
183+
184+
Again, it is decisive to use opportune data structures. By using a priority queue or a sophisticated heap to store the candidate edges, we can considerably improve the inner loop and achieve $O(E * log(V)) $ time complexity overall. Otherwise, we would clash with quadratic time complexity or worse.
185+
186+
### Conclusion
187+
188+
Both the Kruskal's and the Prim’s algorithm are greedy algorithms that find the minimum spanning tree for a connected, undirected graph. Both have many real life applications and are important in the context of graph theory.
189+
190+
The difference betweeen both algorithms lies in how the minimum spanning tree is constructed. Kruskal's algorithm sorts the edges according to their weights and adds them in ascending order. On the other hand, Prim's algorithm starts with one node. Starting from this node, a spanning tree is gradually formed until all nodes are considered. On this basis, their time complexity can attain, with the appropiate data structures, $O(E * log(E))$ respectively $O(E * log(V))$. In principle, because of this difference, Kruskal's algorithm is rather suitable for sparse graphs (with few edges) while Prim's algorithm should be preferred for dense graphs (with a lot of edges).
191+
192+
Data structures play a powerful role when it comes to algorithms. In this specific case, Kruskal's algorithm detects cycles in amortized linear time complexity with few lines of code thanks to the disjoint-set data structure and Prim's algorithm benefits from a well-designed priority queue or heap.

0 commit comments

Comments
 (0)