Skip to content

Commit 1320e21

Browse files
pjhartoutpjhartout
authored
add results tables to docs and README.md (#37)
* add results tables * add tables to the paper
1 parent 2479140 commit 1320e21

File tree

2 files changed

+513
-3
lines changed

2 files changed

+513
-3
lines changed

‎README.md‎

Lines changed: 255 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,62 @@
1212
</picture>
1313
</p>
1414

15-
# PolyGraph
16-
1715
PolyGraph is a Python library for evaluating graph generative models by providing standardized datasets and metrics
18-
(including PolyGraphDiscrepancy).
16+
(including PolyGraph Discrepancy).
17+
18+
PolyGraph discrepancy is a new metric we introduced, which provides the following advantages over maxmimum mean discrepancy (MMD):
19+
<style>
20+
table {
21+
font-size: 90%;
22+
margin: 0 auto;
23+
}
24+
th, td {
25+
text-align: center;
26+
padding: 4px 8px;
27+
}
28+
th:first-child, td:first-child {
29+
text-align: left;
30+
}
31+
</style>
32+
33+
<table>
34+
<thead>
35+
<tr>
36+
<th>Property</th>
37+
<th>MMD</th>
38+
<th>PGD</th>
39+
</tr>
40+
</thead>
41+
<tbody>
42+
<tr>
43+
<td>Range</td>
44+
<td>[0, ∞)</td>
45+
<td>[0, 1]</td>
46+
</tr>
47+
<tr>
48+
<td>Intrinsic Scale</td>
49+
<td style="color:red;">❌</td>
50+
<td style="color:green;">✅</td>
51+
</tr>
52+
<tr>
53+
<td>Descriptor Comparison</td>
54+
<td style="color:red;">❌</td>
55+
<td style="color:green;">✅</td>
56+
</tr>
57+
<tr>
58+
<td>Multi-Descriptor Aggregation</td>
59+
<td style="color:red;">❌</td>
60+
<td style="color:green;">✅</td>
61+
</tr>
62+
<tr>
63+
<td>Single Ranking</td>
64+
<td style="color:red;">❌</td>
65+
<td style="color:green;">✅</td>
66+
</tr>
67+
</tbody>
68+
</table>
69+
70+
It also provides a number of other advantages over MMD which we discuss in our paper.
1971

2072
## Installation
2173

@@ -61,6 +113,206 @@ We provide larger datasets that should be used instead:
61113

62114
</details>
63115

116+
## Benchmarking snapshot
117+
118+
Here is an example benchmark one can generate with this library using multiple different models and datasets. The details of the generation of this benchmark are given in our paper.
119+
120+
<style>
121+
table {
122+
font-size: 85%;
123+
border-collapse: collapse;
124+
}
125+
th, td {
126+
text-align: center;
127+
padding: 4px 6px;
128+
border: 1px solid #ddd;
129+
}
130+
th:first-child, td:first-child {
131+
text-align: left;
132+
}
133+
th:nth-child(2), td:nth-child(2) {
134+
text-align: left;
135+
}
136+
</style>
137+
138+
<table>
139+
<thead>
140+
<tr>
141+
<th rowspan="2">Dataset</th>
142+
<th rowspan="2">Model</th>
143+
<th rowspan="2">VUN (↑)</th>
144+
<th rowspan="2">PGD (↓)</th>
145+
<th colspan="6">PGD subscores</th>
146+
</tr>
147+
<tr>
148+
<th>Clust. (↓)</th>
149+
<th>Deg. (↓)</th>
150+
<th>GIN (↓)</th>
151+
<th>Orb5. (↓)</th>
152+
<th>Orb4. (↓)</th>
153+
<th>Eig. (↓)</th>
154+
</tr>
155+
</thead>
156+
<tbody>
157+
158+
<!-- Planar-L -->
159+
<tr><td rowspan="4"><b>Planar-L</b></td>
160+
<td>AutoGraph</td>
161+
<td><i>85.1</i></td>
162+
<td><b>34.0 ± 1.8</b></td>
163+
<td><b>7.0 ± 2.9</b></td>
164+
<td><b>7.8 ± 3.2</b></td>
165+
<td><b>8.8 ± 3.0</b></td>
166+
<td><b>34.0 ± 1.8</b></td>
167+
<td><b>28.5 ± 1.5</b></td>
168+
<td><b>26.9 ± 2.3</b></td></tr>
169+
<tr><td>DiGress</td>
170+
<td>80.1</td>
171+
<td>45.2 ± 1.8</td>
172+
<td>24.8 ± 2.0</td>
173+
<td>23.3 ± 1.2</td>
174+
<td><i>29.0 ± 1.1</i></td>
175+
<td>45.2 ± 1.8</td>
176+
<td><i>40.3 ± 1.8</i></td>
177+
<td>39.4 ± 2.0</td></tr>
178+
<tr><td>GRAN</td>
179+
<td>1.6</td>
180+
<td>99.7 ± 0.2</td>
181+
<td>99.3 ± 0.2</td>
182+
<td>98.3 ± 0.3</td>
183+
<td>98.3 ± 0.3</td>
184+
<td>99.7 ± 0.1</td>
185+
<td>99.2 ± 0.2</td>
186+
<td>98.5 ± 0.4</td></tr>
187+
<tr><td>ESGG</td>
188+
<td><b>93.9</b></td>
189+
<td><i>45.0 ± 1.4</i></td>
190+
<td><i>10.9 ± 3.2</i></td>
191+
<td><i>21.7 ± 3.0</i></td>
192+
<td>32.9 ± 2.2</td>
193+
<td><i>45.0 ± 1.4</i></td>
194+
<td>42.8 ± 1.9</td>
195+
<td><i>29.6 ± 1.6</i></td></tr>
196+
197+
<!-- Lobster-L -->
198+
<tr><td rowspan="4"><b>Lobster-L</b></td>
199+
<td>AutoGraph</td>
200+
<td><i>83.1</i></td>
201+
<td><i>18.0 ± 1.6</i></td>
202+
<td>4.2 ± 1.9</td>
203+
<td><i>12.1 ± 1.6</i></td>
204+
<td><i>14.8 ± 1.5</i></td>
205+
<td><i>18.0 ± 1.6</i></td>
206+
<td><i>16.1 ± 1.6</i></td>
207+
<td><i>13.0 ± 1.1</i></td></tr>
208+
<tr><td>DiGress</td>
209+
<td><b>91.4</b></td>
210+
<td><b>3.2 ± 2.6</b></td>
211+
<td><i>2.0 ± 1.3</i></td>
212+
<td><b>1.2 ± 1.5</b></td>
213+
<td><b>2.3 ± 2.0</b></td>
214+
<td><b>3.0 ± 3.1</b></td>
215+
<td><b>4.5 ± 2.3</b></td>
216+
<td><b>1.3 ± 1.1</b></td></tr>
217+
<tr><td>GRAN</td>
218+
<td>41.3</td>
219+
<td>85.4 ± 0.5</td>
220+
<td>20.8 ± 1.1</td>
221+
<td>77.1 ± 1.2</td>
222+
<td>79.8 ± 0.6</td>
223+
<td>85.4 ± 0.5</td>
224+
<td>85.0 ± 0.6</td>
225+
<td>69.8 ± 1.2</td></tr>
226+
<tr><td>ESGG</td>
227+
<td>70.9</td>
228+
<td>69.9 ± 0.6</td>
229+
<td><b>0.0 ± 0.0</b></td>
230+
<td>63.4 ± 1.1</td>
231+
<td>66.8 ± 1.0</td>
232+
<td>69.9 ± 0.6</td>
233+
<td>66.0 ± 0.6</td>
234+
<td>51.7 ± 1.8</td></tr>
235+
236+
<!-- SBM-L -->
237+
<tr><td rowspan="4"><b>SBM-L</b></td>
238+
<td>AutoGraph</td>
239+
<td><b>85.6</b></td>
240+
<td><b>5.6 ± 1.5</b></td>
241+
<td><b>0.3 ± 0.6</b></td>
242+
<td><b>6.2 ± 1.4</b></td>
243+
<td><b>6.3 ± 1.3</b></td>
244+
<td><b>3.2 ± 2.2</b></td>
245+
<td><b>4.4 ± 2.0</b></td>
246+
<td><b>2.5 ± 2.2</b></td></tr>
247+
<tr><td>DiGress</td>
248+
<td><i>73.0</i></td>
249+
<td><i>17.4 ± 2.3</i></td>
250+
<td><i>5.7 ± 2.8</i></td>
251+
<td><i>8.2 ± 3.3</i></td>
252+
<td><i>13.8 ± 1.7</i></td>
253+
<td><i>17.4 ± 2.3</i></td>
254+
<td><i>14.8 ± 2.5</i></td>
255+
<td><i>8.7 ± 3.0</i></td></tr>
256+
<tr><td>GRAN</td>
257+
<td>21.4</td>
258+
<td>69.1 ± 1.4</td>
259+
<td>50.2 ± 1.9</td>
260+
<td>58.6 ± 1.4</td>
261+
<td>69.1 ± 1.4</td>
262+
<td>65.7 ± 1.3</td>
263+
<td>62.8 ± 1.3</td>
264+
<td>55.9 ± 1.5</td></tr>
265+
<tr><td>ESGG</td>
266+
<td>10.4</td>
267+
<td>99.4 ± 0.2</td>
268+
<td>97.9 ± 0.5</td>
269+
<td>97.5 ± 0.6</td>
270+
<td>98.3 ± 0.4</td>
271+
<td>96.8 ± 0.4</td>
272+
<td>89.2 ± 0.7</td>
273+
<td>99.4 ± 0.2</td></tr>
274+
275+
<!-- Proteins -->
276+
<tr><td rowspan="4"><b>Proteins</b></td>
277+
<td>AutoGraph</td>
278+
<td>–</td>
279+
<td><b>67.7 ± 7.4</b></td>
280+
<td><i>47.7 ± 5.7</i></td>
281+
<td><i>31.5 ± 8.5</i></td>
282+
<td><i>45.3 ± 5.1</i></td>
283+
<td><b>67.7 ± 7.4</b></td>
284+
<td><b>47.4 ± 7.0</b></td>
285+
<td>53.2 ± 6.9</td></tr>
286+
<tr><td>DiGress</td>
287+
<td>–</td>
288+
<td>88.1 ± 3.1</td>
289+
<td><b>36.1 ± 4.3</b></td>
290+
<td><b>29.2 ± 5.0</b></td>
291+
<td><b>23.2 ± 5.3</b></td>
292+
<td>88.1 ± 3.1</td>
293+
<td><i>60.8 ± 3.6</i></td>
294+
<td><b>23.4 ± 11.8</b></td></tr>
295+
<tr><td>GRAN</td>
296+
<td>–</td>
297+
<td>89.7 ± 2.7</td>
298+
<td>86.0 ± 2.0</td>
299+
<td>70.6 ± 3.1</td>
300+
<td>71.5 ± 3.0</td>
301+
<td>90.4 ± 2.4</td>
302+
<td>84.4 ± 3.3</td>
303+
<td>76.7 ± 4.7</td></tr>
304+
<tr><td>ESGG</td>
305+
<td>–</td>
306+
<td><i>79.2 ± 4.3</i></td>
307+
<td>58.2 ± 3.6</td>
308+
<td>54.0 ± 3.6</td>
309+
<td>57.4 ± 4.1</td>
310+
<td><i>80.2 ± 3.1</i></td>
311+
<td>72.5 ± 3.0</td>
312+
<td><i>24.3 ± 11.0</i></td></tr>
313+
314+
</tbody>
315+
</table>
64316

65317
## Tutorial
66318

0 commit comments

Comments
 (0)