|
3 | 3 | [](https://travis-ci.org/JuliaStats/Distances.jl)
|
4 | 4 | [](https://coveralls.io/github/JuliaStats/Distances.jl?branch=master)
|
5 | 5 |
|
6 |
| -[](http://pkg.julialang.org/?pkg=Distances&ver=0.4) |
7 | 6 | [](http://pkg.julialang.org/?pkg=Distances)
|
8 | 7 |
|
9 | 8 | A Julia package for evaluating distances(metrics) between vectors.
|
@@ -125,32 +124,32 @@ This type system has practical significance. For example, when computing pairwis
|
125 | 124 |
|
126 | 125 | Each distance corresponds to a distance type. The type name and the corresponding mathematical definitions of the distances are listed in the following table.
|
127 | 126 |
|
128 |
| -| type name | convenient syntax | math definition | |
129 |
| -| -------------------- | ------------------------ | --------------------| |
130 |
| -| Euclidean | euclidean(x, y) | sqrt(sum((x - y) .^ 2)) | |
131 |
| -| SqEuclidean | sqeuclidean(x, y) | sum((x - y).^2) | |
132 |
| -| Cityblock | cityblock(x, y) | sum(abs(x - y)) | |
133 |
| -| Chebyshev | chebyshev(x, y) | max(abs(x - y)) | |
134 |
| -| Minkowski | minkowski(x, y, p) | sum(abs(x - y).^p) ^ (1/p) | |
135 |
| -| Hamming | hamming(x, y) | sum(x .!= y) | |
136 |
| -| Rogers-Tanimoto | rogerstanimoto(x, y) | 2(sum(x&!y) + sum(!x&y)) / (2(sum(x&!y) + sum(!x&y)) + sum(x&y) + sum(!x&!y)) | |
137 |
| -| Jaccard | jaccard(x, y) | 1 - sum(min(x, y)) / sum(max(x, y)) | |
138 |
| -| CosineDist | cosine_dist(x, y) | 1 - dot(x, y) / (norm(x) * norm(y)) | |
139 |
| -| CorrDist | corr_dist(x, y) | cosine_dist(x - mean(x), y - mean(y)) | |
140 |
| -| ChiSqDist | chisq_dist(x, y) | sum((x - y).^2 / (x + y)) | |
141 |
| -| KLDivergence | kl_divergence(x, y) | sum(p .* log(p ./ q)) | |
142 |
| -| RenyiDivergence | renyi_divergence(x, y, k)| log(sum( x .* (x ./ y) .^ (k - 1))) / (k - 1) | |
143 |
| -| JSDivergence | js_divergence(x, y) | KL(x, m) / 2 + KL(y, m) / 2 with m = (x + y) / 2 | |
144 |
| -| SpanNormDist | spannorm_dist(x, y) | max(x - y) - min(x - y ) | |
145 |
| -| BhattacharyyaDist | bhattacharyya(x, y) | -log(sum(sqrt(x .* y) / sqrt(sum(x) * sum(y))) | |
146 |
| -| HellingerDist | hellinger(x, y) | sqrt(1 - sum(sqrt(x .* y) / sqrt(sum(x) * sum(y)))) | |
147 |
| -| Mahalanobis | mahalanobis(x, y, Q) | sqrt((x - y)' * Q * (x - y)) | |
148 |
| -| SqMahalanobis | sqmahalanobis(x, y, Q) | (x - y)' * Q * (x - y) | |
149 |
| -| WeightedEuclidean | weuclidean(x, y, w) | sqrt(sum((x - y).^2 .* w)) | |
150 |
| -| WeightedSqEuclidean | wsqeuclidean(x, y, w) | sum((x - y).^2 .* w) | |
151 |
| -| WeightedCityblock | wcityblock(x, y, w) | sum(abs(x - y) .* w) | |
152 |
| -| WeightedMinkowski | wminkowski(x, y, w, p) | sum(abs(x - y).^p .* w) ^ (1/p) | |
153 |
| -| WeightedHamming | whamming(x, y, w) | sum((x .!= y) .* w) | |
| 127 | +| type name | convenient syntax | math definition | |
| 128 | +| -------------------- | -------------------------- | --------------------| |
| 129 | +| Euclidean | `euclidean(x, y)` | `sqrt(sum((x - y) .^ 2))` | |
| 130 | +| SqEuclidean | `sqeuclidean(x, y)` | `sum((x - y).^2)` | |
| 131 | +| Cityblock | `cityblock(x, y)` | `sum(abs(x - y))` | |
| 132 | +| Chebyshev | `chebyshev(x, y)` | `max(abs(x - y))` | |
| 133 | +| Minkowski | `minkowski(x, y, p)` | `sum(abs(x - y).^p) ^ (1/p)` | |
| 134 | +| Hamming | `hamming(x, y)` | `sum(x .!= y)` | |
| 135 | +| Rogers-Tanimoto | `rogerstanimoto(x, y)` | `2(sum(x&!y) + sum(!x&y)) / (2(sum(x&!y) + sum(!x&y)) + sum(x&y) + sum(!x&!y))` | |
| 136 | +| Jaccard | `jaccard(x, y)` | `1 - sum(min(x, y)) / sum(max(x, y))` | |
| 137 | +| CosineDist | `cosine_dist(x, y)` | `1 - dot(x, y) / (norm(x) * norm(y))` | |
| 138 | +| CorrDist | `corr_dist(x, y)` | `cosine_dist(x - mean(x), y - mean(y))` | |
| 139 | +| ChiSqDist | `chisq_dist(x, y)` | `sum((x - y).^2 / (x + y))` | |
| 140 | +| KLDivergence | `kl_divergence(x, y)` | `sum(p .* log(p ./ q))` | |
| 141 | +| RenyiDivergence | `renyi_divergence(x, y, k)`| `log(sum( x .* (x ./ y) .^ (k - 1))) / (k - 1)` | |
| 142 | +| JSDivergence | `js_divergence(x, y)` | `KL(x, m) / 2 + KL(y, m) / 2 with m = (x + y) / 2` | |
| 143 | +| SpanNormDist | `spannorm_dist(x, y)` | `max(x - y) - min(x - y )` | |
| 144 | +| BhattacharyyaDist | `bhattacharyya(x, y)` | `-log(sum(sqrt(x .* y) / sqrt(sum(x) * sum(y)))` | |
| 145 | +| HellingerDist | `hellinger(x, y) ` | `sqrt(1 - sum(sqrt(x .* y) / sqrt(sum(x) * sum(y))))` | |
| 146 | +| Mahalanobis | `mahalanobis(x, y, Q)` | `sqrt((x - y)' * Q * (x - y))` | |
| 147 | +| SqMahalanobis | `sqmahalanobis(x, y, Q)` | ` (x - y)' * Q * (x - y)` | |
| 148 | +| WeightedEuclidean | `weuclidean(x, y, w)` | `sqrt(sum((x - y).^2 .* w))` | |
| 149 | +| WeightedSqEuclidean | `wsqeuclidean(x, y, w)` | `sum((x - y).^2 .* w)` | |
| 150 | +| WeightedCityblock | `wcityblock(x, y, w)` | `sum(abs(x - y) .* w)` | |
| 151 | +| WeightedMinkowski | `wminkowski(x, y, w, p)` | `sum(abs(x - y).^p .* w) ^ (1/p)` | |
| 152 | +| WeightedHamming | `whamming(x, y, w)` | `sum((x .!= y) .* w)` | |
154 | 153 |
|
155 | 154 | **Note:** The formulas above are using *Julia*'s functions. These formulas are mainly for conveying the math concepts in a concise way. The actual implementation may use a faster way.
|
156 | 155 |
|
@@ -186,58 +185,62 @@ julia> pairwise(Euclidean(1e-12), x, x)
|
186 | 185 |
|
187 | 186 | The implementation has been carefully optimized based on benchmarks. The Julia scripts ``test/bench_colwise.jl`` and ``test/bench_pairwise.jl`` run the benchmarks on a variety of distances, respectively under column-wise and pairwise settings.
|
188 | 187 |
|
189 |
| -Here are the benchmarks that I obtained on Mac OS X 10.8 with Intel Core i7 2.6 GHz. |
| 188 | +Here are benchmarks obtained on Linux with Intel Core i7-4770K 3.5 GHz. |
190 | 189 |
|
191 | 190 | #### Column-wise benchmark
|
192 | 191 |
|
193 | 192 | The table below compares the performance (measured in terms of average elapsed time of each iteration) of a straightforward loop implementation and an optimized implementation provided in *Distances.jl*. The task in each iteration is to compute a specific distance between corresponding columns in two ``200-by-10000`` matrices.
|
194 | 193 |
|
195 |
| -| distance | loop | colwise | gain | |
196 |
| -|------------ | --------| ------------| -----------| |
197 |
| -| SqEuclidean | 0.046962 | 0.002782 | 16.8782 | |
198 |
| -| Euclidean | 0.046667 | 0.0029 | 16.0937 | |
199 |
| -| Cityblock | 0.046619 | 0.0031 | 15.039 | |
200 |
| -| Chebyshev | 0.053578 | 0.010856 | 4.9356 | |
201 |
| -| Minkowski | 0.061804 | 0.02357 | 2.6221 | |
202 |
| -| Hamming | 0.044047 | 0.00219 | 20.1131 | |
203 |
| -| CosineDist | 0.04496 | 0.002855 | 15.7457 | |
204 |
| -| CorrDist | 0.080828 | 0.029708 | 2.7207 | |
205 |
| -| ChiSqDist | 0.051009 | 0.008088 | 6.307 | |
206 |
| -| KLDivergence | 0.079598 | 0.035353 | 2.2515 | |
207 |
| -| JSDivergence | 0.545789 | 0.493362 | 1.1063 | |
208 |
| -| WeightedSqEuclidean | 0.046182 | 0.003219 | 14.3477 | |
209 |
| -| WeightedEuclidean | 0.046831 | 0.004122 | 11.3603 | |
210 |
| -| WeightedCityblock | 0.046457 | 0.003636 | 12.7781 | |
211 |
| -| WeightedMinkowski | 0.062532 | 0.020486 | 3.0524 | |
212 |
| -| WeightedHamming | 0.046217 | 0.002269 | 20.3667 | |
213 |
| -| SqMahalanobis | 0.150364 | 0.042335 | 3.5518 | |
214 |
| -| Mahalanobis | 0.159638 | 0.041071 | 3.8869 | |
215 |
| - |
216 |
| -We can see that using ``colwise`` instead of a simple loop yields considerable gain (2x - 9x), especially when the internal computation of each distance is simple. Nonetheless, when the computaton of a single distance is heavy enough (e.g. *Minkowski* and *JSDivergence*), the gain is not as significant. |
| 194 | +| distance | loop | colwise | gain | |
| 195 | +|----------- | -------| ----------| -------| |
| 196 | +| SqEuclidean | 0.012308s | 0.003860s | 3.1884 | |
| 197 | +| Euclidean | 0.012484s | 0.003995s | 3.1246 | |
| 198 | +| Cityblock | 0.012463s | 0.003927s | 3.1735 | |
| 199 | +| Chebyshev | 0.014897s | 0.005898s | 2.5258 | |
| 200 | +| Minkowski | 0.028154s | 0.017812s | 1.5806 | |
| 201 | +| Hamming | 0.012200s | 0.003896s | 3.1317 | |
| 202 | +| CosineDist | 0.013816s | 0.004670s | 2.9583 | |
| 203 | +| CorrDist | 0.023349s | 0.016626s | 1.4044 | |
| 204 | +| ChiSqDist | 0.015375s | 0.004788s | 3.2109 | |
| 205 | +| KLDivergence | 0.044360s | 0.036123s | 1.2280 | |
| 206 | +| JSDivergence | 0.098587s | 0.085595s | 1.1518 | |
| 207 | +| BhattacharyyaDist | 0.023103s | 0.013002s | 1.7769 | |
| 208 | +| HellingerDist | 0.023329s | 0.012555s | 1.8581 | |
| 209 | +| WeightedSqEuclidean | 0.012136s | 0.003758s | 3.2296 | |
| 210 | +| WeightedEuclidean | 0.012307s | 0.003789s | 3.2482 | |
| 211 | +| WeightedCityblock | 0.012287s | 0.003923s | 3.1321 | |
| 212 | +| WeightedMinkowski | 0.029895s | 0.018471s | 1.6185 | |
| 213 | +| WeightedHamming | 0.013427s | 0.004082s | 3.2896 | |
| 214 | +| SqMahalanobis | 0.121636s | 0.019370s | 6.2796 | |
| 215 | +| Mahalanobis | 0.117871s | 0.019939s | 5.9117 | |
| 216 | + |
| 217 | +We can see that using ``colwise`` instead of a simple loop yields considerable gain (2x - 6x), especially when the internal computation of each distance is simple. Nonetheless, when the computaton of a single distance is heavy enough (e.g. *Minkowski* and *JSDivergence*), the gain is not as significant. |
217 | 218 |
|
218 | 219 | #### Pairwise benchmark
|
219 | 220 |
|
220 | 221 | The table below compares the performance (measured in terms of average elapsed time of each iteration) of a straightforward loop implementation and an optimized implementation provided in *Distances.jl*. The task in each iteration is to compute a specific distance in a pairwise manner between columns in a ``100-by-200`` and ``100-by-250`` matrices, which will result in a ``200-by-250`` distance matrix.
|
221 | 222 |
|
222 |
| -| distance | loop | pairwise | gain | |
223 |
| -|------------ | --------| ------------| -----------| |
224 |
| -| SqEuclidean | 0.119961 | 0.00037 | **324.6457** | |
225 |
| -| Euclidean | 0.122645 | 0.000678 | **180.9180** | |
226 |
| -| Cityblock | 0.116956 | 0.007997 | 14.6251 | |
227 |
| -| Chebyshev | 0.137985 | 0.028489 | 4.8434 | |
228 |
| -| Minkowski | 0.170101 | 0.059991 | 2.8354 | |
229 |
| -| Hamming | 0.110742 | 0.004781 | 23.1627 | |
230 |
| -| CosineDist | 0.110913 | 0.000514 | **215.8028** | |
231 |
| -| CorrDist | 0.1992 | 0.000808 | 246.4574 | |
232 |
| -| ChiSqDist | 0.124782 | 0.020781 | 6.0046 | |
233 |
| -| KLDivergence | 0.1994 | 0.088366 | 2.2565 | |
234 |
| -| JSDivergence | 1.35502 | 1.215785 | 1.1145 | |
235 |
| -| WeightedSqEuclidean | 0.119797 | 0.000444 | **269.531** | |
236 |
| -| WeightedEuclidean | 0.126304 | 0.000712 | **177.5122** | |
237 |
| -| WeightedCityblock | 0.117185 | 0.011475 | 10.2122 | |
238 |
| -| WeightedMinkowski | 0.172614 | 0.061693 | 2.7979 | |
239 |
| -| WeightedHamming | 0.112525 | 0.005072 | 22.1871 | |
240 |
| -| SqMahalanobis | 0.377342 | 0.000577 | **653.9759** | |
241 |
| -| Mahalanobis | 0.373796 | 0.002359 | **158.4337** | |
| 223 | +| distance | loop | pairwise | gain | |
| 224 | +|----------- | -------| ----------| -------| |
| 225 | +| SqEuclidean | 0.032179s | 0.000170s | **189.7468** | |
| 226 | +| Euclidean | 0.031646s | 0.000326s | **97.1773** | |
| 227 | +| Cityblock | 0.031594s | 0.002771s | 11.4032 | |
| 228 | +| Chebyshev | 0.036732s | 0.011575s | 3.1735 | |
| 229 | +| Minkowski | 0.073685s | 0.047725s | 1.5440 | |
| 230 | +| Hamming | 0.030016s | 0.002539s | 11.8236 | |
| 231 | +| CosineDist | 0.035426s | 0.000235s | **150.8504** | |
| 232 | +| CorrDist | 0.061430s | 0.000341s | **180.1693** | |
| 233 | +| ChiSqDist | 0.037702s | 0.011709s | 3.2199 | |
| 234 | +| KLDivergence | 0.119043s | 0.086861s | 1.3705 | |
| 235 | +| JSDivergence | 0.255449s | 0.227079s | 1.1249 | |
| 236 | +| BhattacharyyaDist | 0.059165s | 0.033330s | 1.7751 | |
| 237 | +| HellingerDist | 0.056953s | 0.031163s | 1.8276 | |
| 238 | +| WeightedSqEuclidean | 0.031781s | 0.000218s | **145.9820** | |
| 239 | +| WeightedEuclidean | 0.031365s | 0.000410s | **76.4517** | |
| 240 | +| WeightedCityblock | 0.031239s | 0.003242s | 9.6360 | |
| 241 | +| WeightedMinkowski | 0.077039s | 0.049319s | 1.5621 | |
| 242 | +| WeightedHamming | 0.032584s | 0.005673s | 5.7442 | |
| 243 | +| SqMahalanobis | 0.280485s | 0.000297s | **943.6018** | |
| 244 | +| Mahalanobis | 0.295715s | 0.000498s | **593.6096** | |
242 | 245 |
|
243 | 246 | For distances of which a major part of the computation is a quadratic form (e.g. *Euclidean*, *CosineDist*, *Mahalanobis*), the performance can be drastically improved by restructuring the computation and delegating the core part to ``GEMM`` in *BLAS*. The use of this strategy can easily lead to 100x performance gain over simple loops (see the highlighted part of the table above).
|
0 commit comments