Skip to content

Commit fec5ea0

Browse files
author
bib-dedupe evaluator
committed
Update evaluation (2024-11-04-20-39)
1 parent a610195 commit fec5ea0

File tree

2 files changed

+58
-17
lines changed

2 files changed

+58
-17
lines changed

‎output/current_results.md‎

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -13,18 +13,18 @@
1313

1414
| package | FP | TP | FN | TN | false_positive_rate | specificity | sensitivity | precision | f1 | runtime |
1515
|:-----------|-----:|-----:|-----:|-----:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
16-
| bib-dedupe | 0 | 408 | 28 | 1552 | 0 | 1 | 0.9358 | 1 | 0.9668 | 0:00:10 |
16+
| bib-dedupe | 0 | 408 | 28 | 1552 | 0 | 1 | 0.9358 | 1 | 0.9668 | 0:00:09 |
1717
| asysd | 0 | 399 | 37 | 1552 | 0 | 1 | 0.9151 | 1 | 0.9557 | 0:00:04 |
1818
| asreview | 14 | 399 | 37 | 1538 | 0.009 | 0.991 | 0.9151 | 0.9661 | 0.9399 | 0:00:00 |
19-
| buhos | 13 | 395 | 41 | 1539 | 0.0084 | 0.9916 | 0.906 | 0.9681 | 0.936 | 0:05:15 |
19+
| buhos | 13 | 395 | 41 | 1539 | 0.0084 | 0.9916 | 0.906 | 0.9681 | 0.936 | 0:05:11 |
2020

2121
## digital_work (n=7159)
2222

2323
| package | FP | TP | FN | TN | false_positive_rate | specificity | sensitivity | precision | f1 | runtime |
2424
|:-----------|-----:|-----:|-----:|-----:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
25-
| bib-dedupe | 0 | 368 | 1 | 6790 | 0 | 1 | 0.9973 | 1 | 0.9986 | 0:00:37 |
26-
| asysd | 20 | 307 | 62 | 6770 | 0.0029 | 0.9971 | 0.832 | 0.9388 | 0.8822 | 0:00:15 |
27-
| buhos | 66 | 306 | 63 | 6724 | 0.0097 | 0.9903 | 0.8293 | 0.8226 | 0.8259 | 0:30:54 |
25+
| bib-dedupe | 0 | 368 | 1 | 6790 | 0 | 1 | 0.9973 | 1 | 0.9986 | 0:00:35 |
26+
| asysd | 20 | 307 | 62 | 6770 | 0.0029 | 0.9971 | 0.832 | 0.9388 | 0.8822 | 0:00:14 |
27+
| buhos | 66 | 306 | 63 | 6724 | 0.0097 | 0.9903 | 0.8293 | 0.8226 | 0.8259 | 0:30:39 |
2828
| asreview | 217 | 336 | 33 | 6573 | 0.032 | 0.968 | 0.9106 | 0.6076 | 0.7289 | 0:00:00 |
2929

3030
## haematology (n=1415)
@@ -33,33 +33,33 @@
3333
|:-----------|-----:|-----:|-----:|-----:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
3434
| bib-dedupe | 0 | 120 | 15 | 1278 | 0 | 1 | 0.8889 | 1 | 0.9412 | 0:00:06 |
3535
| asysd | 0 | 109 | 26 | 1278 | 0 | 1 | 0.8074 | 1 | 0.8934 | 0:00:04 |
36-
| buhos | 18 | 108 | 27 | 1260 | 0.0141 | 0.9859 | 0.8 | 0.8571 | 0.8276 | 0:03:38 |
36+
| buhos | 18 | 108 | 27 | 1260 | 0.0141 | 0.9859 | 0.8 | 0.8571 | 0.8276 | 0:03:32 |
3737
| asreview | 43 | 106 | 29 | 1235 | 0.0336 | 0.9664 | 0.7852 | 0.7114 | 0.7465 | 0:00:00 |
3838

3939
## stroke (n=1292)
4040

4141
| package | FP | TP | FN | TN | false_positive_rate | specificity | sensitivity | precision | f1 | runtime |
4242
|:-----------|-----:|-----:|-----:|-----:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
4343
| bib-dedupe | 0 | 312 | 2 | 978 | 0 | 1 | 0.9936 | 1 | 0.9968 | 0:00:05 |
44-
| asysd | 0 | 302 | 12 | 978 | 0 | 1 | 0.9618 | 1 | 0.9805 | 0:00:04 |
44+
| asysd | 0 | 302 | 12 | 978 | 0 | 1 | 0.9618 | 1 | 0.9805 | 0:00:03 |
4545
| asreview | 12 | 309 | 5 | 966 | 0.0123 | 0.9877 | 0.9841 | 0.9626 | 0.9732 | 0:00:00 |
46-
| buhos | 8 | 303 | 11 | 970 | 0.0082 | 0.9918 | 0.965 | 0.9743 | 0.9696 | 0:05:26 |
46+
| buhos | 8 | 303 | 11 | 970 | 0.0082 | 0.9918 | 0.965 | 0.9743 | 0.9696 | 0:05:21 |
4747

4848
## cytology_screening (n=1856)
4949

5050
| package | FP | TP | FN | TN | false_positive_rate | specificity | sensitivity | precision | f1 | runtime |
5151
|:-----------|-----:|-----:|-----:|-----:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
5252
| bib-dedupe | 0 | 766 | 6 | 1084 | 0 | 1 | 0.9922 | 1 | 0.9961 | 0:00:08 |
53-
| asysd | 1 | 751 | 21 | 1083 | 0.0009 | 0.9991 | 0.9728 | 0.9987 | 0.9856 | 0:00:04 |
53+
| asysd | 1 | 751 | 21 | 1083 | 0.0009 | 0.9991 | 0.9728 | 0.9987 | 0.9856 | 0:00:03 |
5454
| asreview | 12 | 753 | 19 | 1072 | 0.0111 | 0.9889 | 0.9754 | 0.9843 | 0.9798 | 0:00:00 |
55-
| buhos | 29 | 741 | 31 | 1055 | 0.0268 | 0.9732 | 0.9598 | 0.9623 | 0.9611 | 0:07:07 |
55+
| buhos | 29 | 741 | 31 | 1055 | 0.0268 | 0.9732 | 0.9598 | 0.9623 | 0.9611 | 0:06:57 |
5656

5757
## srsr (n=53001)
5858

5959
| package | FP | TP | FN | TN | false_positive_rate | specificity | sensitivity | precision | f1 | runtime |
6060
|:-----------|-----:|------:|-----:|------:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
61-
| bib-dedupe | 0 | 16916 | 68 | 36005 | 0 | 1 | 0.996 | 1 | 0.998 | 0:08:16 |
62-
| asysd | 25 | 16554 | 430 | 35980 | 0.0007 | 0.9993 | 0.9747 | 0.9985 | 0.9864 | 0:01:24 |
61+
| bib-dedupe | 0 | 16916 | 68 | 36005 | 0 | 1 | 0.996 | 1 | 0.998 | 0:07:37 |
62+
| asysd | 25 | 16554 | 430 | 35980 | 0.0007 | 0.9993 | 0.9747 | 0.9985 | 0.9864 | 0:01:19 |
6363
| asreview | 1769 | 16061 | 923 | 34236 | 0.0491 | 0.9509 | 0.9457 | 0.9008 | 0.9227 | 0:00:02 |
6464

6565
## diabetes (n=1845)
@@ -69,13 +69,13 @@
6969
| bib-dedupe | 0 | 1261 | 0 | 584 | 0 | 1 | 1 | 1 | 1 | 0:00:09 |
7070
| asysd | 0 | 1259 | 2 | 584 | 0 | 1 | 0.9984 | 1 | 0.9992 | 0:00:06 |
7171
| asreview | 24 | 1251 | 10 | 560 | 0.0411 | 0.9589 | 0.9921 | 0.9812 | 0.9866 | 0:00:00 |
72-
| buhos | 121 | 1141 | 120 | 463 | 0.2072 | 0.7928 | 0.9048 | 0.9041 | 0.9045 | 0:08:33 |
72+
| buhos | 121 | 1141 | 120 | 463 | 0.2072 | 0.7928 | 0.9048 | 0.9041 | 0.9045 | 0:08:28 |
7373

7474
## depression (n=79880)
7575

7676
| package | FP | TP | FN | TN | false_positive_rate | specificity | sensitivity | precision | f1 | runtime |
7777
|:-----------|-----:|------:|-----:|------:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
78-
| bib-dedupe | 0 | 10072 | 88 | 69712 | 0 | 1 | 0.9913 | 1 | 0.9957 | 0:10:57 |
78+
| bib-dedupe | 0 | 10072 | 88 | 69712 | 0 | 1 | 0.9913 | 1 | 0.9957 | 0:09:53 |
7979
| asreview | 3297 | 6114 | 4046 | 66415 | 0.0473 | 0.9527 | 0.6018 | 0.6497 | 0.6248 | 0:00:03 |
8080

8181
## neuroimaging (n=3438)
@@ -85,16 +85,16 @@
8585
| bib-dedupe | 0 | 1292 | 7 | 2139 | 0 | 1 | 0.9946 | 1 | 0.9973 | 0:00:16 |
8686
| asysd | 3 | 1278 | 21 | 2136 | 0.0014 | 0.9986 | 0.9838 | 0.9977 | 0.9907 | 0:00:07 |
8787
| asreview | 8 | 1261 | 38 | 2131 | 0.0037 | 0.9963 | 0.9707 | 0.9937 | 0.9821 | 0:00:00 |
88-
| buhos | 211 | 1065 | 234 | 1928 | 0.0986 | 0.9014 | 0.8199 | 0.8346 | 0.8272 | 0:16:12 |
88+
| buhos | 211 | 1065 | 234 | 1928 | 0.0986 | 0.9014 | 0.8199 | 0.8346 | 0.8272 | 0:16:01 |
8989

9090
## cardiac (n=8948)
9191

9292
| package | FP | TP | FN | TN | false_positive_rate | specificity | sensitivity | precision | f1 | runtime |
9393
|:-----------|-----:|-----:|-----:|-----:|----------------------:|--------------:|--------------:|------------:|-------:|:----------|
94-
| bib-dedupe | 0 | 3520 | 12 | 5415 | 0 | 1 | 0.9966 | 1 | 0.9983 | 0:00:51 |
94+
| bib-dedupe | 0 | 3520 | 12 | 5415 | 0 | 1 | 0.9966 | 1 | 0.9983 | 0:00:50 |
9595
| asysd | 4 | 3504 | 28 | 5411 | 0.0007 | 0.9993 | 0.9921 | 0.9989 | 0.9955 | 0:00:19 |
9696
| asreview | 124 | 3163 | 369 | 5291 | 0.0229 | 0.9771 | 0.8955 | 0.9623 | 0.9277 | 0:00:00 |
97-
| buhos | 313 | 3061 | 471 | 5102 | 0.0578 | 0.9422 | 0.8666 | 0.9072 | 0.8865 | 1:16:26 |
97+
| buhos | 313 | 3061 | 471 | 5102 | 0.0578 | 0.9422 | 0.8666 | 0.9072 | 0.8865 | 1:14:51 |
9898

9999
## special_cases (n=6)
100100

‎output/evaluation.csv‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7186,3 +7186,44 @@ bib-dedupe,2024-11-03 05:02,special_cases,1,0,2,9,0.0,1.0,0.3333333333333333,1.0
71867186
asreview,2024-11-03 05:02,special_cases,2,2,1,7,0.2222222222222222,0.7777777777777778,0.6666666666666666,0.5,0.5714285714285715,0:00:00
71877187
asysd,2024-11-03 05:02,special_cases,1,0,2,9,0.0,1.0,0.3333333333333333,1.0,0.5,0:00:03
71887188
buhos,2024-11-03 05:02,special_cases,1,2,2,7,0.2222222222222222,0.7777777777777778,0.3333333333333333,0.3333333333333333,0.3333333333333333,0:00:00
7189+
bib-dedupe,2024-11-04 16:58,haematology,120,0,15,1278,0.0,1.0,0.8888888888888888,1.0,0.9411764705882352,0:00:06
7190+
asreview,2024-11-04 16:58,haematology,106,43,29,1235,0.0336463223787167,0.9663536776212832,0.7851851851851852,0.7114093959731543,0.7464788732394365,0:00:00
7191+
asysd,2024-11-04 16:58,haematology,109,0,26,1278,0.0,1.0,0.8074074074074075,1.0,0.8934426229508198,0:00:04
7192+
buhos,2024-11-04 17:02,haematology,108,18,27,1260,0.0140845070422535,0.9859154929577464,0.8,0.8571428571428571,0.8275862068965518,0:03:32
7193+
bib-dedupe,2024-11-04 17:03,cardiac,3520,0,12,5415,0.0,1.0,0.9966024915062288,1.0,0.998298355076574,0:00:50
7194+
asreview,2024-11-04 17:03,cardiac,3163,124,369,5291,0.022899353647276,0.977100646352724,0.8955266138165345,0.9622756312747186,0.9277020090922424,0:00:00
7195+
asysd,2024-11-04 17:04,cardiac,3504,4,28,5411,0.0007386888273314,0.9992613111726684,0.9920724801812004,0.998859749144812,0.9954545454545456,0:00:19
7196+
buhos,2024-11-04 18:19,cardiac,3061,313,471,5102,0.0578024007386888,0.9421975992613112,0.866647791619479,0.9072317723770006,0.8864755285259194,1:14:51
7197+
bib-dedupe,2024-11-04 18:19,stroke,312,0,2,978,0.0,1.0,0.9936305732484076,1.0,0.9968051118210862,0:00:05
7198+
asreview,2024-11-04 18:19,stroke,309,12,5,966,0.0122699386503067,0.9877300613496932,0.9840764331210192,0.9626168224299064,0.973228346456693,0:00:00
7199+
asysd,2024-11-04 18:19,stroke,302,0,12,978,0.0,1.0,0.961783439490446,1.0,0.9805194805194806,0:00:03
7200+
buhos,2024-11-04 18:24,stroke,303,8,11,970,0.0081799591002045,0.9918200408997956,0.964968152866242,0.9742765273311896,0.9696,0:05:21
7201+
bib-dedupe,2024-11-04 18:33,srsr,16916,0,68,36005,0.0,1.0,0.995996231747527,1.0,0.9979941002949853,0:07:37
7202+
asreview,2024-11-04 18:46,srsr,16061,1769,923,34236,0.0491320649909734,0.9508679350090266,0.9456547338671693,0.900785193494111,0.92267478600563,0:00:02
7203+
asysd,2024-11-04 18:48,srsr,16554,25,430,35980,0.0006943480072212,0.9993056519927788,0.9746820536975978,0.9984920682791484,0.9864434049399636,0:01:19
7204+
bib-dedupe,2024-11-04 18:59,depression,10072,0,88,69712,0.0,1.0,0.9913385826771652,1.0,0.9956504547251878,0:09:53
7205+
asreview,2024-11-04 19:29,depression,6114,3297,4046,66415,0.0472945834289648,0.9527054165710352,0.6017716535433071,0.649665285304431,0.6248020029635685,0:00:03
7206+
bib-dedupe,2024-11-04 19:29,cytology_screening,766,0,6,1084,0.0,1.0,0.9922279792746114,1.0,0.9960988296488946,0:00:08
7207+
asreview,2024-11-04 19:29,cytology_screening,753,12,19,1072,0.011070110701107,0.988929889298893,0.9753886010362696,0.984313725490196,0.9798308392973324,0:00:00
7208+
asysd,2024-11-04 19:29,cytology_screening,751,1,21,1083,0.0009225092250922,0.9990774907749076,0.97279792746114,0.9986702127659576,0.9855643044619424,0:00:03
7209+
buhos,2024-11-04 19:36,cytology_screening,741,29,31,1055,0.0267527675276752,0.9732472324723248,0.9598445595854922,0.9623376623376624,0.9610894941634242,0:06:57
7210+
bib-dedupe,2024-11-04 19:36,diabetes,1261,0,0,584,0.0,1.0,1.0,1.0,1.0,0:00:09
7211+
asreview,2024-11-04 19:36,diabetes,1251,24,10,560,0.0410958904109589,0.958904109589041,0.9920697858842188,0.9811764705882352,0.9865930599369084,0:00:00
7212+
asysd,2024-11-04 19:36,diabetes,1259,0,2,584,0.0,1.0,0.9984139571768438,1.0,0.9992063492063492,0:00:06
7213+
buhos,2024-11-04 19:45,diabetes,1141,121,120,463,0.2071917808219178,0.7928082191780822,0.9048374306106264,0.904120443740095,0.904478795085216,0:08:28
7214+
bib-dedupe,2024-11-04 19:45,respiratory,408,0,28,1552,0.0,1.0,0.9357798165137616,1.0,0.966824644549763,0:00:09
7215+
asreview,2024-11-04 19:45,respiratory,399,14,37,1538,0.009020618556701,0.990979381443299,0.9151376146788992,0.9661016949152542,0.939929328621908,0:00:00
7216+
asysd,2024-11-04 19:45,respiratory,399,0,37,1552,0.0,1.0,0.9151376146788992,1.0,0.9556886227544912,0:00:04
7217+
buhos,2024-11-04 19:50,respiratory,395,13,41,1539,0.0083762886597938,0.9916237113402062,0.9059633027522936,0.9681372549019608,0.9360189573459716,0:05:11
7218+
bib-dedupe,2024-11-04 19:51,neuroimaging,1292,0,7,2139,0.0,1.0,0.9946112394149346,1.0,0.9972983404091084,0:00:16
7219+
asreview,2024-11-04 19:51,neuroimaging,1261,8,38,2131,0.0037400654511453,0.9962599345488546,0.970746728252502,0.9936958234830576,0.9820872274143302,0:00:00
7220+
asysd,2024-11-04 19:51,neuroimaging,1278,3,21,2136,0.0014025245441795,0.9985974754558204,0.9838337182448036,0.9976580796252928,0.9906976744186048,0:00:07
7221+
buhos,2024-11-04 20:07,neuroimaging,1065,211,234,1928,0.0986442262739598,0.9013557737260404,0.8198614318706697,0.8346394984326019,0.8271844660194174,0:16:01
7222+
bib-dedupe,2024-11-04 20:07,digital_work,368,0,1,6790,0.0,1.0,0.997289972899729,1.0,0.9986431478968792,0:00:35
7223+
asreview,2024-11-04 20:08,digital_work,336,217,33,6573,0.0319587628865979,0.968041237113402,0.9105691056910568,0.6075949367088608,0.7288503253796095,0:00:00
7224+
asysd,2024-11-04 20:08,digital_work,307,20,62,6770,0.0029455081001472,0.9970544918998528,0.8319783197831978,0.9388379204892966,0.882183908045977,0:00:14
7225+
buhos,2024-11-04 20:39,digital_work,306,66,63,6724,0.009720176730486,0.990279823269514,0.8292682926829268,0.8225806451612904,0.8259109311740891,0:30:39
7226+
bib-dedupe,2024-11-04 20:39,special_cases,1,0,2,9,0.0,1.0,0.3333333333333333,1.0,0.5,0:00:00
7227+
asreview,2024-11-04 20:39,special_cases,2,2,1,7,0.2222222222222222,0.7777777777777778,0.6666666666666666,0.5,0.5714285714285715,0:00:00
7228+
asysd,2024-11-04 20:39,special_cases,1,0,2,9,0.0,1.0,0.3333333333333333,1.0,0.5,0:00:03
7229+
buhos,2024-11-04 20:39,special_cases,1,2,2,7,0.2222222222222222,0.7777777777777778,0.3333333333333333,0.3333333333333333,0.3333333333333333,0:00:00

0 commit comments

Comments
 (0)