-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathoutput.txt
More file actions
570 lines (481 loc) · 20 KB
/
output.txt
File metadata and controls
570 lines (481 loc) · 20 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
============================================================
Graph Feature Generation Pipeline
Dataset: SMALL
Evaluation: Enabled
Leiden: Enabled
Rank Stability: Enabled
============================================================
Extracting unique temporal windows...
Found 9 windows to process.
Pre-computing per-window fraud labels with DuckDB...
Launching ProcessPoolExecutor with 3 workers...
Processing days: 100%|███████████████████████████████████████████████████████████████████████████████| 9/9 [03:58<00:00, 26.51s/day]
Applying sequential rank stability post-processing...
Using pre-aggregated is_fraud labels from aggregated_nodes.parquet...
Saving features to data/HI_Small/sliding_window_features.parquet via DuckDB out-of-core join...
Cleaning temporary artifacts...
Saving metrics to data/HI_Small/sliding_window_metrics.parquet...
============================================================
Aggregate Evaluation Summary (Mean & Median Across All Days)
============================================================
[PR_VOL_DEEP]
----------------------------------------
Mean ROC-AUC: 0.5544
Median ROC-AUC: 0.5436
Mean Average Precision: 0.0525
Median Average Precision:0.0517
Mean Precision@K:
@ 1: 100.0000% (lift: 98.88x)
@ 2: 100.0000% (lift: 98.88x)
@ 10: 95.5556% (lift: 88.70x)
@ 50: 49.1111% (lift: 43.03x)
@ 100: 39.2222% (lift: 32.49x)
@ 500: 20.8000% (lift: 15.12x)
Median Precision@K:
@ 1: 100.0000% (lift: 66.19x)
@ 2: 100.0000% (lift: 66.19x)
@ 10: 100.0000% (lift: 66.19x)
@ 50: 50.0000% (lift: 29.47x)
@ 100: 40.0000% (lift: 24.11x)
@ 500: 21.6000% (lift: 13.26x)
[PR_VOL_SHALLOW]
----------------------------------------
Mean ROC-AUC: 0.5544
Median ROC-AUC: 0.5433
Mean Average Precision: 0.0547
Median Average Precision:0.0540
Mean Precision@K:
@ 1: 100.0000% (lift: 98.88x)
@ 2: 100.0000% (lift: 98.88x)
@ 10: 96.6667% (lift: 89.64x)
@ 50: 51.1111% (lift: 44.31x)
@ 100: 41.2222% (lift: 33.62x)
@ 500: 22.0444% (lift: 15.89x)
Median Precision@K:
@ 1: 100.0000% (lift: 66.19x)
@ 2: 100.0000% (lift: 66.19x)
@ 10: 100.0000% (lift: 66.19x)
@ 50: 52.0000% (lift: 32.15x)
@ 100: 41.0000% (lift: 25.45x)
@ 500: 23.0000% (lift: 14.33x)
[PR_COUNT]
----------------------------------------
Mean ROC-AUC: 0.5594
Median ROC-AUC: 0.5509
Mean Average Precision: 0.0667
Median Average Precision:0.0649
Mean Precision@K:
@ 1: 100.0000% (lift: 98.88x)
@ 2: 100.0000% (lift: 98.88x)
@ 10: 96.6667% (lift: 89.64x)
@ 50: 63.5556% (lift: 55.11x)
@ 100: 50.8889% (lift: 40.96x)
@ 500: 26.9556% (lift: 19.86x)
Median Precision@K:
@ 1: 100.0000% (lift: 66.19x)
@ 2: 100.0000% (lift: 66.19x)
@ 10: 100.0000% (lift: 66.19x)
@ 50: 66.0000% (lift: 38.85x)
@ 100: 51.0000% (lift: 30.14x)
@ 500: 27.8000% (lift: 16.54x)
[HITS_HUB]
----------------------------------------
Mean ROC-AUC: 0.5670
Median ROC-AUC: 0.5602
Mean Average Precision: 0.0274
Median Average Precision:0.0300
Mean Precision@K:
@ 1: 55.5556% (lift: 66.30x)
@ 2: 61.1111% (lift: 53.14x)
@ 10: 44.4444% (lift: 35.30x)
@ 50: 17.5556% (lift: 15.02x)
@ 100: 13.2222% (lift: 11.42x)
@ 500: 6.7556% (lift: 5.35x)
Median Precision@K:
@ 1: 100.0000% (lift: 40.47x)
@ 2: 50.0000% (lift: 40.47x)
@ 10: 40.0000% (lift: 36.42x)
@ 50: 16.0000% (lift: 13.24x)
@ 100: 13.0000% (lift: 7.94x)
@ 500: 6.8000% (lift: 5.31x)
[HITS_AUTH]
----------------------------------------
Mean ROC-AUC: 0.5542
Median ROC-AUC: 0.5570
Mean Average Precision: 0.0228
Median Average Precision:0.0240
Mean Precision@K:
@ 1: 66.6667% (lift: 72.30x)
@ 2: 55.5556% (lift: 50.54x)
@ 10: 37.7778% (lift: 28.83x)
@ 50: 12.6667% (lift: 9.69x)
@ 100: 8.3333% (lift: 6.32x)
@ 500: 4.0000% (lift: 2.97x)
Median Precision@K:
@ 1: 100.0000% (lift: 54.05x)
@ 2: 50.0000% (lift: 40.47x)
@ 10: 30.0000% (lift: 32.38x)
@ 50: 14.0000% (lift: 9.27x)
@ 100: 8.0000% (lift: 5.42x)
@ 500: 4.0000% (lift: 2.60x)
[K_CORE]
----------------------------------------
Mean ROC-AUC: 0.6435
Median ROC-AUC: 0.6407
Mean Average Precision: 0.0553
Median Average Precision:0.0610
Mean Precision@K:
@ 1: 66.6667% (lift: 39.50x)
@ 2: 61.1111% (lift: 36.07x)
@ 10: 63.3333% (lift: 41.65x)
@ 50: 55.3333% (lift: 36.84x)
@ 100: 50.0000% (lift: 34.96x)
@ 500: 29.7111% (lift: 21.08x)
Median Precision@K:
@ 1: 100.0000% (lift: 46.86x)
@ 2: 50.0000% (lift: 33.09x)
@ 10: 70.0000% (lift: 42.17x)
@ 50: 62.0000% (lift: 34.38x)
@ 100: 56.0000% (lift: 35.08x)
@ 500: 31.6000% (lift: 19.78x)
[BETWEENNESS]
----------------------------------------
Mean ROC-AUC: 0.6108
Median ROC-AUC: 0.6064
Mean Average Precision: 0.0574
Median Average Precision:0.0575
Mean Precision@K:
@ 1: 100.0000% (lift: 98.88x)
@ 2: 94.4444% (lift: 95.21x)
@ 10: 78.8889% (lift: 71.81x)
@ 50: 42.2222% (lift: 35.27x)
@ 100: 36.6667% (lift: 29.37x)
@ 500: 23.5778% (lift: 17.41x)
Median Precision@K:
@ 1: 100.0000% (lift: 66.19x)
@ 2: 100.0000% (lift: 54.05x)
@ 10: 80.0000% (lift: 48.64x)
@ 50: 44.0000% (lift: 29.19x)
@ 100: 38.0000% (lift: 23.83x)
@ 500: 24.0000% (lift: 15.89x)
============================================================
Leiden Community Detection Analysis
============================================================
Analysis based on final day: 2022-09-10 00:00:00
Total nodes with community assignment: 1,124
Total communities detected: 437
Overall fraud rate: 0.0000%
[Community Size Distribution]
--------------------------------------------------
Size Communities Nodes Fraud Rate Lift
------------------------------------------------------------
1 261 261 0 0.0000% nanx
2-4 147 360 0 0.0000% nanx
5-9 14 77 0 0.0000% nanx
10-49 14 362 0 0.0000% nanx
50-99 1 64 0 0.0000% nanx
[Top 10 Communities by Fraud Rate (min 3 members)]
----------------------------------------------------------------------
Comm ID Size Fraud Rate Lift
--------------------------------------------------
0 47 0 0.0000% 0.00x
1 64 0 0.0000% 0.00x
2 43 0 0.0000% 0.00x
3 26 0 0.0000% 0.00x
4 24 0 0.0000% 0.00x
5 30 0 0.0000% 0.00x
6 24 0 0.0000% 0.00x
7 36 0 0.0000% 0.00x
8 21 0 0.0000% 0.00x
9 25 0 0.0000% 0.00x
[Leiden Effectiveness Summary]
--------------------------------------------------
Mean Leiden Modularity (Q): 0.6822
Average community fraud rate: 0.0000%
Overall fraud rate: 0.0000%
============================================================
Feature Statistics Summary
============================================================
pr_vol_deep pr_vol_shallow pr_count ... leiden_micro_size leiden_macro_modularity leiden_micro_modularity
count 1.116737e+06 1.116737e+06 1.116737e+06 ... 1.116737e+06 1.116737e+06 1.116737e+06
mean 6.788958e-06 6.780498e-06 6.792196e-06 ... 1.030025e+03 6.413791e-01 6.299573e-01
std 1.461161e-05 1.242807e-05 1.286375e-05 ... 2.082025e+03 1.946014e-01 1.940634e-01
min 0.000000e+00 0.000000e+00 0.000000e+00 ... 0.000000e+00 0.000000e+00 0.000000e+00
25% 3.537675e-06 3.910482e-06 4.043463e-06 ... 2.100000e+01 5.744702e-01 5.581744e-01
50% 5.751505e-06 6.159583e-06 5.740327e-06 ... 1.090000e+02 6.777551e-01 6.672881e-01
75% 7.963210e-06 7.953112e-06 8.219375e-06 ... 8.320000e+02 6.817074e-01 6.698234e-01
max 3.402760e-03 2.853358e-03 3.058359e-03 ... 8.873000e+03 8.750989e-01 8.681814e-01
[8 rows x 18 columns]
============================================================
Rank Anomaly Analysis (reference: pr_vol_deep)
============================================================
Total rank anomaly flags: 44,667
Fraud among rank anomalies: 360 (0.81%)
Overall fraud rate: 0.49%
Lift from rank anomaly detection: 1.63x
============================================================
Rank Stability Processing
============================================================
Rank stability features were materialized via out-of-core DuckDB joins.
============================================================
Graph Feature Generation Pipeline
Dataset: SMALL
Evaluation: Enabled
Leiden: Enabled
Rank Stability: Enabled
============================================================
Extracting unique temporal windows...
Found 9 windows to process.
Pre-computing per-window fraud labels with DuckDB...
Launching ProcessPoolExecutor with 3 workers...
Processing days: 100%|███████████████████████████| 9/9 [06:49<00:00, 45.47s/day]
Applying sequential rank stability post-processing...
Using pre-aggregated is_fraud labels from aggregated_nodes.parquet...
Saving features to data/LI_Small/sliding_window_features.parquet via DuckDB out-of-core join...
Cleaning temporary artifacts...
Saving metrics to data/LI_Small/sliding_window_metrics.parquet...
============================================================
Aggregate Evaluation Summary (Mean & Median Across All Days)
============================================================
[PR_VOL_DEEP]
----------------------------------------
Mean ROC-AUC: 0.5371
Median ROC-AUC: 0.5356
Mean Average Precision: 0.0517
Median Average Precision:0.0526
Mean Precision@K:
@ 1: 100.0000% (lift: 151.99x)
@ 2: 100.0000% (lift: 151.99x)
@ 10: 100.0000% (lift: 151.99x)
@ 50: 60.8889% (lift: 85.90x)
@ 100: 43.2222% (lift: 57.85x)
@ 500: 19.8889% (lift: 23.94x)
Median Precision@K:
@ 1: 100.0000% (lift: 94.48x)
@ 2: 100.0000% (lift: 94.48x)
@ 10: 100.0000% (lift: 94.48x)
@ 50: 66.0000% (lift: 43.46x)
@ 100: 44.0000% (lift: 34.96x)
@ 500: 20.6000% (lift: 18.14x)
[PR_VOL_SHALLOW]
----------------------------------------
Mean ROC-AUC: 0.5372
Median ROC-AUC: 0.5355
Mean Average Precision: 0.0562
Median Average Precision:0.0543
Mean Precision@K:
@ 1: 100.0000% (lift: 151.99x)
@ 2: 100.0000% (lift: 151.99x)
@ 10: 100.0000% (lift: 151.99x)
@ 50: 66.2222% (lift: 91.32x)
@ 100: 47.6667% (lift: 62.02x)
@ 500: 21.6222% (lift: 25.73x)
Median Precision@K:
@ 1: 100.0000% (lift: 94.48x)
@ 2: 100.0000% (lift: 94.48x)
@ 10: 100.0000% (lift: 94.48x)
@ 50: 70.0000% (lift: 47.24x)
@ 100: 49.0000% (lift: 36.02x)
@ 500: 22.8000% (lift: 20.79x)
[PR_COUNT]
----------------------------------------
Mean ROC-AUC: 0.5429
Median ROC-AUC: 0.5411
Mean Average Precision: 0.0696
Median Average Precision:0.0662
Mean Precision@K:
@ 1: 100.0000% (lift: 151.99x)
@ 2: 100.0000% (lift: 151.99x)
@ 10: 100.0000% (lift: 151.99x)
@ 50: 76.6667% (lift: 103.00x)
@ 100: 56.1111% (lift: 72.70x)
@ 500: 26.5333% (lift: 31.90x)
Median Precision@K:
@ 1: 100.0000% (lift: 94.48x)
@ 2: 100.0000% (lift: 94.48x)
@ 10: 100.0000% (lift: 94.48x)
@ 50: 78.0000% (lift: 60.47x)
@ 100: 57.0000% (lift: 43.46x)
@ 500: 27.6000% (lift: 22.49x)
[HITS_HUB]
----------------------------------------
Mean ROC-AUC: 0.6468
Median ROC-AUC: 0.6378
Mean Average Precision: 0.0332
Median Average Precision:0.0332
Mean Precision@K:
@ 1: 100.0000% (lift: 151.99x)
@ 2: 100.0000% (lift: 151.99x)
@ 10: 92.2222% (lift: 127.14x)
@ 50: 46.0000% (lift: 57.68x)
@ 100: 26.5556% (lift: 34.09x)
@ 500: 9.8444% (lift: 11.75x)
Median Precision@K:
@ 1: 100.0000% (lift: 94.48x)
@ 2: 100.0000% (lift: 94.48x)
@ 10: 100.0000% (lift: 94.48x)
@ 50: 52.0000% (lift: 49.13x)
@ 100: 27.0000% (lift: 26.46x)
@ 500: 9.4000% (lift: 8.73x)
[HITS_AUTH]
----------------------------------------
Mean ROC-AUC: 0.6059
Median ROC-AUC: 0.5967
Mean Average Precision: 0.0270
Median Average Precision:0.0266
Mean Precision@K:
@ 1: 100.0000% (lift: 151.99x)
@ 2: 100.0000% (lift: 151.99x)
@ 10: 94.4444% (lift: 127.51x)
@ 50: 40.6667% (lift: 49.57x)
@ 100: 21.3333% (lift: 26.05x)
@ 500: 5.7111% (lift: 7.02x)
Median Precision@K:
@ 1: 100.0000% (lift: 94.48x)
@ 2: 100.0000% (lift: 94.48x)
@ 10: 100.0000% (lift: 94.48x)
@ 50: 46.0000% (lift: 43.46x)
@ 100: 25.0000% (lift: 23.62x)
@ 500: 6.0000% (lift: 5.67x)
[K_CORE]
----------------------------------------
Mean ROC-AUC: 0.6654
Median ROC-AUC: 0.6598
Mean Average Precision: 0.0557
Median Average Precision:0.0598
Mean Precision@K:
@ 1: 88.8889% (lift: 125.48x)
@ 2: 88.8889% (lift: 129.65x)
@ 10: 75.5556% (lift: 100.07x)
@ 50: 59.7778% (lift: 68.92x)
@ 100: 49.1111% (lift: 56.95x)
@ 500: 27.8222% (lift: 30.09x)
Median Precision@K:
@ 1: 100.0000% (lift: 78.30x)
@ 2: 100.0000% (lift: 81.75x)
@ 10: 80.0000% (lift: 62.64x)
@ 50: 68.0000% (lift: 64.25x)
@ 100: 55.0000% (lift: 51.97x)
@ 500: 29.6000% (lift: 27.97x)
[BETWEENNESS]
----------------------------------------
Mean ROC-AUC: 0.6399
Median ROC-AUC: 0.6401
Mean Average Precision: 0.0536
Median Average Precision:0.0524
Mean Precision@K:
@ 1: 100.0000% (lift: 151.99x)
@ 2: 100.0000% (lift: 151.99x)
@ 10: 82.2222% (lift: 129.00x)
@ 50: 37.7778% (lift: 50.18x)
@ 100: 30.6667% (lift: 40.24x)
@ 500: 22.9333% (lift: 25.90x)
Median Precision@K:
@ 1: 100.0000% (lift: 94.48x)
@ 2: 100.0000% (lift: 94.48x)
@ 10: 80.0000% (lift: 66.14x)
@ 50: 40.0000% (lift: 45.35x)
@ 100: 31.0000% (lift: 30.23x)
@ 500: 23.6000% (lift: 22.30x)
============================================================
Leiden Community Detection Analysis
============================================================
Analysis based on final day: 2022-09-10 00:00:00
Total nodes with community assignment: 213,679
Total communities detected: 3,945
Overall fraud rate: 1.8776%
[Community Size Distribution]
--------------------------------------------------
Size Communities Nodes Fraud Rate Lift
------------------------------------------------------------
1 1 1 0 0.0000% 0.00x
2-4 1652 4192 61 1.4552% 0.78x
5-9 504 3332 35 1.0504% 0.56x
10-49 934 22066 344 1.5590% 0.83x
50-99 401 28556 472 1.6529% 0.88x
100-499 426 74442 1373 1.8444% 0.98x
500+ 27 81090 1727 2.1297% 1.13x
[Top 10 Communities by Fraud Rate (min 3 members)]
----------------------------------------------------------------------
Comm ID Size Fraud Rate Lift
--------------------------------------------------
2807 3 2 66.6667% 35.51x
2295 4 2 50.0000% 26.63x
2372 4 2 50.0000% 26.63x
2434 4 2 50.0000% 26.63x
2447 4 2 50.0000% 26.63x
2507 4 2 50.0000% 26.63x
1847 9 3 33.3333% 17.75x
2544 3 1 33.3333% 17.75x
2599 3 1 33.3333% 17.75x
2666 3 1 33.3333% 17.75x
[Potential Fraud Rings: 100% Fraud Communities (size >= 2)]
--------------------------------------------------
Found 4 communities that are 100% fraudulent
Total nodes in these communities: 8
Size distribution: {'count': 4.0, 'mean': 2.0, 'std': 0.0, 'min': 2.0, '25%': 2.0, '50%': 2.0, '75%': 2.0, 'max': 2.0}
[Leiden Effectiveness Summary]
--------------------------------------------------
Mean Leiden Modularity (Q): 0.5301
Fraud in small communities (size <= 10): 106 / 4,012 (2.64%)
Average community fraud rate: 1.4602%
Overall fraud rate: 1.8776%
Fraud in high-concentration communities (>2x avg rate): 728 / 4,012 (18.15%)
============================================================
Feature Statistics Summary
============================================================
pr_vol_deep ... leiden_micro_modularity
count 1.921760e+06 ... 1.921760e+06
mean 4.683207e-06 ... 5.141204e-01
std 8.917120e-06 ... 8.230547e-02
min 7.401595e-07 ... 3.955688e-01
25% 2.427835e-06 ... 4.426400e-01
50% 3.687125e-06 ... 5.600660e-01
75% 5.264592e-06 ... 5.924410e-01
max 2.237592e-03 ... 5.972130e-01
[8 rows x 18 columns]
============================================================
Rank Anomaly Analysis (reference: pr_vol_deep)
============================================================
Total rank anomaly flags: 84,120
Fraud among rank anomalies: 2,647 (3.15%)
Overall fraud rate: 1.88%
Lift from rank anomaly detection: 1.67x
============================================================
Rank Stability Processing
============================================================
Rank stability features were materialized via out-of-core DuckDB joins.
Loading feature matrix from data/HI_Small/sliding_window_features.parquet...
Is time_variance in the dataset?: True
[Temporal Split Enforced]
Training set: 670,042 rows (up to 2022-09-07)
Validation set: 223,347 rows (up to 2022-09-08)
Testing set: 223,348 rows (up to 2022-09-10)
Class Imbalance Ratio (Neg/Pos) in Train: 201.74
Optimal Parameters Found via Successive Halving: {'colsample_bytree': np.float64(0.7838501639099957), 'learning_rate': np.float64(0.003523464812190403), 'max_depth': 2, 'n_estimators': 739, 'reg_lambda': np.float64(0.48861563555387233), 'scale_pos_weight': np.float64(4.55635212401633), 'subsample': np.float64(0.9633294328968971)}
Extracting Optimal F1 Threshold from Validation Set...
Optimal Threshold Found: 0.1281
Evaluating on Future Temporal Window (Test Set)...
Test AUPRC: 0.1062
Test ROC-AUC: 0.8041
Test F1-Score: 0.1530 (at 0.1281 threshold)
Test Precision@K:
@ 10: 90.0000% (Lift: 177.42x)
@ 50: 80.0000% (Lift: 157.70x)
@ 100: 62.0000% (Lift: 122.22x)
@ 500: 26.8000% (Lift: 52.83x)
========================================
[SUPERVISED BENCHMARK] Training LightGBM with Successive Halving...
========================================
Optimal LightGBM Parameters Found: {'learning_rate': np.float64(0.009055671373369193), 'n_estimators': 574, 'num_leaves': 7042, 'reg_lambda': np.float64(4.71822569445811), 'scale_pos_weight': np.float64(4.961372443656412)}
Extracting Optimal F1 Threshold for LightGBM from Validation Set...
Optimal LightGBM Threshold Found: 0.0804
Evaluating LightGBM on Test Set...
LightGBM Test AUPRC: 0.1215
LightGBM Test ROC-AUC: 0.8403
LightGBM Test F1-Score: 0.1142 (at 0.0804 threshold)
LightGBM Test Precision@K:
@ 10: 100.0000% (Lift: 197.13x)
@ 50: 92.0000% (Lift: 181.36x)
@ 100: 74.0000% (Lift: 145.88x)
@ 500: 27.0000% (Lift: 53.23x)