Skip to content

Migrate ML model to PyTorch #679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Dmitriy-NK wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Migrate ML model to PyTorch #679

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions .ci/benchmark.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,21 +228,21 @@ TOTAL: 10343 16352935 12112 46625 49
credsweeper result_cnt : 11860, lost_cnt : 0, true_cnt : 11648, false_cnt : 212
Rules Positives Negatives Templates Reported TP FP TN FN FPR FNR ACC PRC RCL F1
------------------------------ ----------- ----------- ----------- ---------- ----- ---- ----- ---- -------- -------- -------- -------- -------- --------
API 125 3172 187 122 122 0 3359 3 0.000000 0.024000 0.999139 1.000000 0.976000 0.987854
API 125 3172 187 119 118 1 3358 7 0.000298 0.056000 0.997704 0.991597 0.944000 0.967213
AWS Client ID 170 19 0 162 162 0 19 8 0.000000 0.047059 0.957672 1.000000 0.952941 0.975904
AWS Multi 82 10 0 84 82 1 9 0 0.100000 0.000000 0.989130 0.987952 1.000000 0.993939
AWS S3 Bucket 67 23 0 92 67 23 0 0 1.000000 0.000000 0.744444 0.744444 1.000000 0.853503
Atlassian Old PAT token 3 7 0 10 3 7 0 0 1.000000 0.000000 0.300000 0.300000 1.000000 0.461538
Auth 417 2744 81 396 395 1 2824 22 0.000354 0.052758 0.992906 0.997475 0.947242 0.971710
Auth 417 2744 81 398 393 5 2820 24 0.001770 0.057554 0.991055 0.987437 0.942446 0.964417
Azure Access Token 19 0 0 12 12 0 0 7 0.368421 0.631579 1.000000 0.631579 0.774194
BASE64 Private Key 12 4 0 12 12 0 4 0 0.000000 0.000000 1.000000 1.000000 1.000000 1.000000
BASE64 encoded PEM Private Key 7 0 0 5 5 0 0 2 0.285714 0.714286 1.000000 0.714286 0.833333
Bitbucket Client ID 19 52 0 72 17 52 0 2 1.000000 0.105263 0.239437 0.246377 0.894737 0.386364
Bitbucket Client Secret 29 75 1 104 27 75 1 2 0.986842 0.068966 0.266667 0.264706 0.931034 0.412214
CMD ConvertTo-SecureString 13 4 0 13 13 0 4 0 0.000000 0.000000 1.000000 1.000000 1.000000 1.000000
CMD Password 27 128 0 25 25 0 128 2 0.000000 0.074074 0.987097 1.000000 0.925926 0.961538
CMD Password 21 128 6 21 21 0 134 0 0.000000 0.000000 1.000000 1.000000 1.000000 1.000000
CMD Secret 1 1 0 1 1 0 1 0 0.000000 0.000000 1.000000 1.000000 1.000000 1.000000
CMD Token 6 0 0 6 6 0 0 0 0.000000 1.000000 1.000000 1.000000 1.000000
CMD Token 6 0 0 5 5 0 0 1 0.166667 0.833333 1.000000 0.833333 0.909091
Certificate 24 471 0 19 19 0 471 5 0.000000 0.208333 0.989899 1.000000 0.791667 0.883721
Credential 91 422 76 90 90 0 498 1 0.000000 0.010989 0.998302 1.000000 0.989011 0.994475
Docker Swarm Token 2 0 0 1 1 0 0 1 0.500000 0.500000 1.000000 0.500000 0.666667
Expand All @@ -259,21 +259,21 @@ Grafana Provisioned API Key 22 1 0
JSON Web Token 170 61 0 131 131 0 61 39 0.000000 0.229412 0.831169 1.000000 0.770588 0.870432
Jira / Confluence PAT token 0 4 0 0 0 4 0 0.000000 1.000000
Jira 2FA 21 0 1 15 15 0 1 6 0.000000 0.285714 0.727273 1.000000 0.714286 0.833333
Key 3916 15714 482 3921 3902 19 16177 14 0.001173 0.003575 0.998359 0.995154 0.996425 0.995789
Nonce 93 49 0 93 92 1 48 1 0.020408 0.010753 0.985915 0.989247 0.989247 0.989247
Key 3911 15717 483 3953 3894 59 16141 17 0.003642 0.004347 0.996221 0.985075 0.995653 0.990336
Nonce 93 49 0 92 92 0 49 1 0.000000 0.010753 0.992958 1.000000 0.989247 0.994595
Other 9 7450 5 0 0 7455 9 0.000000 1.000000 0.998794 0.000000
PEM Private Key 1019 1483 0 1023 1019 4 1479 0 0.002697 0.000000 0.998401 0.996090 1.000000 0.998041
Password 2032 7527 2539 1951 1946 5 10061 86 0.000497 0.042323 0.992478 0.997437 0.957677 0.977153
SQL Password 44 13 0 41 41 0 13 3 0.000000 0.068182 0.947368 1.000000 0.931818 0.964706
Password 1941 7534 2623 1883 1835 48 10109 106 0.004726 0.054611 0.987271 0.974509 0.945389 0.959728
SQL Password 44 13 0 42 41 1 12 3 0.076923 0.068182 0.929825 0.976190 0.931818 0.953488
Salesforce Credentials 2 0 0 2 2 0 0 0 0.000000 1.000000 1.000000 1.000000 1.000000
Salt 49 74 1 46 46 0 75 3 0.000000 0.061224 0.975806 1.000000 0.938776 0.968421
Secret 1310 1567 799 1303 1303 0 2366 7 0.000000 0.005344 0.998096 1.000000 0.994656 0.997321
Salt 48 75 1 45 43 2 74 5 0.026316 0.104167 0.943548 0.955556 0.895833 0.924731
Secret 1310 1567 799 1299 1297 2 2364 13 0.000845 0.009924 0.995919 0.998460 0.990076 0.994251
Seed 1 6 0 0 0 6 1 0.000000 1.000000 0.857143 0.000000
Slack Token 4 1 0 4 4 0 1 0 0.000000 0.000000 1.000000 1.000000 1.000000 1.000000
Stripe Credentials 2 0 0 2 2 0 0 0 0.000000 1.000000 1.000000 1.000000 1.000000
Tencent WeChat API App ID 6 0 0 6 6 0 0 0 0.000000 1.000000 1.000000 1.000000 1.000000
Token 647 4169 453 626 626 0 4622 21 0.000000 0.032457 0.996014 1.000000 0.967543 0.983504
Token 645 4170 453 619 615 4 4619 30 0.000865 0.046512 0.993546 0.993538 0.953488 0.973101
Twilio Credentials 30 39 0 30 30 0 39 0 0.000000 0.000000 1.000000 1.000000 1.000000 1.000000
URL Credentials 224 168 197 223 223 0 365 1 0.000000 0.004464 0.998302 1.000000 0.995536 0.997763
UUID 1075 265 0 1074 1073 1 264 2 0.003774 0.001860 0.997761 0.999069 0.998140 0.998604
12112 46625 4907 11872 11648 212 46413 464 0.004547 0.038309 0.988491 0.982125 0.961691 0.971800
12000 46639 5003 11809 11486 311 46328 514 0.006668 0.042833 0.985931 0.973637 0.957167 0.965332
4 changes: 2 additions & 2 deletions .github/workflows/check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ jobs:
- name: Check ml_config.json and ml_model.onnx integrity
if: ${{ always() && steps.code_checkout.conclusion == 'success' }}
run: |
md5sum --binary credsweeper/ml_model/ml_config.json | grep 3a4bfcd6f3ea74461b158d4ec073cc06
md5sum --binary credsweeper/ml_model/ml_model.onnx | grep 9725b166e07e60f94929fea986f84ae2
md5sum --binary credsweeper/ml_model/ml_config.json | grep dae9d27fe1b6de565c6c4c994ee7ba36
md5sum --binary credsweeper/ml_model/ml_model.onnx | grep c4c2a33a675e38dcc7024c176d9317ee

# # # line ending

Expand Down
Binary file modified credsweeper/ml_model/ml_model.onnx
View file
Open in desktop
Binary file not shown.
Loading