AI Detection False Positive

[Rigdon]

Enabled the new "Use AI detection to find additional secrets" feature today and got an immediate false positive for the value: 'testClientSecret', which was a placeholder value used for some Jest unit tests.

[courtneycl]

Hi @Rigdon, thanks for the report -- can you let us know the filepath this detection was found in?

[Rigdon]

Hi @courtneycl! I'm going to obfuscate part of the path, but it looks something like: tf-module-1/actions/tests/function-name.test.js

[AshwinMohan86]

test findings should be addressed now.

[AleksandrKudriavtcev]

"AccessToken123" used in python doctests interpreted as exposed password

[manuel-cohere]

channel-id

It has a message "This secret was detected by AI. False positive rates will improve over time."

Anyhow, I just disable it since it just gave us a bunch of FP

[courtneycl]

Hi @manuel-cohere -- sorry to hear the feature is too noisy. If you're willing to share examples of false positives (the literal and the filepath, or even just screenshots) that would help us improve our detection and reduce false positives going forward 🙏