DATE_TIME

[Camfrm336]

ID:15

DATE_TIME

In the anonymized output we received:

The coverage date was changed from ##-## to the correct ####-##-##
Anonymized: The coverage date was changed from ##-## to the correct <DATE_TIME>

It looks the program is not correct anonymizing the DATE_TIME variable as it shows the inputted variable.

[c-rubio]

I took a look at the Presidio Date Pattern Recognizer code, and what I found is interesting.

It supports recognizing numbered months (1, 2, 3,...,12) as well as three letter months (such as JAN, FEB, MAR,...,DEC). Let's refer to Day, Month, and Year each individually as "date factors." If you have three date factors, you may use a variety of separators between them (such as dashes, e.g. 01-01-2000) and in various orders. However, when you use only two date factors (such as month-year or day-month), not all potential cases are considered.

When using only two date factors, the detection is based on the use of the "-" or "/" symbols. When using "-" a lettered month is required to be found as a pattern under the recognizer regex (e.g. JAN-01 is valid, 01-01 is not). Meanwhile, for the "/" symbol, a numbered month is detected, but not a lettered month (e.g. 01/01 is valid, JAN/01 is not).

So, we may say that either the date given was wrong, or the Presidio analyzer was wrong. I'd prefer to side with greater caution, so I believe it may be worth it to investigate more two factor numbered dates patterns using the "-" separator, even if they carry a lower detection score. This way, when used in conjunction with context words (in the Presidio code these are "date" and "birthday") they would be detected and anonymized.

[claesmk]

I'm confused. On my side I see:

DATE_TIME

ID:15:Original : The coverage date was changed from 05-21 to the correct 2021-05-21
ID:15:Anonymized: The coverage date was changed from <DATE_TIME> to the correct <DATE_TIME>

I don't think anything was not anonymized correctly. I also don't think I exposed an potential PII in my reply because a date by itself is not PII

[c-rubio]

I just ran test_pii_scan.py to check if my pii_scan.py file is ok.

I agree with your point a date without context isn't PII. However, I think we just kept cautious since it was detailing an event (start of insurance coverage). I just reran pii_scan.py to double check, and can confirm that my results were the following:

# DATE_TIME
ID:15:Original  : The coverage date was changed from XX-XX to the correct XXXX-XX-XX
ID:15:Anonymized: The coverage date was changed from XX-XX to the correct <DATE_TIME>

I also double checked the tutorial video for this sprint 3, and can confirm that the results in the video line up with the results given here. (skip to time 8:22 on the video)

To get to the bottom of this, some deeper investigation is needed; @claesmk perhaps it is possible we have different versions of presidio running? The easiest way to check if we're running the same code is to compare our DATE_TIME regex patterns, but that would take a bit of work to find the files again and coordinate this. I'll discuss with my team between now and sprint 4 to see if everyone gets the same results.

My version for dependencies are as follows:

space: 3.8.2
presidio_analyzer: 2.2.355
presidio-anonymizer: 2.2.355
requests: 2.32.3

[claesmk]

Very interesting - I'll do some further investigations