Tests will sometimes randomly report duplicates until the cache is cleared

[C-Loftus]

Left the repo over the weekend, then came back and ran the same tests but getting duplicate issues.
Once I clear cache, they go away.

Unclear why this is an issue since the TTL is 72 hours so nothing should be expired and it should be the same data as last week when it was passing fine.

FAILED rise/lib/location_test.py::test_filter_everything_by_wkt - AssertionError: 3424 witn name Hoover Power Plant Unit N8 was found in both https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=4&itemsPerPage=100 and https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=3&itemsPerPage=100. You may need to clear the cache for https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=3&itemsPerPage=100
FAILED rise/rise_edr_test.py::test_cube - AssertionError: 3424 is a duplicate with name Hoover Power Plant Unit N8 in https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=4&itemsPerPage=100
FAILED rise/rise_edr_test.py::test_get_or_fetch_all_param_filtered_pages - AssertionError: 3424 witn name Hoover Power Plant Unit N8 was found in both https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=4&itemsPerPage=100 and https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=3&itemsPerPage=100. You may need to clear the cache for https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=3&itemsPerPage=100
FAILED rise/rise_edr_test.py::test_area - AssertionError: 3424 is a duplicate with name Hoover Power Plant Unit N8 in https://data.usbr.gov/rise/api/location?include=catalogRecords.catalogItems&itemStructureId=1&page=4&itemsPerPage=100

[webb-ben]

Seems like there may be a bigger issue here: A feature may change what page it is on without notice? I have seen default adding a sortby to the query as a way of ensuring the page the feature is will only change if the feature set itself has changed

[C-Loftus]

Since the pages are being fetched together they should have the same TTL so even if they switch page it shouldn't affect us. Since once the TTL expires we just fetch the new pages and it should be fine

[webb-ben]

understood, but wouldn't it also address the issue you are seeing here. Because for whatever reason there is an issue where they are not all expiring at the same time or TTL?