Unique index on url_previews causes media to lose the information that it was the result a URL preview #649
Description
I am aware this project is not being actively worked on. This is information to help those who need it. I am not a Go developer.
It was very common in our media database that there was seemingly media uploaded by users who never uploaded it, which turned out to be the result of URL previews. This is not only is a problem for bloating the media storage, but it makes moderation tricky when it seems that users directly uploaded images that they didn't.
I believe what happens is that MMR creates and saves the url preview image, stores it as a media record, but once the final step of adding it to the url_previews table fails, it does nothing to roll back the creation of that media.
I assume the constraint fails when there's concurrent requests for the same URL preview.
An easy fix for us was to remove the UNIQUE constraint from url_previews_index:
DROP INDEX url_previews_index;
CREATE INDEX IF NOT EXISTS url_previews_index ON url_previews (url, error_code, bucket_ts, language_header);I don't believe this is related to issue #615