Multisite: Multiple copies of the same image in Media Library from syndicated posts

[peteschiebel]

Issue
When we syndicate posts between subsites in a multisite environment, we sometimes end up with multiple versions of the same image in the Media Library.

Details

• We recently switched from having multiple single-sites hosted on WordPress.com to having one multisite environment where all sites are now subsites.
• The issue didn't start immediately, but we didn't have it before then
• The issue isn't consistent, it doesn't happen every time
• Even when it does happen, it doesn't happen on every subsite the post is syndicated to. Ex: A post is syndicated to 4 other sites. 2 may experience the issue, with differing numbers of duplicates, and the other 2 may not experience the issue
• Here's a screenshot of the issue:
  

[GaryJones]

Investigation Summary

After investigation, this issue stems from the thumbnail tracking mechanism not handling multisite migrations well.

Root Cause

The XMLRPC client tracks synced thumbnails via post meta (_syn_push_thumb_*) which stores a mapping of site_ID → local_thumbnail_id. The deduplication logic at class-syndication-wp-xmlrpc-client.php:75 compares the current local thumbnail ID against what was previously synced.

After migrating from single-sites to multisite:

1. Attachment IDs likely changed, invalidating the tracking meta
2. The tracking meta may contain stale data pointing to old IDs
3. Without valid tracking data, each sync creates a new image

Why It's Inconsistent

• Different sites received different amounts of syndication activity post-migration
• Some tracking meta may have been updated while others remained stale
• Object caching inconsistencies in multisite can cause stale reads

Why We're Closing

A proper fix requires:

1. Server-side image deduplication (hash-based matching before upload)
2. Migration tools to clean/rebuild tracking meta
3. Enhanced locking to prevent race conditions

Given the issue is 5+ years old and the fix requires significant architectural changes that are better suited for a major refactor, we're closing this.

Workaround: After migration, delete the _syn_push_thumb_* post meta from affected posts to force a clean re-sync, then manually deduplicate the media library.