RetroAchievements ROM Matcher

A tool to match RetroAchievements supported games with Redump DAT files and download ROMs from Internet Archive collections with automatic CHD conversion.

Features

27 Gaming Platforms Supported - disc-based (Redump), cartridge-based (No-Intro/TOSEC), and filename-based (Myrient)
Smart caching system to minimize API requests and improve performance
Automatic DAT file management - downloads and extracts Redump and No-Intro DAT files
Intelligent game matching between RetroAchievements and ROM databases
ZIP→CHD conversion workflow - downloads Redump ZIP files and converts to CHD format
Multi-threaded downloads with progress tracking and graceful interruption
Multi-process safety - file locking prevents conflicts when running multiple instances
Checksum verification ensures download integrity
Configurable collection sources - supports multiple Internet Archive collections
Resume capability - handles interruptions and crashed processes gracefully

Setup

Clone the repository:

git clone <repository-url>
cd retroachievements-matcher

Install dependencies:

pip install -r requirements.txt

# Install MAME tools for CHD conversion (Ubuntu/Debian)
sudo apt install mame-tools

Configure Internet Archive access:
```
ia configure
```
- Enter your Internet Archive email and password when prompted
- This is required for downloading ROMs from IA collections
- Creates ~/.config/ia.ini with your credentials
Create your config file:
```
cp config_template.py config.py
```
Add your RetroAchievements credentials to config.py:
- Get your API key and username from: https://retroachievements.org/controlpanel.php
- The config.py file is gitignored to keep your credentials secure

Usage

Multi-Platform Support

This tool now supports multiple platforms through a unified interface:

Disc-Based Platforms (Redump):

# Find matches (default behavior)
./ra_matcher.py 3do     # 3DO Interactive Multiplayer
./ra_matcher.py dc      # Dreamcast
./ra_matcher.py ps2     # PlayStation 2
./ra_matcher.py psp     # PlayStation Portable
./ra_matcher.py psx     # PlayStation 1
./ra_matcher.py saturn  # Sega Saturn

# Download matched ROMs
./ra_matcher.py 3do --download
./ra_matcher.py dc --download
./ra_matcher.py ps2 --download
./ra_matcher.py psp --download
./ra_matcher.py psx --download
./ra_matcher.py saturn --download

Cartridge-Based Platforms (No-Intro):

# Find matches (default behavior)
./ra_matcher.py atari2600       # Atari 2600
./ra_matcher.py atari7800       # Atari 7800
./ra_matcher.py gamegear        # Sega Game Gear
./ra_matcher.py gb              # Game Boy
./ra_matcher.py gba             # Game Boy Advance
./ra_matcher.py gbc             # Game Boy Color
./ra_matcher.py genesis         # Sega Genesis/Mega Drive
./ra_matcher.py intellivision   # Intellivision
./ra_matcher.py jaguar          # Jaguar
./ra_matcher.py lynx            # Atari Lynx
./ra_matcher.py n64             # Nintendo 64
./ra_matcher.py neo-geo-pocket  # Neo Geo Pocket
./ra_matcher.py neo-geo-pocket-color # Neo Geo Pocket Color
./ra_matcher.py nes             # Nintendo Entertainment System
./ra_matcher.py pokemon-mini    # Pokemon Mini
./ra_matcher.py sms             # Sega Master System
./ra_matcher.py snes            # Super Nintendo Entertainment System
./ra_matcher.py tg16            # PC Engine/TurboGrafx-16

# Download matched ROMs
./ra_matcher.py atari2600 --download
./ra_matcher.py atari7800 --download
./ra_matcher.py gamegear --download
./ra_matcher.py gb --download
./ra_matcher.py gba --download
./ra_matcher.py gbc --download
./ra_matcher.py genesis --download
./ra_matcher.py intellivision --download
./ra_matcher.py jaguar --download
./ra_matcher.py lynx --download
./ra_matcher.py n64 --download
./ra_matcher.py neo-geo-pocket --download
./ra_matcher.py neo-geo-pocket-color --download
./ra_matcher.py nes --download
./ra_matcher.py pokemon-mini --download
./ra_matcher.py sms --download
./ra_matcher.py snes --download
./ra_matcher.py tg16 --download

# ... etc for all platforms

No-Intro DAT-Based Platforms:

# Find matches (default behavior) - platforms with custom hashes
./ra_matcher.py ds              # Nintendo DS

# Download matched ROMs
./ra_matcher.py ds --download

TOSEC-Based Platforms:

# Apple II uses TOSEC collection (RetroAchievements preferred source)
./ra_matcher.py appleii         # Apple II
./ra_matcher.py appleii --download

Advanced Options

All platforms support the same advanced options:

# Force refresh cached data
./ra_matcher.py ps2 --force-refresh

# Clear platform cache
./ra_matcher.py ps2 --clear-cache

# Download specific files only
./ra_matcher.py ps2 --download-files game1.zip game2.zip

# Limit preview results
./ra_matcher.py ps2 --limit 50

# Control API rate limiting
./ra_matcher.py ps2 --delay 2.0

# Skip preview and go straight to action
./ra_matcher.py ps2 --no-preview

Download Directory Configuration

To customize the base download directory: Edit your config.py file:

# In config.py
BASE_DOWNLOAD_DIR = "./roms"  # Change to your preferred base directory

All platforms will create subdirectories under this base path (e.g., ./roms/snes/, ./roms/ps2/, etc.).

Subfolder ROM Organization

NoIntro/TOSEC platforms support optional subfolder extraction for better ROM organization:

To enable for all No-Intro platforms: Edit your config.py file:

# In config.py
NOINTRO_EXTRACT_TO_SUBFOLDERS = True  # Set to True to extract each ROM to its own subfolder

Special cases:

Apple II: Always uses subfolders (hardcoded True) since it's a computer platform that benefits from organization
All other No-Intro platforms: Use the shared NOINTRO_EXTRACT_TO_SUBFOLDERS setting from config.py

When enabled:

Each ROM is placed in its own subfolder named after the game
Zip files are automatically extracted and the zip is removed
Useful for computer platforms (Apple II) and organized ROM libraries
Example: Apple Bowl (1979)(Apple Computer)/Apple Bowl (1979)(Apple Computer).do

Supported Platforms: All NoIntro/TOSEC platforms (appleii, atari2600, atari7800, gamegear, gb, gba, gbc, genesis, intellivision, jaguar, lynx, n64, neo-geo-pocket, neo-geo-pocket-color, nes, pokemon-mini, sms, snes, tg16)

Advanced Options:

# Force refresh cache and download
./ra_matcher.py ps2 --download --force-refresh

# Custom API rate limiting
./ra_matcher.py ps2 --download --delay 3.0

# Limit number of matches shown
./ra_matcher.py ps2 --limit 50

Download Workflow

The system follows this automated workflow:

Parse Redump DAT → Extract game information
Match with RetroAchievements → Find supported games
Download ZIP files → From Internet Archive collections
Verify checksums → Ensure download integrity
Convert to CHD → Automatic ZIP→CHD conversion using chdman
Clean up → Remove temporary files and organize final CHDs

Expected Output:

🚀 Internet Archive Downloader
Collection: ps2-redump-usa-chd-part-A
🔄 CHD conversion: Enabled
🔄 [1/23] Starting: Ace Combat 04 - Shattered Skies
🔄 [1/23] Converting to CHD: Ace Combat 04 - Shattered Skies
✅ [1/23] CHD created: Ace Combat 04 - Shattered Skies (847.2 MiB CHD)

Command Line Options

Required:

platform: Platform to process (e.g., ps2, snes, nes)

API Credentials (required if not in config.py):

--api-key: Your RetroAchievements Web API key
--api-user: Your RetroAchievements username

Optional:

--download: Download all matching ROM files (default: just find matches)
--download-dat: Download/update the platform DAT file only
--download-files FILE [FILE ...]: Download specific files by name
--download-dir: Directory to download ROMs (uses platform default if not specified)
--limit N: Limit number of games to process (for testing/preview)
--collections COLLECTION [COLLECTION ...]: Internet Archive collections to search (overrides platform default)
--max-workers N: Maximum number of download workers (default: 4)
--delay N: Delay between RetroAchievements API requests in seconds (default: 2.0)
--clear-cache: Clear platform cache before running
--force-refresh: Force refresh of cached data
--no-preview: Skip showing match preview (go straight to action)

Multiple Collection Support

You can run multiple instances simultaneously for different platforms without conflicts:

# Terminal 1
./ra_matcher.py ps2 --download

# Terminal 2 (safe to run simultaneously)
./ra_matcher.py snes --download

# Terminal 3 (also safe)
./ra_matcher.py nes --download --limit 10

File locking prevents conflicts when the same file exists in multiple collections.

Project Structure

retroachievements-matcher/
├── .gitignore
├── README.md
├── requirements.txt
├── config_template.py      # Template for user configuration
├── config.py              # User config (created from template, gitignored)
├── ra_matcher.py          # Main CLI entry point
└── platforms/
    ├── common/
    │   ├── cache.py        # Smart caching system
    │   ├── downloader.py   # IA downloader with CHD conversion
    │   ├── matcher.py      # Common matching utilities
    │   ├── nointro_matcher.py  # No-Intro ROM set matching
    │   ├── nointro_dat_matcher.py  # No-Intro DAT filename matching
    │   └── redump_matcher.py   # Redump DAT file matching
    ├── 3do/platform_config.py
    ├── appleii/platform_config.py
    ├── atari2600/platform_config.py
    ├── atari7800/platform_config.py
    ├── dc/platform_config.py
    ├── ds/platform_config.py
    ├── dsi/platform_config.py
    ├── gamegear/platform_config.py
    ├── gb/platform_config.py
    ├── gba/platform_config.py
    ├── gbc/platform_config.py
    ├── genesis/platform_config.py
    ├── intellivision/platform_config.py
    ├── jaguar/platform_config.py
    ├── lynx/platform_config.py
    ├── n64/platform_config.py
    ├── neo-geo-pocket/platform_config.py
    ├── neo-geo-pocket-color/platform_config.py
    ├── nes/platform_config.py
    ├── pokemon-mini/platform_config.py
    ├── ps2/platform_config.py
    ├── psp/platform_config.py
    ├── psx/platform_config.py
    ├── saturn/platform_config.py
    ├── sms/platform_config.py
    ├── snes/platform_config.py
    └── tg16/platform_config.py

# Cache directories (auto-created when needed):
platforms/[platform]/cache/     # Platform-specific API response cache

System Requirements

Python 3.7+
chdman (from MAME tools) for CHD conversion
Internet connection for API calls and downloads
Sufficient disk space for ROM downloads and temporary conversion files

Platform Types

This tool supports four different types of gaming platforms, each with their own matching approach:

Disc-Based Platforms (Redump)

Systems: PlayStation 1/2, Sega Saturn, Dreamcast, PlayStation Portable, 3DO Interactive Multiplayer
Method: Downloads compact Redump DAT files containing game hashes
Matching: Compares RetroAchievements hashes against DAT database
ROM Sources: Internet Archive collections with disc images
Formats: CHD (compressed), ZIP fallback
Database Size: Small DAT files (~few MB)

Cartridge-Based Platforms (No-Intro)

Systems: NES, SNES, Game Boy family, Nintendo 64, Genesis, Atari systems, Neo Geo Pocket, Lynx, Intellivision, Pokemon Mini, Jaguar
Method: Downloads complete ROM collections to build hash database
Matching: Extracts and hashes ROMs to compare with RetroAchievements
ROM Sources: Internet Archive ni-roms collection (complete sets)
Formats: ZIP archives containing original ROMs
Database Size: Complete ROM collections (~GB per platform)

No-Intro DAT-Based Platforms

Systems: Nintendo DS (platforms with custom RetroAchievements hashes)
Method: Downloads No-Intro DAT files for filename-based matching
Matching: Compares RetroAchievements filenames against DAT game names
ROM Sources: Internet Archive ni-roms collection (selective downloads)
Formats: ZIP archives containing original ROMs
Database Size: Small DAT files (~few MB) + matched ROMs only

Myrient-Based Platforms

Systems: Nintendo DS (temporary workaround for missing IA sources)
Method: Scrapes Myrient directory listings for available ROMs
Matching: Fetches individual game hashes from RetroAchievements API, then matches by filename
ROM Sources: Myrient No-Intro collections (decrypted DS ROMs)
Formats: ZIP archives containing decrypted ROMs
Database Size: Cached ROM directory listings + individual game hashes

TOSEC-Based Platforms

Systems: Apple II (preferred by RetroAchievements for this platform)
Method: Downloads TOSEC collection archives (similar to No-Intro workflow)
Matching: Extracts and hashes ROMs using NoIntroMatcher infrastructure
ROM Sources: Internet Archive TOSEC collections
Formats: Various (DSK, DO, PO, WOZ for Apple II)
Database Size: Platform-specific collections

Key Differences

Aspect	Redump (Disc)	No-Intro (Cartridge)	No-Intro DAT	TOSEC	Myrient
Setup	Download small DAT file	Download complete ROM collection	Download small DAT file	Download TOSEC collection	Scrape directory listings
Matching	Hash lookup in DAT	Extract and hash ROMs directly	Filename matching against DAT	Extract and hash ROMs directly	Fetch individual hashes, filename match
Storage	DATs + matched ROMs	Full ROM collection	DATs + matched ROMs	Full collection	Cached listings + matched ROMs
Speed	Fast (pre-computed hashes)	Slower initial setup, fast after	Fast (filename matching)	Slower initial setup, fast after	Slower initial cache, fast after
Coverage	Verified dumps only	Complete preservation sets	Complete preservation sets	Comprehensive archives	No-Intro quality, direct access
Use Case	Standard disc platforms	Hash-based cartridge platforms	Custom hash platforms	Specialized collections	IA source unavailable

Disc-Based Platforms (Redump):

Input: Redump DAT XML files
Download: ZIP files from Internet Archive
Output: CHD files (compressed disc images)
Fallback: ZIP files (if CHD conversion fails or is disabled)

Cartridge-Based Platforms (No-Intro):

Input: No-Intro ROM collections from Internet Archive ni-roms
Download: ZIP archives containing entire platform ROM sets
Processing: Individual ROM ZIPs extracted and hashed for matching
Output: ZIP files containing original ROMs (space-efficient)

No-Intro DAT-Based Platforms:

Input: No-Intro DAT XML files
Download: Individual ROM files from Internet Archive ni-roms collections
Processing: Filename matching between RetroAchievements and DAT entries
Output: ZIP files containing matched ROMs (selective downloads)

TOSEC-Based Platforms:

Input: TOSEC collection archives from Internet Archive
Download: ZIP archives containing TOSEC preservation sets
Processing: Individual ROM files extracted and hashed for matching
Output: Various formats (DSK, WOZ, etc.) depending on platform

Myrient-Based Platforms:

Input: Myrient directory listings (scraped via HTTP)
Download: Individual ROM files from Myrient servers
Processing: Fetch individual game hashes from RetroAchievements API, filename matching
Output: ZIP files containing decrypted ROMs (direct download)

Default Download Structure

ROMs are organized by platform in subdirectories under the base download directory. The default base directory is ./roms/ (configurable in config.py as BASE_DOWNLOAD_DIR):

Disc-Based Platforms:

3DO: ./roms/3do/
DC: ./roms/dc/
PS2: ./roms/ps2/
PSP: ./roms/psp/
PSX: ./roms/psx/
Saturn: ./roms/saturn/

Cartridge-Based Platforms:

Atari 2600: ./roms/atari2600/
Atari 7800: ./roms/atari7800/
Atari Lynx: ./roms/lynx/
Game Gear: ./roms/gamegear/
GB: ./roms/gb/
GBA: ./roms/gba/
GBC: ./roms/gbc/
Genesis: ./roms/genesis/
Intellivision: ./roms/intellivision/
Jaguar: ./roms/jaguar/
Master System: ./roms/sms/
N64: ./roms/n64/
Neo Geo Pocket: ./roms/neo-geo-pocket/
Neo Geo Pocket Color: ./roms/neo-geo-pocket-color/
NES: ./roms/nes/
PC Engine: ./roms/tg16/
Pokemon Mini: ./roms/pokemon-mini/
SNES: ./roms/snes/

No-Intro DAT-Based Platforms:

Nintendo DS: ./roms/ds/

TOSEC-Based Platforms:

Apple II: ./roms/appleii/

To customize the base directory: Edit BASE_DOWNLOAD_DIR in your config.py file.

Each platform directory contains the final ROM files ready for emulation.

Adding New Platforms

Create a new directory under platforms/
Implement platform-specific logic using common utilities from platforms/common/
Add platform-specific console IDs, collection mappings, and file extensions
Inherit from common classes for caching and downloading functionality
Update README with platform-specific usage examples

Troubleshooting

CHD Conversion Issues

Ensure chdman is installed: sudo apt install mame-tools
Check that chdman is in your PATH: which chdman
System will fall back to ZIP files if CHD conversion fails

Download Issues

Check your internet connection
Verify Internet Archive collection IDs exist
Use --delay to slow down requests if getting rate limited
Use --no-threading for debugging single file issues
Internet Archive authentication: Run ia configure if downloads fail with authentication errors
Check that ~/.config/ia.ini exists and contains valid credentials

Cache Issues

Use --force-refresh to rebuild cache
Use --clear-cache to clear platform-specific cache before running
Cache files are stored in platforms/[platform]/cache/ directories
Cache files are automatically managed and expire appropriately

Performance Notes

Multi-threading speeds up downloads significantly
Caching reduces API calls on subsequent runs
CHD compression typically reduces file sizes by 30-50% compared to uncompressed ISOs
File locking allows safe parallel execution without corruption

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
platforms		platforms
.gitignore		.gitignore
README.md		README.md
config_template.py		config_template.py
ra_matcher.py		ra_matcher.py
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

RetroAchievements ROM Matcher

Features

Setup

Usage

Multi-Platform Support

Advanced Options

Download Directory Configuration

Subfolder ROM Organization

Advanced Options:

Download Workflow

Command Line Options

Required:

API Credentials (required if not in config.py):

Optional:

Multiple Collection Support

Project Structure

System Requirements

Platform Types

Disc-Based Platforms (Redump)

Cartridge-Based Platforms (No-Intro)

No-Intro DAT-Based Platforms

Myrient-Based Platforms

TOSEC-Based Platforms

Key Differences

Default Download Structure

Adding New Platforms

Troubleshooting

CHD Conversion Issues

Download Issues

Cache Issues

Performance Notes

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages