Melodyo Generator

A Go-based automated music file processor that downloads audio files from an FTP server, generates multiple bitrate versions (64kbps, 128kbps, 320kbps) using FFmpeg, and uploads them back to the server. The application is designed to work with a REST API for task management and authentication.

Features

• Multi-threaded Processing: Concurrent processing of multiple albums/directories using configurable worker pools
• Automated Bitrate Generation: Creates lower bitrate versions of audio files (64k, 128k, 320k) based on source quality
• Smart Skip Logic: Automatically skips files that already have all required bitrate versions
• FTP Integration: Downloads and uploads files to/from an FTP server
• API Integration: Authenticates and retrieves processing tasks from a REST API
• Docker Support: Fully containerized with Docker and Docker Compose
• FFmpeg Integration: Uses FFmpeg for audio bitrate detection and conversion

Architecture

The application consists of three main components:

1. API Client (src/api.go): Handles authentication and communication with the Melodyo API
2. Directory Operator (src/directory_operator.go): Core processing logic for downloading, converting, and uploading files
3. Main Worker (main.go): Orchestrates FTP connections and manages concurrent workers

Prerequisites

• Docker and Docker Compose (recommended)
• OR:
  • Go 1.16+
  • FFmpeg (for audio processing)
  • Access to an FTP server
  • Access to Melodyo API

Installation & Setup

Using Docker (Recommended)

1. Clone the repository:

git clone https://github.com/melodyo/melodyo-generator.git
cd melodyo-generator

2. Copy the environment example file and configure it:

cp .env.example .env

3. Edit the .env file with your credentials:

# API Configuration
API_URL=https://your-api-url.com
API_AUTH_ENDPOINT=/auth/login
API_AUTH_EMAIL=your-email@example.com
API_AUTH_PASSWORD=your-password
API_GENERATOR_ENDPOINT=/api/generator

# FTP Configuration
FTP_URL=ftp.your-server.com
FTP_USERNAME=your-ftp-username
FTP_PASSWORD=your-ftp-password
FTP_ROOT=/path/to/music/directory
FTP_PORT=21

# Processing Configuration
CORE_NUMS=4  # Number of concurrent workers (defaults to CPU cores if not set)

4. Build and run with Docker Compose:

docker-compose up --build -d

5. View logs:

docker logs -f melodyo-generator

Manual Installation

1. Install Go 1.16 or higher
2. Install FFmpeg:

# Ubuntu/Debian
sudo apt-get install ffmpeg

# macOS
brew install ffmpeg

3. Clone and build:

git clone https://github.com/melodyo/melodyo-generator.git
cd melodyo-generator
go mod download
go build -o melodyo-generator

4. Set environment variables and run:

export API_URL=https://your-api-url.com
# ... set other environment variables
./melodyo-generator

How It Works

1. Authentication: The application authenticates with the Melodyo API using provided credentials
2. Fetch Tasks: Retrieves a list of album/directory IDs to process from the API
3. FTP Connection Pool: Creates a pool of FTP connections based on CORE_NUMS setting
4. Concurrent Processing: For each directory:
   • Downloads original music files from FTP
   • Checks the bitrate of each file using FFmpeg
   • Determines which bitrate versions are missing (64k, 128k, 320k)
   • Generates missing bitrate versions using FFmpeg
   • Uploads generated files back to the FTP server
   • Notifies the API that processing is complete
5. Cleanup: Removes temporary local files after processing

Directory Structure

.
├── main.go                 # Entry point and worker orchestration
├── src/
│   ├── api.go             # API authentication and communication
│   └── directory_operator.go  # File processing logic
├── Dockerfile             # Docker container definition
├── docker-compose.yml     # Docker Compose configuration
├── go.mod                 # Go module dependencies
├── go.sum                 # Dependency checksums
├── .env.example           # Environment variables template
└── downloads/             # Temporary directory for file processing (created at runtime)

Docker Commands

# Build and start the container
docker-compose up --build -d

# View logs
docker logs -f melodyo-generator

# Restart the container
docker restart melodyo-generator

# Stop the container
docker stop melodyo-generator

# Remove the container
docker rm -f melodyo-generator

Configuration

Environment Variables

Variable	Description	Example
API_URL	Base URL of the Melodyo API	https://api.melodyo.com
API_AUTH_ENDPOINT	Authentication endpoint path	/auth/login
API_AUTH_EMAIL	Email for API authentication	user@example.com
API_AUTH_PASSWORD	Password for API authentication	your-password
API_GENERATOR_ENDPOINT	Generator endpoint path	/api/generator
FTP_URL	FTP server hostname	ftp.example.com
FTP_USERNAME	FTP username	ftpuser
FTP_PASSWORD	FTP password	ftppass
FTP_ROOT	Root directory on FTP server	/music
FTP_PORT	FTP port number	21
CORE_NUMS	Number of concurrent workers	4

Bitrate Conversion Logic

The application intelligently determines which bitrates to generate based on the source file quality:

• Source > 320kbps: Generates 320k, 128k, and 64k versions
• Source > 128kbps: Generates 128k and 64k versions
• Source > 64kbps: Generates 64k version only
• Source ≤ 64kbps: No conversion needed

Dependencies

• jlaffaye/ftp - FTP client library
• joho/godotenv - Environment variable management
• FFmpeg - Audio processing (included in Docker image)

Development

Building Locally

go build -o melodyo-generator

Running Tests

go test ./...

CI/CD

The project includes a .gitlab-ci.yml file for GitLab CI/CD pipeline integration.

Troubleshooting

Common Issues

1. FFmpeg not found: Ensure FFmpeg is installed and in your PATH
2. FTP connection errors: Verify FTP credentials and network connectivity
3. API authentication failures: Check API credentials and endpoint URLs
4. Out of memory: Reduce CORE_NUMS to process fewer files concurrently

Logs

Check Docker logs for detailed processing information:

docker logs melodyo-generator

License

Please check with the repository owner for licensing information.

Contributing

Contributions are welcome! Please submit pull requests or open issues for bugs and feature requests.