Kindle Automator

An Android Kindle Use Agent for Programmatic Book Reading

Kindle Automator is an open-source automation platform that turns Android emulators into programmable Kindle reading agents. It provides a REST API to control the Amazon Kindle Android app, enabling developers to build applications that can read books, navigate pages, extract text via OCR, and manage entire Kindle libraries programmatically.

---

Why Does This Project Exist?

The Kindle ecosystem is notoriously locked down. Amazon provides no official API for third-party applications to interact with your purchased Kindle books. This project bridges that gap by using Android emulation and UI automation to provide programmatic access to Kindle functionality.

Use cases include:

• Building accessible reading applications for users with disabilities
• Creating custom reading interfaces and experiences
• Developing AI-powered reading assistants
• Automating book organization and library management
• Research applications requiring bulk text extraction
• Building cross-platform reading synchronization tools

---

Features

• 👥 Multi-User Support - Run multiple Android emulators simultaneously, each with its own authenticated Amazon account and isolated profile data

• 🔌 REST API - Complete HTTP API for all Kindle operations including authentication, library browsing, book opening, page navigation, and screenshots

• 📝 OCR Text Extraction - Extract readable text from any page using Google Document AI or Mistral OCR with automatic UI element filtering

• 📡 Real-Time Streaming - Server-Sent Events (SSE) endpoint for streaming book library data as it loads, with cover image extraction

• 🖼️ Book Cover Extraction - Automatically extract and cache book cover images from the library view for use in your applications

• 📑 Table of Contents Navigation - Retrieve chapter listings and jump directly to any chapter in a book

• 🎯 Absolute Page Navigation - Navigate to specific page positions with session tracking to maintain reading progress across requests

• 👀 Preview Mode - Look ahead or behind in a book without changing your actual reading position

• 🖥️ VNC Remote Access - View and interact with emulators in real-time through browser-based VNC streaming via WebSocket

• 🧬 Profile Cloning - Create new user profiles from authenticated "seed" templates for rapid deployment

• 🧊 Cold Storage Archival - Automatically archive inactive user profiles to cloud storage (DigitalOcean Spaces) and restore on demand

• 🤖 State Machine Architecture - Intelligent handling of Kindle app states including login screens, dialogs, popups, and reading modes

• 🔄 Request Deduplication - Smart request queuing with priority handling to prevent conflicting automation commands

• 💤 Idle Management - Automatic shutdown of inactive emulators to conserve server resources

• 📸 Snapshot Recovery - Save and restore emulator states for fast startup and crash recovery

• 📊 Admin Dashboard - Built-in Flask-Admin interface for monitoring users, VNC instances, and system state

• 🐛 Sentry Integration - Production-ready error tracking and monitoring with intelligent error filtering

• 🎭 Device Fingerprint Randomization - Randomize device identifiers (MAC addresses, serial numbers, Android IDs) to avoid detection

---

Quick Start

To get started with the automation setup, you need to install Appium and the necessary drivers. Follow the steps below:

1. Install Appium: Appium is a tool for automating mobile applications. You can install it using npm (Node Package Manager). If you don't have npm installed, you need to install Node.js first, which includes npm.

   npm install -g appium

2. Install UiAutomator2 Driver: After installing Appium, you need to install the UiAutomator2 driver, which is required for Android automation.

   appium driver install uiautomator2

3. Verify Installation: To verify that Appium and the UiAutomator2 driver are installed correctly, you can run the following command:

   appium -v

   This should display the version of Appium installed. Additionally, you can check the installed drivers by running:

   appium driver list

   This should list uiautomator2 among the installed drivers.

Now you are ready to proceed with the automation setup and run the scripts.

4. Install Android Studio: To run Android emulators and use Android SDK tools, you need to install Android Studio. Follow the steps below to install Android Studio:

   • Download Android Studio from the official website: https://developer.android.com/studio
   • Follow the installation instructions for your operating system (Windows, macOS, or Linux).
   • During the installation, make sure to install the Android SDK and Android Virtual Device (AVD) components.

5. Set Up Android SDK: After installing Android Studio, you need to set up the Android SDK and ensure that the adb (Android Debug Bridge) tool is available in your system's PATH.

   • Open Android Studio and go to Preferences (macOS) or Settings (Windows/Linux).
   • Navigate to Appearance & Behavior > System Settings > Android SDK.
   • Make sure the SDK Path is set to the default location or a location of your choice.
   • Under the SDK Tools tab, ensure that the following tools are installed:
     • Android SDK Build-Tools
     • Android Emulator
     • Android SDK Platform-Tools
     • Android SDK Tools

6. Add Android SDK to PATH: To use adb and other Android SDK tools from the command line, you need to add the Android SDK's platform-tools directory to your system's PATH.

   • For macOS/Linux:

     echo 'export ANDROID_HOME=~/Library/Android/sdk' >> ~/.bash_profile
     echo 'export PATH=$PATH:$ANDROID_HOME/platform-tools' >> ~/.bash_profile
     source ~/.bash_profile

   • For Windows:

     • Open System Properties (right-click on This PC or My Computer and select Properties).
     • Click on Advanced system settings and then Environment Variables.
     • Under System variables, find the Path variable, select it, and click Edit.
     • Add the path to the platform-tools directory (e.g., C:\Users\<YourUsername>\AppData\Local\Android\Sdk\platform-tools).

7. Verify Android SDK Installation: To verify that the Android SDK is installed correctly and adb is available, you can run the following command:

   adb devices

   This should list any connected Android devices or running emulators. If no devices are listed, ensure that your device is connected and USB debugging is enabled, or that an emulator is running.

Now you are ready to proceed with the automation setup and run the scripts.

8. Set Up Virtual Environment: To ensure that all dependencies are installed in an isolated environment, you should set up a virtual environment using virtualenv.

   • Install virtualenv if you haven't already:

     uv install virtualenv

   • Create a virtual environment named kindle-automator:

     virtualenv kindle-automator

   • Activate the virtual environment:

     • For macOS/Linux:

       source kindle-automator/bin/activate

     • For Windows:

       .\kindle-automator\Scripts\activate

9. Install Dependencies: Once the virtual environment is activated, install the required dependencies using make deps:

   make deps

10. Database Setup (Required for first-time setup): The application requires PostgreSQL and Redis to be running. These are provided via Docker containers from the web-app project:

    # Start Docker containers (PostgreSQL and Redis)
    cd ../web-app && make fast
    cd ../kindle-automator
    
    # Initialize the database (creates kindle_dev database, runs migrations, imports data)
    make db-setup

    This only needs to be done once when setting up a new development environment. The make db-setup command will:

    • Create the kindle_dev database if it doesn't exist
    • Run all database migrations
    • Import any existing user data from user_data/users.json

    Note: Make sure Docker Desktop is running and the sol_postgres container is available on port 5496.

11. Configuration: The application uses a .env file for API keys and other configuration. Create your .env file by copying the example:

    cp .env.example .env

    Then edit the .env file to add your API keys:

    # API Keys 
    MISTRAL_API_KEY=your-actual-mistral-api-key
    

    This file is listed in .gitignore to ensure your API keys are never committed to the repository.

    Note: Amazon credentials (email and password) must be provided in the /auth API request, and captcha solutions (if needed) must be provided to the /captcha endpoint. These are not read from environment variables or configuration files. The system will automatically initialize when needed, but you must authenticate with the /auth endpoint before accessing other features.

12. Running Tests: To run the integration tests, you need to generate authentication tokens first:

    # Generate staff authentication token
    make test-staff-auth
    
    # Generate web authentication token (see ../web-app/CLAUDE.md for details)
    make test-web-auth

    After generating the tokens, add them to your .env file:

    INTEGRATION_TEST_STAFF_AUTH_TOKEN=<your-staff-token>
    WEB_INTEGRATION_TEST_AUTH_TOKEN=<your-web-token>
    

    Then run the tests:

    make test

13. Run the Script: After setting up the virtual environment, installing dependencies, and configuring your environment, you can run the automation script using:

    make run

    This will install the Kindle APK and start the Flask server so it's ready to automate the app.

User Profiles

The system now includes AVD profile management that creates and manages separate Android Virtual Devices for each user account. This allows you to:

• Maintain multiple authenticated Kindle accounts simultaneously
• Switch between accounts without having to re-authenticate each time
• Preserve the state of each account, including downloaded books and login credentials

Each time a new email address is sent to the /auth endpoint, the system automatically:

1. Checks if a profile already exists for this email
2. If it exists, switches to that profile
3. If not, creates a new AVD profile for the email
4. Initializes the Kindle app in the selected profile

Profile Management API Endpoints

• GET /profiles - List all available profiles and the current active profile
• POST /profiles - Create, delete, or switch profiles with action parameter:
  • {"action": "create", "email": "user@example.com"}
  • {"action": "delete", "email": "user@example.com"}
  • {"action": "switch", "email": "user@example.com"}
• When using /auth, you can add a recreate parameter to delete and recreate a profile:
  • {"email": "user@example.com", "password": "password123", "recreate": true}
  • This is useful when you want to start fresh with a clean profile

Using AVDs Created in Android Studio

For M1/M2/M4 Mac users who need to create and manage AVDs directly in Android Studio, we've added a special workflow:

1. Create your AVD in Android Studio with your preferred settings
2. Register the AVD with Kindle Automator using one of the following:

   # Interactive registration
   make register-avd
   
   # Complete workflow (recommended)
   make android-studio-avd

3. Start the emulator manually from Android Studio
4. Run the Kindle Automator server with:

   make server

This approach allows you to use manually created AVDs with the profile tracking system. The system will associate your Android Studio AVD with a specific email account and track which one is active, without trying to start the emulator itself.

Special Notes for M1/M2/M4 Mac Users

Due to compatibility issues with Android emulation on ARM-based Macs, the profile system has been adapted to:

1. Create and track profiles without attempting to start the emulator automatically
2. Provide several ways to manage the emulator:

Development Workflow Commands

• Start both emulator and server together:

  make dev

  This single command starts the emulator for the current profile and then launches the server.

• Start just the emulator for the current profile:

  make run-emulator

  This automatically selects the AVD associated with the current profile (e.g., after using make profile-switch).

• Choose which AVD to start:

  make run-emulator-choose

  This shows a list of all available AVDs and lets you select which one to run.

• Register an Android Studio AVD:

  make register-avd

  This allows you to associate an Android Studio AVD with an email profile.

Typical Development Workflow

1. Create or switch to a profile:

   make profile-create EMAIL=user@example.com

   or

   make profile-switch EMAIL=user@example.com

2. Start everything with one command:

   make dev

3. Test the API endpoints:

   make test-auth EMAIL=user@example.com PASSWORD=yourpassword

The system will track which profile is "current" even if the emulator starts separately.

Android Studio Workflow (for M4 Macs)

1. Create AVDs directly in Android Studio
2. Register them with Kindle Automator:

   make android-studio-avd

3. Start the emulator from Android Studio
4. Start the server:

   make server

5. Test the API endpoints:

   make test-auth EMAIL=user@example.com PASSWORD=yourpassword

In production environments (Linux servers), the emulator starts automatically as part of the profile switching process without needing manual intervention.

Important files

• server/server.py: Flask server on port 4098
• views/state_machine.py: Handles Kindle app states and transitions
• views/core/app_state.py: Kindle views and app states
• views/core/avd_profile_manager.py: Manages AVD profiles for different user accounts
• views/transition.py: Controller between states and handlers
• views/view_inspector.py: App view identifying, interacting with appium driver
• handlers/*_handler.py: Various handlers for different app states
• automator.py: Glue between the Kindle side and the server side, ensuring driver is running
• driver.py: Handles Appium driver initialization and management