AGV Panadería - Automated Guided Vehicle for Contactless Bakery Store

An autonomous cyber-physical system simulating a contactless store environment using RFID technology, AWS cloud services, and computer vision. The AGV autonomously performs purchases on behalf of users, enabling real-time item tracking, cloud-based processing, and automated delivery within a simulated bakery environment.

🎯 Project Overview

• Developed a cyber-physical system simulating a contactless store by integrating RFID technology and AWS cloud services with an AGV that autonomously performs purchases on behalf of the user. Enabled real-time item tracking, cloud-based processing, and automated delivery within the simulated environment.

• Deployed low-level SSD object detection in C++ on a Raspberry Pi for real-time, resource-constrained inference.

• Programmed an RFID module to read customer tags and verify item availability by sending requests through an AWS IoT Gateway. Triggered a AWS Lambda function to check inventory stored in S3 and update the list by removing items selected for purchase, enabling an automated shopping experience.

🏗️ System Architecture

┌─────────────────────────────────────────────────────┐
│         Control System (Keyboard Interface)         │
│         MQTT Client - Wireless Command Relay        │
└────────────────────┬────────────────────────────────┘
                     │ MQTT Broker (192.168.137.1)
                     ▼
      ┌──────────────────────────────────┐
      │  Main Orchestrator (main.py)     │
      │  Process Manager & MQTT Router   │
      └──────────────┬───────────────────┘
                     │
        ┌────────────┼────────────┐
        │            │            │
        ▼            ▼            ▼
   ┌────────┐  ┌──────────┐ ┌──────────┐
   │ Motors │  │Rekognition│ │  RFID    │
   │Control │  │ Detection │ │ Scanner  │
   └────┬───┘  └─────┬────┘ └─────┬────┘
        │            │            │
        │            │            │
        ▼            ▼            ▼
   AGV Motion   Face Detection  AWS IoT
   (Serial)     + YOLOv4 Bread   Core
                                  │
                                  ▼
                          ┌──────────────┐
                          │ AWS Services │
                          │ - DynamoDB   │
                          │ - Lambda     │
                          │ - S3         │
                          └──────────────┘

🔧 Hardware Components

Raspberry Pi Configuration

• Platform: Raspberry Pi (primary compute module)
• Camera: OpenCV VideoCapture for real-time vision
• RFID Reader: RC522 MIFARE module (SPI interface)
• Motor Controller: Serial communication via /dev/ttyS0 (115200 baud)
• GPIO: RPi.GPIO library for hardware interfacing

Motor Control Encoding

0b0001 = Forward (w)
0b0010 = Left (a)
0b0100 = Backward (s)
0b1000 = Right (d)

☁️ AWS Cloud Integration

Services Used

1. AWS IoT Core

• Endpoint: a1bo3h1pmj4c6m-ats.iot.us-east-2.amazonaws.com:8883
• Protocol: MQTT over TLS 1.2
• Authentication: X.509 certificates
• Purpose: Secure RFID data transmission from AGV to cloud

2. AWS Rekognition

• Region: us-east-2
• Collection: profesores (authorized personnel database)
• Purpose: Real-time facial recognition for access control
• Features: Face indexing, matching, audio alerts

3. AWS DynamoDB

Tables:

• profesores - Face recognition mapping (RekognitionId → FullName)
• Compras - Transaction records (Pan, Timestamp, Precio, Fecha, Cantidad)
• Inventario - Real-time stock tracking (decremented per purchase)

4. AWS S3

• Bucket: panaderia-equipo5
• Contents:
  • Facial recognition training images
  • Analytics graphs and reports
  • Historical transaction data

5. AWS Lambda Functions

• Rekognition Lambda: Indexes uploaded face images into collection
• RFID Lambda: Processes purchase transactions, generates analytics, updates inventory

🤖 Computer Vision & Object Detection

YOLOv4-Tiny Implementation (C++)

• Model: yolov4-tiny-opt.bin (23.5 MB optimized for Raspberry Pi)
• Framework: NCNN (Tencent's neural network inference framework)
• Confidence Threshold: 80%
• Target Platform: ARM-based edge devices (resource-constrained inference)

Bread Detection Classes (5 types)

Class ID	Product	Description
1, 4	BOLILLO	Traditional Mexican roll
2, 5	CONCHA	Sweet shell-shaped bread
3, 6	DONA	Donut
7, 9	OREJITA	Ear-shaped pastry
8, 10	MUFFIN	Muffin

📡 MQTT Communication Protocol

Topic Architecture

AGV/Control          - Main AGV commands (Start/Stop/DetectRFID)
AGV/Motores          - Motor movement commands (4-bit control)
Camara/Deteccion     - Camera/detection commands (Pan detection)
Fin                  - Process completion signals
iot/Pan              - RFID data publishing to AWS IoT
Sim/Inicio           - Simulation initialization
Sim/Reset            - Simulation reset
Sim/Estacion[1-3]    - Store location selection

Message Flow

1. Control system publishes commands via MQTT
2. Main orchestrator routes messages to appropriate subsystems
3. Subsystems complete tasks and signal completion
4. Cloud services process data and update databases

🔐 RFID Workflow

Purchase Process

1. AGV navigates to store location (1-3)
2. RFID reader scans customer tag
3. Reads blocks 4-18 containing purchase data
4. Formats JSON payload:

   {
     "Compras": [
       {"Pan": "CONCHA", "Precio": 15, "Fecha": "2025-01-16", "Can": 2}
     ],
     "Tienda": 1
   }

5. Publishes to AWS IoT Core via TLS
6. Lambda function validates inventory
7. DynamoDB updated (transaction logged, inventory decremented)
8. Analytics graphs generated and stored in S3

📊 Analytics & Reporting

Generated Insights

• Per-store daily quantity/price reports
• Historical cross-store analysis
• Inventory tracking graphs
• Timestamp-based trend analysis

Trigger Mechanism

• Store #3 visits trigger comprehensive historical analysis
• Matplotlib graphs saved to S3
• Accessible for business intelligence

🚀 System Workflow

1. Initialization

python main.py  # Start main orchestrator

2. Control Commands (Sistema_De_Control.py)

• i - Initialize simulation
• k - Trigger RFID detection
• m - Start bread detection (YOLOv4)
• p - Pause/stop
• r - Reset simulation
• w/a/s/d - Motor control (forward/left/backward/right)

3. Autonomous Operation

1. Navigation: AGV moves between 3 store locations
2. Recognition: Facial recognition greets authorized personnel
3. Detection: YOLOv4 identifies bread products in camera feed
4. Scanning: RFID reader captures purchase information
5. Processing: Cloud Lambda validates and logs transaction
6. Inventory: Real-time stock updates in DynamoDB
7. Analytics: Historical data visualization in S3

📁 Project Structure

AGVPanaderia/
├── main.py                    # Central MQTT orchestrator (70 lines)
├── ControlMotor/
│   └── comunicacion_serial.py # Motor serial interface (37 lines)
├── Rekognition/
│   ├── deteccion_profes.py    # Facial recognition system (176 lines)
│   └── Lambda.py              # Face indexing Lambda (71 lines)
├── RFID/
│   ├── read_block.py          # RFID scanner (140 lines)
│   ├── Lambda.py              # Transaction processing (197 lines)
│   └── Certificates/          # AWS IoT TLS certificates
│       ├── root-ca.pem
│       ├── private.pem.key
│       └── certificate.pem.crt
├── YoloDetect/
│   ├── yolov4.cpp             # C++ object detection (261 lines)
│   ├── yolov4                 # Compiled binary
│   ├── yolov4-tiny-opt.bin    # Optimized model (23.5 MB)
│   └── yolov4-tiny-opt.param  # Model parameters
└── Sistema_De_Control/
    └── Sistema_Control.py     # Keyboard control interface (90 lines)

🛠️ Technologies & Dependencies

Languages

• Python 3 - Primary development language (~1105 lines)
• C++ - YOLOv4 inference engine (~261 lines)

Python Libraries

paho-mqtt              # MQTT messaging
boto3                  # AWS SDK (DynamoDB, S3, Rekognition)
AWSIoTPythonSDK        # AWS IoT Core MQTT client
opencv-python (cv2)    # Computer vision
mfrc522                # RFID RC522 module
RPi.GPIO               # Raspberry Pi GPIO control
pynput                 # Keyboard input handling
pandas                 # Data analysis
numpy                  # Numerical operations
matplotlib             # Graph generation

C++ Dependencies

• NCNN - Neural network inference framework
• OpenCV - Computer vision library

📈 Performance Metrics

• Face Detection Rate: 2 FPS (balanced accuracy/performance)
• RFID Scan Time: <2 seconds per tag
• Object Detection Inference: Real-time on Raspberry Pi
• MQTT Latency: <100ms local network
• AWS IoT Round Trip: <500ms (depends on network)

🔒 Security Features

• TLS 1.2 Encryption: All AWS IoT communications
• X.509 Certificates: Device authentication
• Face Recognition: Access control for authorized personnel
• RFID Validation: Cloud-based inventory verification

🎓 Authorized Personnel Database

The system recognizes the following authorized personnel:

• David Navarro
• Liz Machado
• Luis Yepez
• Adriana
• Rodrigo Regalado

Each recognized person triggers a personalized audio greeting.

📝 Data Flow Summary

Input Sources

• RFID Scans: Store/product/quantity/price information
• Camera Feeds: Real-time bread detection and face recognition
• Control Commands: MQTT keyboard input

Cloud Processing

• IoT Core: Secure message brokering
• Lambda: Event-driven analytics and inventory updates
• DynamoDB: Transactional record storage
• S3: Long-term data archival and graph storage
• Rekognition: Face indexing and matching

Outputs

• Motor Commands: AGV navigation
• Audio Alerts: Personnel greetings
• Analytics Graphs: Business intelligence
• Inventory Updates: Real-time stock tracking

🏪 Multi-Store Operation

The system supports 3 simulated store locations:

• Store cycles tracked in num_tienda.txt
• Each visit increments counter (1 → 2 → 3 → 1)
• Store #3 triggers comprehensive historical analysis
• Per-store analytics for comparative insights

📊 Recent Development

Git commit history shows active development:

• 14a44b3 - Added control system
• cf89ea7 - Integrated control system
• e3ad034 - Added RFID Lambda integration
• Focus areas: RFID, YOLO detection, serial communication

🔮 Future Enhancements

• Multi-AGV fleet coordination
• Mobile app integration for customer orders
• Expanded product catalog (beyond bread)
• Predictive inventory management using ML
• Enhanced navigation with SLAM algorithms

📄 License

This project was developed as an educational cyber-physical system demonstration.

👥 Contributors

Equipo 5 - Embedded Systems & Cloud Integration Team

---

Note: This system requires Raspberry Pi hardware, AWS account with configured services, and MQTT broker setup for full functionality.