Embench Integration & Workflow Enhancement [compiling workloads using QEMU and BBVs generation] (#273)

# Overview

This pull request significantly enhances the `QEMU-BBVS-Flow` by
integrating the **Embench IoT benchmark suite** and streamlining the
entire analysis workflow. The primary goal is to provide a more
powerful, efficient, and flexible tool for computer architecture
research, addressing the critical challenge of long simulation times by
automating SimPoint analysis for a wider range of standard benchmarks.

By containerizing a pre-built RISC-V toolchain and pre-compiling the
Embench suite, this update drastically reduces setup and execution
times, allowing researchers to focus on analysis rather than environment
configuration.

## Key Features & Enhancements

### 1. Full Embench IoT Suite Integration

- **Pre-built Workloads**: The `Dockerfile` now clones and pre-builds
all 19 Embench workloads for the `riscv32` architecture. This ensures
that the benchmarks are ready for immediate analysis without any
compilation overhead during runtime.
- **Standalone Embench Runner**: A new `run_embench_simple.sh` script
allows for fine-grained control, enabling users to:
  - List all available Embench workloads.
  - Run individual workloads with or without timing.
- Generate Basic Block Vector (BBV) traces for specific benchmarks on
demand.
- **Integrated Analysis Pipeline**: The main `run_workload.sh` script
now recognizes `embench` as a workload type, automating the end-to-end
SimPoint analysis for the entire suite.

### 2. Improved Generic Workload Support

- **Flexible Workload Naming**: The system now supports any custom
workload name (minimum 3 characters), replacing the previously hardcoded
`"custom"` type.
- **Automatic Build System Detection**: The `setup_workload.sh` script
intelligently builds the user-provided source code by looking for either
a `Makefile` or a `build.sh` script within the source directory.

### 3. Massive Efficiency Gains

- **Pre-built RISC-V Toolchain**: The Docker image is now based on
`ribeirovsilva/riscv-toolchain:latest` (by @vinicius-r-silva ), which
includes a pre-compiled RISC-V toolchain. This single change cuts the
Docker build time from over an hour to just a few minutes.
- **Optimized Analysis Scripts**: The `run_analysis.sh` script now
filters binaries based on the selected workload type, ensuring that only
relevant benchmarks are processed, which speeds up batch analysis.

### 4. Robust Orchestration and Testing

- **Unified Scripts**: The `build_docker.sh`, `run_workload.sh`, and
`setup_workload.sh` scripts work in harmony to provide a seamless user
experience, from building the environment to generating final results.
- **Comprehensive Demo**: A new `integration_demo.sh` script is included
to showcase and verify all new features, including Embench integration,
BBV generation, backward compatibility with Dhrystone, and the enhanced
generic workload handling.

## How to Test

### 1. Clone the repository and make all scripts executable
```bash
chmod +x *.sh
````

### 2. Run the complete integration demo

This script will build the image and test all key functionalities
automatically.

```bash
./integration_demo.sh
```

### 3. Manual Verification Steps

#### Build the Docker image

```bash
./build_docker.sh
```

#### List available Embench workloads

```bash
docker run --rm qemu-simpoint-unified /run_embench_simple.sh --list
```

#### Run a single Embench workload and generate its BBV trace

```bash
# Create an output directory on the host
mkdir -p simpoint_output

docker run --rm -v "$(pwd)/simpoint_output:/output" qemu-simpoint-unified /run_embench_simple.sh crc32 --bbv
docker run --rm -v "$(pwd)/simpoint_output:/output" qemu-simpoint-unified /run_embench_simple.sh [workload_name] --bbv

# Verify that the BBV file was created
ls -lh simpoint_output/crc32_bbv.0.bb
```

#### Test the enhanced generic workload handler

(The `integration_demo.sh` script creates a temporary `test_workload`
directory to test this, or you can use your own.)

```bash
# Assuming you have a project in ./my-riscv-project with a Makefile
./run_workload.sh my-riscv-project "$(pwd)/my-riscv-project" "-static" "riscv64"
```

---------

Signed-off-by: Aditi Mehta <[email protected]>

Author: aditi75432

traces/qemu-bbvs-flow/Dockerfile

@@ -1,14 +1,11 @@
-FROM ubuntu:22.04
+FROM ribeirovsilva/riscv-toolchain:latest
 
 # Avoid interactive prompts during installation
 ENV DEBIAN_FRONTEND=noninteractive
 ENV TZ=Etc/UTC
 
-# Update package lists
-RUN apt update && apt upgrade -y
-
-# Install all dependencies for QEMU compilation and workloads
-RUN apt install -y \
+# Update package lists and install additional dependencies
+RUN apt-get update && apt-get install -y \
   git-email \
   libaio-dev \
   libbluetooth-dev \
@@ -67,12 +64,14 @@ RUN apt install -y \
   libexpat-dev \
   cmake \
   libglib2.0-dev \
-  gcc-riscv64-linux-gnu \
   libslirp-dev \
   pkg-config \
   libpixman-1-dev \
   python3-dev \
-  python3-venv
+  python3-venv \
+  qemu-user \
+  qemu-user-static \
+  && rm -rf /var/lib/apt/lists/*
 
 # Clone repositories
 RUN git clone https://gitlab.com/qemu-project/qemu.git
@@ -93,14 +92,33 @@ RUN make -j$(nproc)
 ENV PATH="$PATH:/SimPoint/bin"
 
 # Create workload directory and output directory
-RUN mkdir -p /workloads /output
+RUN mkdir -p /workloads /output /workspace
 
-# Copy workload setup script
-COPY setup_workload.sh /setup_workload.sh
-RUN chmod +x /setup_workload.sh
+# Clone and prepare Embench repository
+WORKDIR /workspace
+RUN git clone --recursive https://github.com/embench/embench-iot.git
+RUN sed -i '36s/^typedef uint8_t bool;/\/\/ &/' /workspace/embench-iot/src/wikisort/libwikisort.c
+
+# Pre-build Embench workloads to avoid build time during analysis
+WORKDIR /workspace/embench-iot
+RUN python3 ./build_all.py \
+    --arch riscv32 \
+    --chip generic \
+    --board ri5cyverilator \
+    --cc riscv64-unknown-elf-gcc \
+    --cflags="-c -O2 -ffunction-sections -march=rv32imfdc -mabi=ilp32d" \
+    --ldflags="-Wl,-gc-sections -march=rv32imfdc -mabi=ilp32d" \
+    --user-libs="-lm"
 
-# Copy analysis script
+# Copy setup and analysis scripts
+COPY setup_workload.sh /setup_workload.sh
 COPY run_analysis.sh /run_analysis.sh
-RUN chmod +x /run_analysis.sh
+COPY run_embench_simple.sh /run_embench_simple.sh
+RUN chmod +x /setup_workload.sh /run_analysis.sh /run_embench_simple.sh
+
+# Verify QEMU works with a test workload
+RUN echo "Testing QEMU with pre-built embench workload..." && \
+    qemu-riscv32 /workspace/embench-iot/bd/src/crc32/crc32 && \
+    echo "QEMU execution test successful"
 
-WORKDIR /
+WORKDIR /

traces/qemu-bbvs-flow/README.md

@@ -1,4 +1,4 @@
-# QEMU BBVs Flow
+# QEMU SimPoint Unified Analysis
 
 A unified, containerized workflow for running various workloads on QEMU with SimPoint analysis for program phase detection and basic block vector generation.
 
@@ -27,24 +27,40 @@ This project provides a unified Docker-based environment for:
 
 ### 1. Clone Repository
 ```bash
-git clone https://github.com/riscv-software-src/riscv-perf-model.git
-cd traces/qemu-bbvs-flow
+git clone https://github.com/yourusername/qemu-bbvs-flow.git
+cd qemu-bbvs-flow
 ```
 
 ### 2. Run Dhrystone Analysis
 ```bash
-./build_and_run.sh dhrystone
+./build_and_run.sh dhrystone 
+# OR
+
+./build_docker.sh 
+./run_workload.sh dhrystone
 ```
 
 ### 3. Run Embench Analysis
 ```bash
-./build_and_run.sh embench
+#Embench gets compiled during the build itself
+./build_docker.sh 
+
+# Run specific Embench workload with BBV 
+docker run --rm -v $(pwd)/simpoint_output:/output qemu-simpoint-unified /run_embench_simple.sh [workload_name] --bbv
+#[For example] – 
+docker run --rm -v $(pwd)/simpoint_output:/output qemu-simpoint-unified /run_embench_simple.sh crc32 --bbv
+
 ```
 
 ### 4. Run Custom Workload
 ```bash
 ./build_and_run.sh custom /path/to/your/source "-O2 -static" riscv64
 ```
+### 5. Individual Component Execution
+```bash
+./build_docker.sh
+./run_workload.sh [workload_type] [parameters]
+```
 
 ## Prerequisites
 
@@ -75,8 +91,8 @@ cd traces/qemu-bbvs-flow
 
 3. **Open WSL 2 Terminal** and clone repository:
    ```bash
-   git clone https://github.com/riscv-software-src/riscv-perf-model.git
-   cd traces/qemu-bbvs-flow
+   git clone https://github.com/yourusername/qemu-bbvs-flow.git
+   cd qemu-bbvs-flow
    chmod +x *.sh
    ```
 
@@ -95,9 +111,8 @@ cd traces/qemu-bbvs-flow
 
 2. **Clone and Setup**:
    ```bash
-   git clone https://github.com/riscv-software-src/riscv-perf-model.git
-   cd traces/qemu-bbvs-flow
-
+   git clone https://github.com/yourusername/qemu-bbvs-flow.git
+   cd qemu-bbvs-flow
    chmod +x *.sh
    ```
 
@@ -129,7 +144,8 @@ cd traces/qemu-bbvs-flow
 
 #### Embench with Custom Settings
 ```bash
-./build_and_run.sh embench "" "-march=rv32i -mabi=ilp32 -static" riscv32 50 20
+#Embench gets compiled during the build itself
+./build_docker.sh
 ```
 
 #### Custom Workload
@@ -392,17 +408,18 @@ ls -lh simpoint_output/*.bb
 
 ```
 qemu-bbvs-flow/
-├── README.md                    # This file
 ├── Dockerfile                   # Unified Docker image
 ├── setup_workload.sh           # Workload setup script
 ├── run_analysis.sh             # Analysis execution script
-├── build_docker.sh             # Docker build automation
+├── run_embench_simple.sh       # embench compilation script
+├── build_docker.sh             # Docker image build script
 ├── run_workload.sh             # Workload execution script
 ├── build_and_run.sh            # Complete workflow script
-├── workload_config.json        # Configuration file
+├── workload_config.json        # Workload configuration
+├── README.md                   # User documentation
 ├── documentation/
-│   └── technical_details.md    # Technical documentation
-└── simpoint_output/            # Results directory (created after execution)
+│   └── technical_details.md    # This file
+└── simpoint_output/            # Generated results (after execution)
 ```
 
 ## Contributing
@@ -452,4 +469,4 @@ For support and questions:
 - Support for Dhrystone, Embench, and custom workloads
 - Configurable compiler flags and QEMU architectures
 - Automated build and run scripts
-- Comprehensive documentation
+- Comprehensive documentation

traces/qemu-bbvs-flow/documentation/Technical_Documentation

@@ -27,6 +27,7 @@ qemu-bbvs-flow/
 ├── Dockerfile                   # Unified Docker image
 ├── setup_workload.sh           # Workload setup script
 ├── run_analysis.sh             # Analysis execution script
+├── run_embench_simple.sh       # embench compilation script
 ├── build_docker.sh             # Docker image build script
 ├── run_workload.sh             # Workload execution script
 ├── build_and_run.sh            # Complete workflow script
@@ -184,22 +185,3 @@ simpoint -loadFVFile input.bb -maxK 30 -saveSimpoints output.simpoints -saveSimp
 3. **Memory Issues**: Increase Docker memory limits
 4. **Network Issues**: Verify internet connectivity for source downloads
 
-### Debug Mode
-Enable verbose output by setting environment variables:
-```bash
-export VERBOSE=1
-export DEBUG=1
-```
-
-## Future Enhancements
-
-### Planned Features
-1. Support for additional architectures (ARM, x86)
-2. Integration with performance monitoring tools
-3. Automated result visualization
-4. Distributed analysis support
-
-### Configuration Extensions
-1. YAML-based configuration
-2. Template-based workload definitions
-3. Profile-based execution modes