Skip to content

Commit 53ef327

Browse files
GOODBOY008GOODBOY008
committed
feat: add benchmark module for FastExcel performance analysis
1 parent 52269ae commit 53ef327

23 files changed

+8518
-0
lines changed

‎fastexcel-benchmark/benchmark.md‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# FastExcel Benchmark Guide
2+
3+
This guide provides a comprehensive overview of the FastExcel benchmark module, including how to run benchmarks, interpret the results, and contribute new benchmarks.
4+
5+
## Overview
6+
7+
The benchmark module is designed to measure and analyze the performance of FastExcel for various Excel operations, such as reading, writing, and filling data. It uses the [Java Microbenchmark Harness (JMH)](https://openjdk.java.net/projects/code-tools/jmh/) to ensure accurate and reliable benchmark results.
8+
9+
The key goals of the benchmark module are:
10+
11+
- To provide a standardized way to measure the performance of FastExcel.
12+
- To track performance regressions and improvements over time.
13+
- To compare the performance of FastExcel with other Excel libraries, such as Apache POI.
14+
- To help users make informed decisions about how to use FastExcel for their specific needs.
15+
16+
## How to Run Benchmarks
17+
18+
There are two primary ways to run the benchmarks: using the `benchmark-runner.sh` script or using Maven profiles.
19+
20+
### Using the `benchmark-runner.sh` Script
21+
22+
The `benchmark-runner.sh` script provides a convenient way to run the benchmarks with various options.
23+
24+
**Usage:**
25+
26+
```bash
27+
./fastexcel-benchmark/scripts/benchmark-runner.sh [OPTIONS]
28+
```
29+
30+
**Options:**
31+
32+
| Option | Description | Default |
33+
|---|---|---|
34+
| `-p`, `--profile` | Benchmark profile (quick, standard, comprehensive) | `standard` |
35+
| `-o`, `--output` | Output directory for results | `benchmark-results` |
36+
| `-j`, `--java-version` | Java version to use | `11` |
37+
| `-m`, `--memory` | JVM heap size | `4g` |
38+
| `-t`, `--pattern` | Benchmark pattern to match | |
39+
| `-d`, `--dataset` | Dataset size (SMALL, MEDIUM, LARGE, EXTRA_LARGE, ALL) | `ALL` |
40+
| `-f`, `--format` | Output format (json, csv, text) | `json` |
41+
| `-r`, `--regression` | Enable regression analysis | |
42+
| `-v`, `--verbose` | Enable verbose output | |
43+
| `-h`, `--help` | Show this help message | |
44+
45+
**Profiles:**
46+
47+
- `quick`: Fast execution for development (2 warmup, 3 measurement, 1 fork).
48+
- `standard`: Balanced execution for CI (3 warmup, 5 measurement, 1 fork).
49+
- `comprehensive`: Thorough execution for nightly (5 warmup, 10 measurement, 2 forks).
50+
51+
**Examples:**
52+
53+
- Run standard benchmarks:
54+
```bash
55+
./fastexcel-benchmark/scripts/benchmark-runner.sh --profile standard
56+
```
57+
- Run quick benchmarks for read operations only:
58+
```bash
59+
./fastexcel-benchmark/scripts/benchmark-runner.sh --profile quick --pattern "ReadBenchmark"
60+
```
61+
- Run comprehensive benchmarks with regression analysis:
62+
```bash
63+
./fastexcel-benchmark/scripts/benchmark-runner.sh --profile comprehensive --regression
64+
```
65+
66+
### Using Maven Profiles
67+
68+
You can also run the benchmarks using Maven profiles. This is useful for integrating the benchmarks into a CI/CD pipeline.
69+
70+
**Usage:**
71+
72+
```bash
73+
mvn clean install -f fastexcel-benchmark/pom.xml -P <profile> -Dbenchmark.pattern=<pattern>
74+
```
75+
76+
**Profiles:**
77+
78+
- `benchmark`: The primary profile for running benchmarks.
79+
80+
**Examples:**
81+
82+
- Run all benchmarks:
83+
```bash
84+
mvn clean install -f fastexcel-benchmark/pom.xml -P benchmark
85+
```
86+
- Run a specific benchmark:
87+
```bash
88+
mvn clean install -f fastexcel-benchmark/pom.xml -P benchmark -Dbenchmark.pattern=ReadBenchmark
89+
```
90+
91+
## Benchmark Suites
92+
93+
The benchmark module includes the following suites:
94+
95+
- **Comparison:** Benchmarks comparing FastExcel with other libraries (e.g., Apache POI).
96+
- **Config:** Benchmarks related to configuration options.
97+
- **Core:** Core benchmark classes and utilities.
98+
- **Data:** Benchmarks related to data handling and processing.
99+
- **Memory:** Benchmarks focused on memory usage.
100+
- **Operations:** Benchmarks for specific operations like read, write, and fill.
101+
- **Streaming:** Benchmarks for streaming operations.
102+
103+
## Interpreting Results
104+
105+
The benchmarks produce output in the format specified by the `--format` option. The default format is JSON.
106+
107+
The output includes the following information:
108+
109+
- **Benchmark:** The name of the benchmark.
110+
- **Mode:** The benchmark mode (e.g., `thrpt` for throughput, `avgt` for average time).
111+
- **Threads:** The number of threads used.
112+
- **Forks:** The number of forks used.
113+
- **Warmup Iterations:** The number of warmup iterations.
114+
- **Measurement Iterations:** The number of measurement iterations.
115+
- **Score:** The benchmark score.
116+
- **Score Error:** The error of the benchmark score.
117+
- **Unit:** The unit of the benchmark score (e.g., `ops/s` for operations per second).

‎fastexcel-benchmark/pom.xml‎

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<modelVersion>4.0.0</modelVersion>
6+
7+
<parent>
8+
<groupId>cn.idev.excel</groupId>
9+
<artifactId>fastexcel-parent</artifactId>
10+
<version>1.3.0</version>
11+
<relativePath>../pom.xml</relativePath>
12+
</parent>
13+
14+
<artifactId>fastexcel-benchmark</artifactId>
15+
<name>fastexcel-benchmark</name>
16+
<description>Comprehensive benchmark module for FastExcel performance analysis</description>
17+
18+
<properties>
19+
<jmh.version>1.37</jmh.version>
20+
<uberjar.name>benchmarks</uberjar.name>
21+
</properties>
22+
23+
<dependencies>
24+
<!-- FastExcel Core -->
25+
<dependency>
26+
<groupId>cn.idev.excel</groupId>
27+
<artifactId>fastexcel</artifactId>
28+
<version>${project.version}</version>
29+
</dependency>
30+
31+
<!-- JMH Dependencies -->
32+
<dependency>
33+
<groupId>org.openjdk.jmh</groupId>
34+
<artifactId>jmh-core</artifactId>
35+
<version>${jmh.version}</version>
36+
</dependency>
37+
<dependency>
38+
<groupId>org.openjdk.jmh</groupId>
39+
<artifactId>jmh-generator-annprocess</artifactId>
40+
<version>${jmh.version}</version>
41+
<scope>provided</scope>
42+
</dependency>
43+
44+
<!-- Apache POI for comparison benchmarks -->
45+
<dependency>
46+
<groupId>org.apache.poi</groupId>
47+
<artifactId>poi</artifactId>
48+
</dependency>
49+
<dependency>
50+
<groupId>org.apache.poi</groupId>
51+
<artifactId>poi-ooxml</artifactId>
52+
</dependency>
53+
54+
<!-- Utilities -->
55+
<dependency>
56+
<groupId>commons-io</groupId>
57+
<artifactId>commons-io</artifactId>
58+
</dependency>
59+
<dependency>
60+
<groupId>org.slf4j</groupId>
61+
<artifactId>slf4j-api</artifactId>
62+
</dependency>
63+
<dependency>
64+
<groupId>ch.qos.logback</groupId>
65+
<artifactId>logback-classic</artifactId>
66+
</dependency>
67+
68+
<!-- JSON handling for reports -->
69+
<dependency>
70+
<groupId>com.alibaba.fastjson2</groupId>
71+
<artifactId>fastjson2</artifactId>
72+
</dependency>
73+
74+
<!-- Testing -->
75+
<dependency>
76+
<groupId>org.junit.jupiter</groupId>
77+
<artifactId>junit-jupiter</artifactId>
78+
<scope>test</scope>
79+
</dependency>
80+
</dependencies>
81+
82+
<build>
83+
<plugins>
84+
<plugin>
85+
<groupId>org.apache.maven.plugins</groupId>
86+
<artifactId>maven-compiler-plugin</artifactId>
87+
<configuration>
88+
<source>${maven.compiler.source}</source>
89+
<target>${maven.compiler.target}</target>
90+
<annotationProcessorPaths>
91+
<path>
92+
<groupId>org.openjdk.jmh</groupId>
93+
<artifactId>jmh-generator-annprocess</artifactId>
94+
<version>${jmh.version}</version>
95+
</path>
96+
</annotationProcessorPaths>
97+
</configuration>
98+
</plugin>
99+
100+
<plugin>
101+
<groupId>org.apache.maven.plugins</groupId>
102+
<artifactId>maven-shade-plugin</artifactId>
103+
<executions>
104+
<execution>
105+
<phase>package</phase>
106+
<goals>
107+
<goal>shade</goal>
108+
</goals>
109+
<configuration>
110+
<finalName>${uberjar.name}</finalName>
111+
<transformers>
112+
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
113+
<mainClass>org.openjdk.jmh.Main</mainClass>
114+
</transformer>
115+
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
116+
</transformers>
117+
<filters>
118+
<filter>
119+
<!--
120+
Shading signed JARs will fail without this.
121+
http://stackoverflow.com/questions/999489/invalid-signature-file-when-attempting-to-run-a-jar
122+
-->
123+
<artifact>*:*</artifact>
124+
<excludes>
125+
<exclude>META-INF/*.SF</exclude>
126+
<exclude>META-INF/*.DSA</exclude>
127+
<exclude>META-INF/*.RSA</exclude>
128+
</excludes>
129+
</filter>
130+
</filters>
131+
</configuration>
132+
</execution>
133+
</executions>
134+
</plugin>
135+
136+
<plugin>
137+
<groupId>org.codehaus.mojo</groupId>
138+
<artifactId>exec-maven-plugin</artifactId>
139+
<version>3.1.0</version>
140+
<executions>
141+
<execution>
142+
<id>run-benchmarks</id>
143+
<phase>integration-test</phase>
144+
<goals>
145+
<goal>exec</goal>
146+
</goals>
147+
<configuration>
148+
<classpathScope>test</classpathScope>
149+
<executable>java</executable>
150+
<commandlineArgs>-classpath %classpath org.openjdk.jmh.Main .*</commandlineArgs>
151+
</configuration>
152+
</execution>
153+
</executions>
154+
</plugin>
155+
</plugins>
156+
</build>
157+
158+
<profiles>
159+
<profile>
160+
<id>benchmark</id>
161+
<build>
162+
<plugins>
163+
<plugin>
164+
<groupId>org.codehaus.mojo</groupId>
165+
<artifactId>exec-maven-plugin</artifactId>
166+
<configuration>
167+
<classpathScope>test</classpathScope>
168+
<executable>java</executable>
169+
<commandlineArgs>-classpath %classpath org.openjdk.jmh.Main ${benchmark.pattern}</commandlineArgs>
170+
</configuration>
171+
</plugin>
172+
</plugins>
173+
</build>
174+
</profile>
175+
</profiles>
176+
</project>

0 commit comments

Comments
 (0)