Skip to content

custom schema #622

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

custom schema #622

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2c88d42
custom schema
Jul 30, 2025
37e170b
2nd commit
Aug 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
162 changes: 162 additions & 0 deletions BUILD_COMMANDS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# BenchBase JAR Build Commands

This document contains the commands to generate benchbase.jar for both Java 17 and Java 23.

## Prerequisites

- Java 17 or Java 23 installed
- Maven (or use the included Maven wrapper `./mvnw`)

## Available Database Profiles

- `cockroachdb`
- `mariadb`
- `mysql`
- `oracle`
- `phoenix`
- `postgres`
- `spanner`
- `sqlite`
- `sqlserver`

## Java 17 Build Commands

### Basic Build (with tests)
```bash
# For postgres profile (most common)
./mvnw clean package -P postgres -Djava.version=17 -Dmaven.compiler.source=17 -Dmaven.compiler.target=17

# For other profiles, replace 'postgres' with desired profile
./mvnw clean package -P mysql -Djava.version=17 -Dmaven.compiler.source=17 -Dmaven.compiler.target=17
```

### Optimized Build (without tests, faster)
```bash
# For postgres profile
./mvnw clean package verify -P postgres -DskipTests -Djava.version=17 -Dmaven.compiler.source=17 -Dmaven.compiler.target=17 -Ddescriptors=src/main/assembly/tgz.xml

# For other profiles
./mvnw clean package verify -P mysql -DskipTests -Djava.version=17 -Dmaven.compiler.source=17 -Dmaven.compiler.target=17 -Ddescriptors=src/main/assembly/tgz.xml
```

## Java 23 Build Commands

### Basic Build (with tests)
```bash
# For postgres profile (most common)
./mvnw clean package -P postgres

# For other profiles
./mvnw clean package -P mysql
```

### Optimized Build (without tests, faster)
```bash
# For postgres profile
./mvnw clean package verify -P postgres -DskipTests -Ddescriptors=src/main/assembly/tgz.xml

# For other profiles
./mvnw clean package verify -P mysql -DskipTests -Ddescriptors=src/main/assembly/tgz.xml
```

## Multi-Profile Builds

### Build all profiles for Java 17
```bash
for profile in cockroachdb mariadb mysql oracle phoenix postgres spanner sqlite sqlserver; do
./mvnw clean package -P $profile -Djava.version=17 -Dmaven.compiler.source=17 -Dmaven.compiler.target=17
done
```

### Build all profiles for Java 23
```bash
for profile in cockroachdb mariadb mysql oracle phoenix postgres spanner sqlite sqlserver; do
./mvnw clean package -P $profile
done
```

## Output Location

After building, the artifacts will be in the `target/` directory:

1. **Archive**: `target/benchbase-<profile>.tgz`
2. **Extracted location**: `target/benchbase-<profile>/`
3. **JAR file**: `target/benchbase-<profile>/benchbase.jar`

## Extraction and Usage

```bash
# Navigate to target directory
cd target

# Extract the built archive
tar xvzf benchbase-postgres.tgz

# Navigate to extracted directory
cd benchbase-postgres

# Run BenchBase (show help)
java -jar benchbase.jar -h

# Example: Run TPCC benchmark
java -jar benchbase.jar -b tpcc -c config/postgres/sample_tpcc_config.xml --create=true --load=true --execute=true
```

## Alternative: Using the Build Script

You can also use the provided build script:

```bash
# Make executable (if not already)
chmod +x build-commands.sh

# Show help and all commands
./build-commands.sh

# Build Java 17 version with postgres profile
./build-commands.sh java17 postgres

# Build Java 23 version with mysql profile
./build-commands.sh java23 mysql

# Build optimized version
./build-commands.sh optimized postgres 17 # Java 17
./build-commands.sh optimized postgres 23 # Java 23
```

## Custom Plugin Configuration

After building, you can now specify a custom plugin.xml file using the `--plugin-config` parameter:

```bash
# Using custom plugin configuration
java -jar benchbase.jar --plugin-config /path/to/custom-plugin.xml -b mybench -c config.xml --execute=true

# Using relative path for custom plugins
java -jar benchbase.jar --plugin-config custom-plugins.xml -b tpcc -c config/postgres/sample_tpcc_config.xml --execute=true

# Show help with custom plugin config
java -jar benchbase.jar --plugin-config /path/to/custom-plugin.xml -h
```

## Custom Plugin.xml Format

Your custom plugin.xml should follow this structure:
```xml
<?xml version="1.0"?>
<plugins>
<plugin name="mybench">com.mycompany.MyCustomBenchmark</plugin>
<plugin name="tpcc">com.oltpbenchmark.benchmarks.tpcc.TPCCBenchmark</plugin>
<plugin name="tpch">com.oltpbenchmark.benchmarks.tpch.TPCHBenchmark</plugin>
<!-- Add your custom benchmarks here -->
</plugins>
```

## Notes

- The project is currently configured for Java 23 by default in `pom.xml`
- For Java 17 builds, we override the compiler settings via Maven properties
- The `-DskipTests` flag speeds up builds by skipping test execution
- Different database profiles include specific JDBC drivers for each database
- The `postgres` profile is the most commonly used and recommended for general testing
- If no `--plugin-config` is specified, it defaults to `config/plugin.xml`
136 changes: 136 additions & 0 deletions BenchBase_Requirements.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# BenchBase Database Benchmarking Requirements

## Executive Summary

This document outlines the technical requirements and rationale for adopting BenchBase as our database benchmarking solution. Our evaluation focuses on three critical performance criteria: high-throughput transaction processing, complex multi-table transaction support, and dynamic workload scaling capabilities.

## Business Requirements

### 1. High-Performance Transaction Processing
**Requirement**: Support for high-throughput workloads up to **10,000 QPS (Queries Per Second)**

**Rationale**:
- Modern applications demand sub-second response times under heavy load
- Peak traffic scenarios require sustained high-throughput performance
- Database capacity planning requires accurate high-load testing
- Performance bottlenecks must be identified before production deployment

**Success Criteria**:
- Sustained 10K+ QPS without performance degradation
- Consistent latency under maximum load
- Resource utilization monitoring during peak throughput
- Scalable to future performance requirements

### 2. Complex Multi-Table Transaction Support
**Requirement**: Comprehensive testing of **multi-table transactional workloads**

**Rationale**:
- Real-world applications involve complex business logic spanning multiple tables
- ACID compliance verification across related data operations
- Foreign key constraint validation under concurrent access
- Cross-table consistency and integrity testing

**Success Criteria**:
- Support for TPC-C style multi-table transactions (5+ tables)
- Configurable transaction complexity and data relationships
- Deadlock detection and resolution testing
- Referential integrity validation under load

### 3. Dynamic Workload Scaling
**Requirement**: **Dynamic TPS adjustment** and adaptive load generation

**Rationale**:
- Production workloads exhibit variable traffic patterns
- Testing must simulate real-world load fluctuations
- Performance characteristics change under different load profiles
- Capacity planning requires understanding of dynamic scaling behavior

**Success Criteria**:
- Real-time TPS adjustment during test execution
- Configurable load patterns (spike, ramp-up, steady-state)
- Multiple workload profiles within single test run
- Performance monitoring across varying load conditions

## Technical Requirements

### Core Capabilities
- **Multi-Database Support**: PostgreSQL, MySQL, Oracle, SQL Server compatibility
- **Configurable Workloads**: Standard benchmarks (TPC-C, TPC-H) with customization options
- **Concurrent Execution**: Multiple terminal/worker support for parallel load generation
- **Isolation Levels**: Support for various transaction isolation levels
- **Monitoring & Metrics**: Comprehensive performance data collection and reporting

### Performance Specifications
| Metric | Requirement | Measurement |
|--------|-------------|-------------|
| Peak TPS | 10,000+ QPS | Sustained for 5+ minutes |
| Latency | <50ms P99 | At maximum throughput |
| Concurrency | 100+ terminals | Simultaneous connections |
| Scalability | Linear scaling | Up to available hardware limits |

### Workload Characteristics
- **Transaction Types**: OLTP, mixed read/write, analytical queries
- **Data Distribution**: Configurable skew and hotspot patterns
- **Table Relationships**: Complex foreign key dependencies
- **Custom Schemas**: Support for application-specific table structures

## Why BenchBase Meets Our Requirements

### ✅ High TPS Capability
- **Proven Performance**: Successfully tested at 10K+ QPS in our environment
- **Optimized Connection Handling**: Efficient connection pooling and batch operations
- **Database-Specific Tuning**: Leverages database-specific optimizations (e.g., `reWriteBatchedInserts` for PostgreSQL)

### ✅ Multi-Table Transaction Excellence
- **TPC-C Benchmark**: Industry-standard multi-table workload (9 tables, complex relationships)
- **Real-World Scenarios**: Order processing, inventory management, customer transactions
- **ACID Compliance**: Comprehensive testing of transaction isolation and consistency

### ✅ Dynamic Load Generation
- **Configurable Rate Limiting**: Real-time TPS adjustment (unlimited to specific targets)
- **Multiple Terminal Support**: Scalable concurrency with 5-100+ terminals
- **Workload Profiles**: Customizable transaction mix and execution patterns

## Implementation Benefits

### Operational Advantages
- **Standardized Benchmarking**: Industry-standard TPC benchmarks for comparability
- **Comprehensive Reporting**: Detailed metrics, histograms, and performance analysis
- **Database Agnostic**: Consistent testing across multiple database platforms
- **Open Source**: No licensing costs, community support, extensible codebase

### Technical Advantages
- **Flexible Configuration**: XML-based configuration for easy workload modification
- **Schema Customization**: Support for custom table structures and data types
- **Performance Monitoring**: Built-in metrics collection and analysis tools
- **CI/CD Integration**: Command-line interface suitable for automated testing

## Risk Mitigation

### Performance Validation
- **Before Production**: Identify performance bottlenecks in controlled environment
- **Capacity Planning**: Accurate resource requirement estimation
- **Regression Testing**: Detect performance degradation across database versions

### Operational Readiness
- **Load Testing**: Validate system behavior under peak traffic conditions
- **Disaster Recovery**: Test database recovery performance under load
- **Scalability Planning**: Understand scaling characteristics for future growth

## Conclusion

BenchBase provides a comprehensive solution for our database performance testing requirements, offering:

1. **Proven high-throughput capabilities** up to 10K+ QPS
2. **Industry-standard multi-table transaction workloads** for realistic testing
3. **Dynamic load generation** with configurable TPS and workload patterns

The platform's flexibility, comprehensive reporting, and open-source nature make it an ideal choice for our performance validation and capacity planning needs.

---

**Recommended Next Steps**:
1. Deploy BenchBase in staging environment
2. Configure workloads matching production patterns
3. Establish baseline performance metrics
4. Integrate into CI/CD pipeline for continuous performance validation
Loading