Skip to content

Configure k6 quote CNC test for 500 RPS #534

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shayancoin merged 1 commit into from
Oct 21, 2025
Merged

Configure k6 quote CNC test for 500 RPS #534

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions tests/perf/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Performance testing

## Quote + CNC 500 RPS load test

Use the `tests/perf/k6-quote-cnc.js` scenario to generate approximately 500 requests per second (RPS) against the quote generation and CNC export APIs.

### Required tooling

- [k6](https://k6.io/docs/getting-started/installation/) CLI v0.45.0 or later.

### Environment variables

Set the following environment variables to target the appropriate environment and authenticate requests:

- `API_BASE_URL` **(recommended)** – Fully qualified base URL for the API (for example, `https://staging.api.example.com`).
- Alternatively, set `API_PROTOCOL` and `API_HOST` if you prefer to build the URL from discrete parts. Defaults to `http://localhost:8000`.
- `FRONTEND_BASE_URL` – Base URL for static frontend assets (optional, enables cache validation checks).
- You may also provide `FRONTEND_PROTOCOL` and `FRONTEND_HOST` for the same purpose.
- `API_WRITE_TOKEN` – Bearer token used to create materials, modules, and quotes. Defaults to `test-write-token` for local environments.
- `RPS` – Target request arrival rate (defaults to `500`).
- `DURATION` – How long to sustain the load (defaults to `5m`).
- `PRE_ALLOCATED_VUS` / `MAX_VUS` – k6 virtual user pool sizing for the constant arrival rate executor (defaults to `150`/`600`).

### Recommended command

Run the following command to execute the load test locally or in CI. Adjust the base URLs and tokens to match the target environment.

```bash
k6 run \
--summary-trend-stats "avg,p(95),p(99),min,max" \
-e API_BASE_URL="https://staging.api.example.com" \
-e FRONTEND_BASE_URL="https://staging.fe.example.com" \
-e API_WRITE_TOKEN="<token>" \
-e RPS=500 \
-e DURATION=5m \
-e PRE_ALLOCATED_VUS=200 \
-e MAX_VUS=600 \
tests/perf/k6-quote-cnc.js
```

The script automatically provisions a material and module during the `setup` stage and reuses their identifiers during the load test. The scenario enforces the `p(95) < 300 ms` latency SLO and fails if more than 1% of the requests result in errors.
43 changes: 33 additions & 10 deletions tests/perf/k6-quote-cnc.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,38 @@
import http from 'k6/http';
import { check, sleep } from 'k6';

const DEFAULT_RATE = Number(__ENV.RPS || '500');
const DEFAULT_DURATION = __ENV.DURATION || '5m';
const DEFAULT_PRE_ALLOCATED_VUS = Number(__ENV.PRE_ALLOCATED_VUS || '150');
const DEFAULT_MAX_VUS = Number(__ENV.MAX_VUS || '600');

export const options = {
vus: 5,
duration: '30s',
scenarios: {
quoteCnc: {
executor: 'constant-arrival-rate',
rate: DEFAULT_RATE,
timeUnit: '1s',
duration: DEFAULT_DURATION,
preAllocatedVUs: DEFAULT_PRE_ALLOCATED_VUS,
maxVUs: DEFAULT_MAX_VUS,
exec: 'quoteCncScenario',
},
},
thresholds: {
http_req_failed: ['rate<0.01'],
http_req_duration: ['p(95)<150'],
http_req_duration: ['p(95)<300'],
},
};

const baseUrl = __ENV.BASE_URL || 'http://localhost:8000';
const feBase = __ENV.FRONTEND_BASE_URL || '';
const apiBaseUrl =
__ENV.API_BASE_URL ||
`${__ENV.API_PROTOCOL || 'http'}://${__ENV.API_HOST || 'localhost:8000'}`;

const feBase =
__ENV.FRONTEND_BASE_URL ||
(__ENV.FRONTEND_HOST
? `${__ENV.FRONTEND_PROTOCOL || 'http'}://${__ENV.FRONTEND_HOST}`
: '');

function createMaterialAndModule() {
const suffix = `${Date.now()}-${Math.floor(Math.random() * 1e6)}`;
Expand All @@ -20,7 +41,7 @@ function createMaterialAndModule() {
Authorization: 'Bearer ' + (__ENV.API_WRITE_TOKEN || 'test-write-token'),
};
const materialRes = http.post(
`${baseUrl}/api/materials/`,
`${apiBaseUrl}/api/materials/`,
JSON.stringify({
name: `Walnut-${suffix}`,
texture_url: null,
Expand All @@ -32,7 +53,7 @@ function createMaterialAndModule() {
const materialId = materialRes.json('id');

const moduleRes = http.post(
`${baseUrl}/api/modules/`,
`${apiBaseUrl}/api/modules/`,
JSON.stringify({
name: `Base600-${suffix}`,
width: 600.0,
Expand Down Expand Up @@ -96,20 +117,20 @@ function buildPayload(moduleId) {
};
}

export default function (data) {
export function quoteCncScenario(data) {
const headers = {
'Content-Type': 'application/json',
Authorization: 'Bearer ' + (__ENV.API_WRITE_TOKEN || 'test-write-token'),
};
const moduleId = data?.moduleId || createMaterialAndModule().moduleId;
Copy link
Contributor

@coderabbitai coderabbitai bot Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fallback resource creation could overwhelm the system.

The fallback createMaterialAndModule() on line 125 would execute for every iteration if setup() fails or returns invalid data. At 500 RPS over 5 minutes, this could create 150,000 materials and modules, overwhelming the database and invalidating test results.

Consider removing the fallback or making setup failures explicit:

- const moduleId = data?.moduleId || createMaterialAndModule().moduleId;
+ if (!data?.moduleId) {
+   throw new Error('Setup failed - moduleId not available');
+ }
+ const moduleId = data.moduleId;

This ensures the test fails fast with a clear error message rather than attempting resource creation in the hot path.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const moduleId = data?.moduleId || createMaterialAndModule().moduleId;
if (!data?.moduleId) {
throw new Error('Setup failed - moduleId not available');
}
const moduleId = data.moduleId;
🤖 Prompt for AI Agents 
In tests/perf/k6-quote-cnc.js around line 125, the inline fallback
createMaterialAndModule() is invoked in the hot path (const moduleId =
data?.moduleId || createMaterialAndModule().moduleId), which would create
resources on every iteration if setup() failed; remove the fallback call and
instead make setup failures explicit by throwing or returning a clear error when
data is missing, so the test fails fast; update the code to validate that
data.moduleId (and other required setup values) exist before the load loop and
abort with a descriptive error/log if they do not, rather than creating
resources during iterations.

const payloads = buildPayload(moduleId);

const quoteResponse = http.post(`${baseUrl}/api/quote/generate`, payloads.quote, {
const quoteResponse = http.post(`${apiBaseUrl}/api/quote/generate`, payloads.quote, {
headers,
});
check(quoteResponse, { 'quote 200': (r) => r.status === 200 });

const cncResponse = http.post(`${baseUrl}/api/v1/exports/cnc`, payloads.cnc, {
const cncResponse = http.post(`${apiBaseUrl}/api/v1/exports/cnc`, payloads.cnc, {
headers,
});
check(cncResponse, {
Expand Down Expand Up @@ -164,3 +185,5 @@ export default function (data) {

sleep(0.5);
}

export default quoteCncScenario;
Loading