Skip to content

Add request-scoped profiler benchmark flag and compressed trace export#109

Open
CrazyDave999 wants to merge 5 commits intosgl-project:mainfrom
CrazyDave999:main
Open

Add request-scoped profiler benchmark flag and compressed trace export#109
CrazyDave999 wants to merge 5 commits intosgl-project:mainfrom
CrazyDave999:main

Conversation

@CrazyDave999
Copy link

Summary

(Based on sgl-project/mini-sglang#14, with some follow-up fixes.)

Clients can send "profile": true and mini-sglang will start a torch.profiler session for that request, then export a Chrome-trace artifact when the request finishes.

What changed

  • API:
    Add benchmark-side support to pass profiling on demand (--profile), forwarding "profile": true in benchmark requests for easier validation.

  • Messaging: propagate profile through tokenizer/backend messages (TokenizeMsg, UserMsg).

  • Scheduler/core: carry profile into the internal Req object and start/stop profiling around the first in-flight request with profile=True.

  • New utility: python/minisgl/utils/profiler.py implements RequestProfiler (start/stop/export).

Test

# Start server
export MINISGL_PROFILE_DIR=/tmp
python -m minisgl --model "Qwen/Qwen3-0.6B" --tp 2 --attention-backend fi

# Then on client side:
curl -X POST http://127.0.0.1:1919/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt":"hello","max_tokens":64,"profile":true}'

# Or:
python benchmark/online/bench_qwen.py --profile

ls /tmp/minisgl*.json.gz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants