Skip to content

Commit e718c6e

Browse files
yuluo-yxyuluo-yx
authored
Merge branch 'main' into 0919-yuluo/add-version
2 parents 844517c + 5d90816 commit e718c6e

File tree

5 files changed

+116
-4
lines changed

5 files changed

+116
-4
lines changed

‎src/semantic-router/pkg/extproc/request_handler.go‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -263,8 +263,7 @@ func (r *OpenAIRouter) handleCaching(ctx *RequestContext) (*ext_proc.ProcessingR
263263
if err != nil {
264264
observability.Errorf("Error searching cache: %v", err)
265265
} else if found {
266-
// Record and log cache hit
267-
metrics.RecordCacheHit()
266+
// Log cache hit
268267
observability.LogEvent("cache_hit", map[string]interface{}{
269268
"request_id": ctx.RequestID,
270269
"model": requestModel,

‎tools/make/linter.mk‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22
# = Everything For Project Linter, markdown, yaml, code spell etc. =
33
# =============================== linter.mk ==========================
44

5-
docs-lint:
5+
docs-lint: docs-install
66
@$(LOG_TARGET)
77
cd website && npm run lint
88

9-
docs-lint-fix:
9+
docs-lint-fix: docs-install
1010
@$(LOG_TARGET)
1111
cd website && npm run lint:fix
1212

‎website/docs/api/router.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,12 @@ semantic_router_cache_size 1247
173173
# Security metrics
174174
semantic_router_pii_detections_total{action="block"} 23
175175
semantic_router_jailbreak_attempts_total{action="block"} 5
176+
177+
# Error metrics
178+
llm_request_errors_total{model="gpt-4",reason="timeout"} 12
179+
llm_request_errors_total{model="claude-3",reason="upstream_5xx"} 3
180+
llm_request_errors_total{model="phi4",reason="upstream_4xx"} 5
181+
llm_request_errors_total{model="phi4",reason="pii_policy_denied"} 8
176182
```
177183

178184
### Reasoning Mode Metrics
@@ -247,6 +253,35 @@ sum by (model) (increase(llm_model_cost_total{currency="USD"}[1h]))
247253
sum by (reason_code) (increase(llm_routing_reason_codes_total[15m]))
248254
```
249255

256+
### Request Error Metrics
257+
258+
The router tracks request-level failures by model and reason so you can monitor both absolute error throughput and the share of requests that fail.
259+
260+
- `llm_request_errors_total{model, reason}`
261+
- Description: Total number of request errors categorized by failure reason
262+
- Labels:
263+
- model: target model name for the failed request
264+
- reason: error category (timeout, upstream_4xx, upstream_5xx, pii_policy_denied, jailbreak_block, parse_error, serialization_error, cancellation, classification_failed, unknown)
265+
266+
Example PromQL queries:
267+
268+
```prometheus
269+
# Total errors by reason over the last hour
270+
sum by (reason) (increase(llm_request_errors_total[1h]))
271+
272+
# Error throughput (errors/sec) by model over the last 15 minutes.
273+
# Helpful for incident response because it shows how many failing requests are impacting users.
274+
sum by (model) (rate(llm_request_errors_total[15m]))
275+
276+
# Error ratio (% of requests failing) by model over the last 15 minutes.
277+
# Use increase() to align numerator and denominator with the same lookback window.
278+
100 * sum by (model) (increase(llm_request_errors_total[15m])) /
279+
sum by (model) (increase(llm_model_requests_total[15m]))
280+
281+
# PII policy blocks over the last 24 hours
282+
sum(increase(llm_request_errors_total{reason="pii_policy_denied"}[24h]))
283+
```
284+
250285
### Pricing Configuration
251286

252287
Provide per-1M pricing for your models so the router can compute request cost and emit metrics/logs.

‎website/src/pages/community/community-page.module.css‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,16 @@
130130
margin-top: 0.25rem;
131131
}
132132

133+
.stepNumberTips {
134+
display: flex;
135+
flex-direction: column;
136+
margin: 0 0 0 1rem;
137+
}
138+
139+
.stepNumberTips p {
140+
line-height: 1.5;
141+
}
142+
133143
.step h4 {
134144
margin: 0 0 0.5rem 0;
135145
color: var(--ifm-color-primary);

‎website/src/pages/community/contributing.js‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,16 @@ export default function Contributing() {
103103
<div>
104104
<h4>Test</h4>
105105
<p>Run tests and ensure your changes don't break existing functionality.</p>
106+
<div className={styles.stepNumberTips}>
107+
<p>1. Run precommit hooks, ensure compliance with the project submission guidelines;</p>
108+
<p>
109+
2. You can refer to
110+
{' '}
111+
<a href="/docs/getting-started/installation">Install the local</a>
112+
{' '}
113+
to start semantic-router locally.
114+
</p>
115+
</div>
106116
</div>
107117
</div>
108118

@@ -117,6 +127,64 @@ export default function Contributing() {
117127
</div>
118128
</section>
119129

130+
<section className={styles.section}>
131+
<h2>⚙️ Precommit hooks</h2>
132+
<p>The Semantic-router project provides a precommit hook to standardize the entire project, including Go, Python, Rust, Markdown, and spelling error checking.</p>
133+
<p>Although these measures may increase the difficulty of contributions, they are necessary. We are currently building a portable Docker precommit environment to reduce the difficulty of contributions, allowing you to focus on functional pull requests.</p>
134+
135+
<div className={styles.card}>
136+
<h3>Manual</h3>
137+
138+
<h4>Some Tips: </h4>
139+
<div className={styles.stepNumberTips}>
140+
<p>1. If the precommit check fails, don't worry. You can also get more information by executing "make help". </p>
141+
<p>2. For the pip installation tool, we recommend that you use venv for installation.</p>
142+
<p>3. You can also directly submit the PR and let GitHub CI test it for you, but this will take a lot of time!</p>
143+
</div>
144+
145+
<div className={styles.steps}>
146+
<div className={styles.step}>
147+
<span className={styles.stepNumber}>1</span>
148+
<div>
149+
<h4>Install precommit</h4>
150+
<p>Run pip install --user precommit</p>
151+
</div>
152+
</div>
153+
<div className={styles.step}>
154+
<span className={styles.stepNumber}>2</span>
155+
<div>
156+
<h4>Install check tools</h4>
157+
<div className={styles.stepNumberTips}>
158+
<p>Markdown: npm install -g markdownlint-cli</p>
159+
<p>Yaml: pip install --user yamllint</p>
160+
<p>CodeSpell: pip install --user codespell</p>
161+
<p>JavaScript: cd website && npm lint</p>
162+
</div>
163+
</div>
164+
</div>
165+
<div className={styles.step}>
166+
<span className={styles.stepNumber}>3</span>
167+
<div>
168+
<h4>Install precommit to git</h4>
169+
<p>Run pre-commit install, then pre-commit installed at .git/hooks/pre-commit</p>
170+
</div>
171+
</div>
172+
<div className={styles.step}>
173+
<span className={styles.stepNumber}>4</span>
174+
<div>
175+
<h4>Run</h4>
176+
<p>Run make precommit-check to check.</p>
177+
</div>
178+
</div>
179+
180+
<hr />
181+
182+
<h3>Docker</h3>
183+
<p>Coming soon!</p>
184+
</div>
185+
</div>
186+
</section>
187+
120188
<section className={styles.section}>
121189
<h2>🏷️ Working Group Areas</h2>
122190
<p>

0 commit comments

Comments
 (0)