fix: avoid double counting cache hits

[cryo-zd]

What type of PR is this?

fix: avoid double counting cache hits

Currently, handleCaching calls metrics.RecordCacheHit() when a cache hit is found, while both InMemoryCache.FindSimilar and MilvusCache.FindSimilar already record cache hits internally. This causes a single cache hit to be counted twice.
This PR removes the duplicate call in handleCaching so that each hit is counted only once.
[InMemoryCache]

semantic-router/src/semantic-router/pkg/cache/inmemory_cache.go

Lines 294 to 304 in fac50b1

	observability.Debugf("InMemoryCache.FindSimilar: CACHE HIT - similarity=%.4f >= threshold=%.4f, response_size=%d bytes",
	results[0].Similarity, c.similarityThreshold, len(results[0].Entry.ResponseBody))
	observability.LogEvent("cache_hit", map[string]interface{}{
	"backend": "memory",
	"similarity": results[0].Similarity,
	"threshold": c.similarityThreshold,
	"model": model,
	})
	metrics.RecordCacheOperation("memory", "find_similar", "hit", time.Since(start).Seconds())
	metrics.RecordCacheHit()
	return results[0].Entry.ResponseBody, true, nil

[Milvus]

semantic-router/src/semantic-router/pkg/cache/milvus_cache.go

Lines 598 to 609 in fac50b1

	observability.Debugf("MilvusCache.FindSimilar: CACHE HIT - similarity=%.4f >= threshold=%.4f, response_size=%d bytes",
	bestSimilarity, c.similarityThreshold, len(bestResponse))
	observability.LogEvent("cache_hit", map[string]interface{}{
	"backend": "milvus",
	"similarity": bestSimilarity,
	"threshold": c.similarityThreshold,
	"model": model,
	"collection": c.collectionName,
	})
	metrics.RecordCacheOperation("milvus", "find_similar", "hit", time.Since(start).Seconds())
	metrics.RecordCacheHit()
	return []byte(bestResponse), true, nil

What this PR does / why we need it:
Accurate cache hit metrics are important for monitoring and analysis.

Which issue(s) this PR fixes:

Fixes #

Release Notes: Yes/No

[netlify]

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	23b07c5
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/68cd78813d3ac50008e360e0
😎 Deploy Preview	https://deploy-preview-177--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 src

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

• src/semantic-router/pkg/extproc/request_handler.go

---

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

[rootfs]

@cryo-zd would it ok if we keep the request handler code but remove the cache hit from inmemory and milvus? So when yet another storage is added, the cache hit is already counted.

[cryo-zd]

@cryo-zd would it ok if we keep the request handler code but remove the cache hit from inmemory and milvus? So when yet another storage is added, the cache hit is already counted.

Ack, keeping the cache hit metric in the request handler and letting each backend only record its own RecordCacheOperation makes the design cleaner and more extensible.

One small thought: right now only hits would be counted in the handler, while misses are still counted in each backend. To keep the logic symmetric and easier to maintain, would it make sense to also move RecordCacheMiss into the handler (e.g., in the else branch)? That way:

• handler is the single place for global hit/miss counters,
• backends focus only on RecordCacheOperation(...) for per-backend stats.

semantic-router/src/semantic-router/pkg/cache/inmemory_cache.go

Lines 276 to 278 in aa6c22f

	metrics.RecordCacheOperation("memory", "find_similar", "miss", time.Since(start).Seconds())
	metrics.RecordCacheMiss()
	return nil, false, nil

semantic-router/src/semantic-router/pkg/cache/inmemory_cache.go

Lines 302 to 304 in aa6c22f

	metrics.RecordCacheOperation("memory", "find_similar", "hit", time.Since(start).Seconds())
	metrics.RecordCacheHit()
	return results[0].Entry.ResponseBody, true, nil

I can update the PR in either direction — do you think this adjustment would be preferable?

[rootfs]

I see, that makes sense. I'll merge this one then.

[rootfs]

@cryo-zd do you like to come up with a doc PR about how to add yet another cache storage (e.g. chroma, redis)?

Also cc @aeft