Skip to content

Conversation

@sanjeed5
Copy link
Contributor

@sanjeed5 sanjeed5 commented Nov 6, 2025

Summary

This PR updates the documentation for metrics to showcase the new ragas.metrics.collections API as the primary recommended approach, while preserving legacy API documentation for backward compatibility.

Changes

Metrics

  • Faithfulness metric documentation updated with collections API examples
  • AnswerAccuracy metric documentation updated with collections API examples (tested and verified)
  • AnswerCorrectness
  • AnswerRelevancy
  • AnswerSimilarity
  • BleuScore
  • ContextEntityRecall
  • ContextPrecision (including WithReference, WithoutReference, ContextUtilization)
  • ContextRelevance
  • NoiseSensitivity
  • RougeScore
  • SemanticSimilarity
  • String metrics (ExactMatch, StringPresence, NonLLMStringSimilarity, DistanceMeasure)
  • SummaryScore

Documentation Pattern

Each metric documentation follows this structure:

  1. Primary Example: Collections-based API (modern, recommended)
    • Uses from ragas.metrics.collections import MetricName
    • Shows llm_factory setup with AsyncOpenAI
    • Direct parameter passing to ascore()
    • Includes note about sync .score() method
  2. How It's Calculated: Conceptual explanation (implementation-agnostic)
  3. Legacy Section: Original API for backward compatibility
    • Uses SingleTurnSample and .single_turn_ascore()
    • Clearly labeled as legacy pattern

Test Plan

  • Build docs locally to verify formatting
  • Test code examples to ensure they work
  • Verify all metrics from collections are documented
  • Check cross-references and links

- Add new primary example using ragas.metrics.collections
- Show modern API with llm_factory and direct parameter passing
- Move legacy SingleTurnSample examples to Legacy section
- Relocate HHEM section to Legacy (no collections version yet)
- Add note about sync .score() method availability
- Legacy API will be deprecated in v0.4
- Complete removal planned for v1.0
- Added warning box to guide users to migrate
- Add new primary example using ragas.metrics.collections
- Show modern API with llm_factory and direct parameter passing
- Move legacy SingleTurnSample examples to Legacy section
- Add deprecation timeline (v0.4 deprecated, v1.0 removed)
- New example tested and verified to work correctly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants