docs(ragas): update metrics documentation and clarify answer correctness#89
Conversation
- Enhanced the description of the `ragas_rag_default` benchmark to specify the inclusion of the answer correctness metric. - Updated the overview of RAGAS metrics to include detailed information about the five core metrics, emphasizing the role of `factual_correctness`. - Clarified the distinction between answer correctness and answer accuracy in the metrics documentation. - Added a note on how to include `factual_correctness` in benchmark configurations.
|
Warning Review limit reached
Next review available in: 50 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR updates RAGAS adapter documentation to clarify that ChangesRAGAS Documentation Clarification
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/content/docs/adapters/ragas/metrics.md (1)
122-141: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winMove
factual_correctnessout of the Extended Metrics section.This subsection now says
factual_correctnessis a core metric, but it still lives under## Extended Metrics, which makes the page internally inconsistent. Please relocate it into the core-metrics section or rename the section to match the new classification.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/content/docs/adapters/ragas/metrics.md` around lines 122 - 141, The metrics documentation is inconsistent because factual_correctness is described as a core RAGAS metric while still being listed under Extended Metrics. Move the factual_correctness subsection into the core metrics area, or rename the surrounding section so the heading matches the classification used in that subsection; update the nearby text in the metrics docs to keep the section organization aligned.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/content/docs/adapters/ragas/index.mdx`:
- Around line 13-15: The metric list in the RAGAS overview mixes
retrieved-context grounding metrics with the reference-based
`factual_correctness` metric, which is misleading. Update the descriptive
sentence in the RAGAS docs content to keep the grounding metrics together by
naming faithfulness, answer relevancy, context precision, and context recall as
the direct grounding measures, and mention answer correctness
(`factual_correctness`) separately. Use the RAGAS adapter intro text as the
location reference.
---
Outside diff comments:
In `@src/content/docs/adapters/ragas/metrics.md`:
- Around line 122-141: The metrics documentation is inconsistent because
factual_correctness is described as a core RAGAS metric while still being listed
under Extended Metrics. Move the factual_correctness subsection into the core
metrics area, or rename the surrounding section so the heading matches the
classification used in that subsection; update the nearby text in the metrics
docs to keep the section organization aligned.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1b1f0df9-8936-4098-87e4-827432c9638f
📒 Files selected for processing (4)
.gitignoresrc/content/docs/adapters/ragas/configuration.mdsrc/content/docs/adapters/ragas/index.mdxsrc/content/docs/adapters/ragas/metrics.md
- Updated the RAGAS overview to specify that grounding metrics assess whether a RAG system correctly grounds its answers in retrieved documents. - Enhanced the description of the metrics, including faithfulness, answer relevancy, context precision, context recall, and answer correctness (`factual_correctness`), to improve clarity and understanding.
2 participants
What and why
Clarifies the RAGAS metrics documentation in the EvalHub docs site. The previous docs were ambiguous about:
factual_correctness(answer correctness) is absent fromragas_rag_defaultfactual_correctnessandnv_accuracy(Answer Accuracy)This adds a dedicated "Five Core RAGAS Metrics" section with a mapping table, a callout distinguishing answer correctness from answer accuracy, and an example showing how to include
factual_correctnessexplicitly. The section heading and category forfactual_correctnessin the metrics table are also corrected.Type
Testing
Breaking changes
None.
Summary by CodeRabbit
Documentation
factual_correctness, and how to include it in results.Chores