Skip to content

fix(tools): make RAG file extension detection case-insensitive - #6755

Open
Kyou12138 wants to merge 1 commit into
crewAIInc:mainfrom
Kyou12138:fix/rag-case-insensitive-extensions-6399
Open

fix(tools): make RAG file extension detection case-insensitive#6755
Kyou12138 wants to merge 1 commit into
crewAIInc:mainfrom
Kyou12138:fix/rag-case-insensitive-extensions-6399

Conversation

@Kyou12138

Copy link
Copy Markdown

Summary

Fixes #6399.

DataTypes.from_content() matched file extensions with case-sensitive path.endswith(ext) against a lowercase-only map. Paths/URLs such as Report.PDF or data.CSV were misclassified as plain text (or website for URLs), so binary documents were chunked/embedded as raw bytes instead of parsed text.

This is common on Windows, scanner exports, and many enterprise document pipelines.

Change

Lowercase the path (or URL path segment used for extension matching) before comparing against the extension map so detection is case-insensitive for all registered types (PDF, DOCX, CSV, …).

Tests

Added test_data_types_from_content.py covering:

  • lowercase extensions (regression)
  • uppercase / mixed-case extensions
  • URL path with uppercase extension

How to test

cd lib/crewai-tools   # or project path that owns crewai_tools
pytest tests/tools/rag/test_data_types_from_content.py -q

Notes

Related open PRs may address the same issue; this change is intentionally minimal (normalize once at match time) with focused unit coverage.

DataTypes.from_content() compared path.endswith(ext) against a
lowercase-only map, so .PDF/.CSV/.DOCX were misclassified as text
(or website for URLs). Lowercase the path before matching.

Closes crewAIInc#6399
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: daa87960-b40a-4c3a-9676-c0055c89d395

📥 Commits

Reviewing files that changed from the base of the PR and between ebe0082 and 75d177f.

📒 Files selected for processing (2)
  • lib/crewai-tools/src/crewai_tools/rag/data_types.py
  • lib/crewai-tools/tests/tools/rag/test_data_types_from_content.py

📝 Walkthrough

Walkthrough

Changes

RAG file-type detection

Layer / File(s) Summary
Case-insensitive extension matching
lib/crewai-tools/src/crewai_tools/rag/data_types.py, lib/crewai-tools/tests/tools/rag/test_data_types_from_content.py
get_file_type lowercases paths before extension checks. Tests cover uppercase and mixed-case local paths, URLs, and Path objects. Extensionless URLs remain classified as WEBSITE.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the case-insensitive RAG file extension detection fix.
Description check ✅ Passed The description explains the detection bug, the normalization change, and the focused regression tests.
Linked Issues check ✅ Passed The implementation and tests satisfy issue #6399 by supporting case-insensitive detection for local paths and URLs.
Out of Scope Changes check ✅ Passed The changes remain limited to extension matching and focused regression tests for the linked issue.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] RAG file-type auto-detection is case-sensitive — uppercase extensions (.PDF, .CSV, .DOCX) misrouted to the text loader

1 participant