Skip to content

chore: Migrate to ty type checker#1625

Merged
vdusek merged 6 commits intomasterfrom
migrate-to-ty
Jan 8, 2026
Merged

chore: Migrate to ty type checker#1625
vdusek merged 6 commits intomasterfrom
migrate-to-ty

Conversation

@vdusek
Copy link
Copy Markdown
Collaborator

@vdusek vdusek commented Dec 17, 2025

No description provided.

@vdusek vdusek added this to the 129th sprint - Tooling team milestone Dec 17, 2025
@vdusek vdusek requested review from Mantisus and janbuchar December 17, 2025 16:26
@vdusek vdusek self-assigned this Dec 17, 2025
@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Dec 17, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 17, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 86.61972% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.41%. Comparing base (6dbfcb5) to head (5c7abc3).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
src/crawlee/http_clients/_curl_impersonate.py 57.57% 14 Missing ⚠️
.../crawlee/storage_clients/_redis/_storage_client.py 66.66% 3 Missing ⚠️
.../crawlee/storage_clients/_redis/_dataset_client.py 66.66% 1 Missing ⚠️
...ee/storage_clients/_redis/_request_queue_client.py 66.66% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #1625      +/-   ##
==========================================
- Coverage   92.49%   92.41%   -0.08%     
==========================================
  Files         157      157              
  Lines       10440    10478      +38     
==========================================
+ Hits         9656     9683      +27     
- Misses        784      795      +11
Flag Coverage Δ
unit 92.41% <86.61%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Mantisus
Mantisus approved these changes Dec 17, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@Mantisus Mantisus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@vdusek vdusek changed the title chore: Migrate to ty type checker chore: Migrate to ty type checker Dec 19, 2025
@vdusek vdusek requested a review from Pijukatel January 5, 2026 08:03
janbuchar
janbuchar approved these changes Jan 5, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@janbuchar janbuchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of nits but probably OK. Will ty let us know when a type-ignore comment becomes obsolete?

Comment thread src/crawlee/browsers/_playwright_browser.py Outdated
Comment thread src/crawlee/crawlers/_abstract_http/_abstract_http_crawler.py
Comment thread src/crawlee/storage_clients/_base/_dataset_client.py
Comment thread src/crawlee/storage_clients/_file_system/_key_value_store_client.py Outdated
Comment thread .gitignore Outdated
Pijukatel
Pijukatel requested changes Jan 5, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@Pijukatel Pijukatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to solve some ignores. I added bunch of suggestions and raised some issues in ty repo

Comment thread pyproject.toml
Comment thread src/crawlee/_utils/recurring_task.py Outdated
Comment thread src/crawlee/_utils/recurring_task.py Outdated
Comment thread src/crawlee/_utils/sitemap.py Outdated
Comment thread src/crawlee/_utils/sitemap.py Outdated
Comment thread src/crawlee/storage_clients/_redis/_dataset_client.py
Comment thread src/crawlee/storage_clients/_redis/_dataset_client.py Outdated
Comment thread src/crawlee/storage_clients/models.py
Comment thread tests/unit/_utils/test_html_to_text.py
Comment thread tests/unit/crawlers/_adaptive_playwright/test_adaptive_playwright_crawler.py
@vdusek
Copy link
Copy Markdown
Collaborator Author

vdusek commented Jan 7, 2026

Will ty let us know when a type-ignore comment becomes obsolete?

Not now, see astral-sh/ty#278 and astral-sh/ty#1501 for more context.

@vdusek vdusek requested a review from Pijukatel January 7, 2026 14:14
@janbuchar
Copy link
Copy Markdown
Collaborator

janbuchar commented Jan 7, 2026

Will ty let us know when a type-ignore comment becomes obsolete?

Not now, see astral-sh/ty#278 and astral-sh/ty#1501 for more context.

Can we make an issue to revisit the ignores in ~2 months or something then?

@vdusek vdusek mentioned this pull request Jan 7, 2026
Closed
@vdusek
Copy link
Copy Markdown
Collaborator Author

vdusek commented Jan 7, 2026

Will ty let us know when a type-ignore comment becomes obsolete?

Not now, see astral-sh/ty#278 and astral-sh/ty#1501 for more context.

Can we make an issue to revisit the ignores in ~2 months or something then?

Sure, -> #1664.

Mantisus
Mantisus reviewed Jan 7, 2026
View reviewed changes
Comment thread src/crawlee/_utils/globs.py Outdated
Comment thread src/crawlee/crawlers/_abstract_http/_abstract_http_crawler.py Outdated
class _ParsedHttpCrawler(
AbstractHttpCrawler[ParsedHttpCrawlingContext[TParseResult], TParseResult, TSelectResult]
):
): # ty: ignore[invalid-generic-class]
Copy link
Copy Markdown
Collaborator

@Mantisus Mantisus Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

class _ParsedHttpCrawler(AbstractHttpCrawler):

Work here

Comment thread src/crawlee/sessions/_cookies.py Outdated
@vdusek vdusek merged commit 6606de4 into master Jan 8, 2026
30 checks passed
@vdusek vdusek deleted the migrate-to-ty branch January 8, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janbuchar janbuchar janbuchar approved these changes

@Pijukatel Pijukatel Pijukatel approved these changes

+1 more reviewer

@Mantisus Mantisus Mantisus approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@vdusek vdusek

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Milestone

129th sprint - Tooling team

Development

Successfully merging this pull request may close these issues.

5 participants

@vdusek @janbuchar @Mantisus @Pijukatel @apify-service-account