Skip to content

perf(dbf): reduce per-value decoding overhead - #337

Merged
luabida merged 1 commit into
AlertaDengue:mainfrom
devdudumuniz:performance/dbf-decoding
Aug 31, 2026
Merged

perf(dbf): reduce per-value decoding overhead#337
luabida merged 1 commit into
AlertaDengue:mainfrom
devdudumuniz:performance/dbf-decoding

Conversation

@devdudumuniz

Copy link
Copy Markdown
Contributor

Summary

  • decode NumPy byte-string columns with a list comprehension
  • centralize the optimized path in _decode_column
  • use the same implementation for full-file and streaming reads

Performance

A local microbenchmark on representative fixed-width NumPy byte strings showed approximately 1.51x faster decoding at both 1,000 and 100,000 values.

Validation

  • pytest -q pysus/tests/data/test_dbf_reader.py pysus/tests/data/test_dbf_reader_parity.py (40 passed)
  • black --check pysus/data/dbf_reader.py
  • flake8 pysus/data/dbf_reader.py

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@2102ba2). Learn more about missing BASE report.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #337   +/-   ##
=======================================
  Coverage        ?   95.23%           
=======================================
  Files           ?      169           
  Lines           ?    20016           
  Branches        ?        0           
=======================================
  Hits            ?    19063           
  Misses          ?      953           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@luabida
luabida merged commit b1be573 into AlertaDengue:main Aug 31, 2026
15 checks passed
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 2.10.7 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants