Skip to content

Conversation

@rburghol
Copy link
Contributor

@rburghol rburghol commented Jan 31, 2026

Ready to merge @PaulDudaRESPEC -- This PR restricts non-dev installs to be < pandas 3.0.0, but adds code that is preliminarily valid for pandas 3.0.0. That is, code the will execute without error under pandas 3.0.0 and and this same code produces identical numerical results to hspf in pandas < 3.0.0. The test coverage that fails when run with pandas-3.0.0 will be covered in #209 (Note: the test error IS related to pandas update, and is what I believe to be both an error in the test code, AND a legit simulation difference that needs to be scoped out -- elaborated on in the issue).
This PR:

  • Adds a close() method to the custom local HDF5 class (yes it is fine to have but no really needed per @austinorr
    • Implements the with statement in run() to insure clean handling in the event of an error in runtime
  • Integrates the changes to utilities.py and main.py from Pandas 3.0, create new HDF5 close() method #216
  • Enhances the changes from a brief comment to trigger git #207 to do all timeseries transformations correctly.
  • Fixes the value comparison problems in python < 3.11
  • Adds a script examples/pretest/cmd_regression.py that creates a brief command prompt friendly tester, as well as leveraging the intermediate products of the RegressTest class for simulation debugging.
  • Updates the class RegressTest in the file tests/convert/regression_base.py to have a silent mode to be a bit more screen friendly (@austinorr this is False by default so no change in previous behavior is made).
  • The model has run successfully under pandas 3.0.0, and I have manually verified the model flow values are identical with hspf. However, this should just be considered a first step in pandas 3.0.0 update, as for the sediment there is an offset of a few days that is troubling see Verify HSP* and Update RegressTest to run under Pandas 3.0.0 #209

THIS IS NOT YET Pandas 3.0 safe. The change to pandas3 are substantial. Most specifically, there have been some things removed from classes, such as the to_timedelta() method of the .delta property of a pandas.timerseries.freq, but most importantly, pandas no longer allows Timedelta to have things like Y, or M me valid intervals, since they are non-ambiguous. Thus, I am not super optimistic about the IF-THEN constructs in the utilities.py transform() function continuing to work since it looks at things like the string value of a frequency to see if they have the characters Y and M. The results look OK with test10, but I think we should consider that with more complicated datasets we could definitely encounter timeseries resampling issues. Or maybe not. We can track that in #209

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant