Feature: Add support for pandas 3 - #443
Merged
akeeste merged 13 commits intoJun 9, 2026
Merged
Conversation
Pandas deprecated lower case d here: pandas-dev/pandas#58998 Upper case d `D` is calendar day: https://pandas.pydata.org/docs/user_guide/timeseries.html#period-aliases
simmsa
marked this pull request as draft
June 8, 2026 14:36
pyarrow provides native string dtype support for mhkit. While technically optional, mhkit benefits from one standardized backend for representing columnar data. At this time pyarrow is that standard.
akeeste
self-requested a review
June 8, 2026 19:42
The narrow hdf5 range causes the solve when pyarrow is included to be slow. Increasing the allowed range improves the pip solver speed.
These should no longer be necessary with the changes to dependencies
This removes any explicit coveralls intalls in favor of the coverallsapp/github-action@v2 which is self contained and contains all binaries necessary for coveralls to run.
Agg is a non interactive matplotlib backend and lets the tests runh
"headless". Without this specification tests can choose their backend
which can sometimes lead to failing tests on headless machines (GH
Actions).
simmsa
marked this pull request as ready for review
June 9, 2026 18:13
Contributor
|
All tests passed on the last significant commit. Approved and merging. |
Contributor
Author
|
@akeeste, thank you! |
3 tasks
simmsa
added a commit
that referenced
this pull request
Jul 8, 2026
# MHKiT-Python v1.1.0 ## Additions * Acoustics: Add millicdecade and WISPR instrument support * Added millidecade spectral conversion * Added a WISPR hydrophone reader * Added a voltage-based `export_audio` resampling option * Refactored band-averaging to avoid losing information at frequency-band boundaries * #447 * Author: @jmcvey3 * Reviewer: @simmsa * DOLfYN: Add [Nortek Aquadopp](https://www.nortekgroup.com/oceanography/aquadopp-series) ADCP support * Added DOLfYN support for reading Aquadopp instruments * Cleaned Nortek parsing code * Simplified handling of the non-cabled ADV orientation flag * #434 * Author: @jmcvey3 * Reviewer: @akeeste * Examples: Add ADCP waves example: `example/adcp_waves_example.ipynb` * Added an example notebook showing how to ingest and analyze wave measurements from a dual-profile Nortek Signature 250 deployment at PacWave. * #430 * Author: @jmcvey3 * Reviewer: @akeeste ## Improvements ### Acoustics - Improved flexibility and robustness of the Acoustics module - Added a configurable FFT length for sound pressure PSDs - Renamed bin/windowing attributes for clarity - Fixed an incorrectly signed gain correction - #433 - Author: @jmcvey3 - Reviewer: @akeeste ### DOLfYN - Refactored PSD calculations to use `scipy.signal.welch` - Replaced DOLfYN's custom Welch-like PSD implementation (built each segment from bin slices, could not overlap FFT segments) with `scipy.signal.welch` - Removed the bin-based segment structure and added a `step` argument to control overlap - Dropped float32 casts in some functions in favor of float64 - Removed `fft.py` and renamed `tools/misc.py` to `tools.py` - Breaking Changes: - NaNs are no longer tolerated in PSD calculations - DOLfYN defaults to 0% overlap to preserve existing array shapes - Acoustics defaults to 50% overlap - PSD output time dimension renamed to `time_psd` - #452 - Author: @jmcvey3 - Reviewer: @akeeste, @simmsa - Improved handling of "averaged" profiles - Fixed handling of Nortek Signature dual-profile ADCP data by defaulting to "_avg" velocity variables when untagged ones are absent. - #430 - Author: @jmcvey3 - Reviewer: @akeeste - Critical PSD bugfix - Fixed a bug where individual FFTs received a 50% overlap twice, which corrupted the first and last spectrum of a timeseries. - #430 - Author: @jmcvey3 - Reviewer: @akeeste ### Examples - Added histograms to ADCP example - #448 - Author: @browniea - Reviewer: @akeeste ### River/IO - Fixed Qhull interpolation and D3D coordinate-system errors - #448 - Author: @browniea - Reviewer: @akeeste - Fixes: #442, #444 - Delft3D module updates - Added new Delft3D coordinate names - Added a new grid-convergence-index calculation function - Added support for xarray/netCDF4 input in the D3D module - #428 - Author: @browniea - Reviewer: @akeeste ### Wave - NDBC Directional Wave Units - Fixed NDBC directional wave spectrum output to return degrees instead of radians - Updated polar plots so 0 deg is at the top and increases clockwise - #437 - Author: @jmcvey3 - Reviewer: @akeeste - Fixes: #427 ### Wave/Hindcast - Added a `hindcast_guard` exception-handling decorator (`hindcast_exceptions.py`) that surfaces a clear error on HSDS request failures, distinguishing the known NLR HSDS outage (#450) from other failures - #449 - Author: @simmsa - Reviewer: @akeeste ## Maintenance - Added Python 3.13 support - #445 - Author: @simmsa - Reviewer: @akeeste - Fixes: #441 - Added [pandas 3](https://pandas.pydata.org/community/blog/pandas-3.0.html) Support - Updated the pandas dependency to allow pandas 3.x - Added a compatibility shim for NDBC missing-value handling related to pandas 3 object to String dtype api changes - Fixed a deprecated period alias in tests - #443 - Author: @simmsa - Reviewer: @akeeste - Fixes: #440 - Updated GitHub Actions CI, expanded installation/developer documentation, refreshed dev environment, and trimmed dependencies - Refactored optional dependencies - Standardized conda/conda-forge environment builds - Scoped black linting to changed files - #436 - Author: @simmsa - Reviewer: @akeeste - Update `rex` dependency to target pypi package to [`NLR-rex[hsds]>=0.5.0`](https://pypi.org/project/NLR-rex/) - #449 - Author: @simmsa - Reviewer: @akeeste
Merged
simmsa
added a commit
to simmsa/MHKiT-Python
that referenced
this pull request
Jul 10, 2026
## Pandas 3 Changes * Updated pandas dependency to allow versions >=2.2.2 without an upper limit in both `environment-dev.yml` and `pyproject.toml`, enabling support for pandas 3.x. * Added a new function `replace_pandas_missing_values_with_nan` in `mhkit/wave/io/ndbc.py` to replace missing values with `NaN` in a way that is compatible with both pandas 2.x and 3.x. ## Test Updates * Increased `test_get_buoy_metadata` latitude and longitude assertions delta tolerance to allow for a small amount real world buoy drift. * Fix deprecation in `test_request_parse_workflow_multiyear` (`mhkit/tests/wave/io/test_cdip.py`) by using `"D"` instead of `"d"` in the `floor` method. Pandas deprecated `d` in 3.0+ pandas-dev/pandas#58998. period aliases are here: https://pandas.pydata.org/docs/user_guide/timeseries.html#period-aliases
simmsa
added a commit
to simmsa/MHKiT-Python
that referenced
this pull request
Jul 10, 2026
# MHKiT-Python v1.1.0 ## Additions * Acoustics: Add millicdecade and WISPR instrument support * Added millidecade spectral conversion * Added a WISPR hydrophone reader * Added a voltage-based `export_audio` resampling option * Refactored band-averaging to avoid losing information at frequency-band boundaries * MHKiT-Software#447 * Author: @jmcvey3 * Reviewer: @simmsa * DOLfYN: Add [Nortek Aquadopp](https://www.nortekgroup.com/oceanography/aquadopp-series) ADCP support * Added DOLfYN support for reading Aquadopp instruments * Cleaned Nortek parsing code * Simplified handling of the non-cabled ADV orientation flag * MHKiT-Software#434 * Author: @jmcvey3 * Reviewer: @akeeste * Examples: Add ADCP waves example: `example/adcp_waves_example.ipynb` * Added an example notebook showing how to ingest and analyze wave measurements from a dual-profile Nortek Signature 250 deployment at PacWave. * MHKiT-Software#430 * Author: @jmcvey3 * Reviewer: @akeeste ## Improvements ### Acoustics - Improved flexibility and robustness of the Acoustics module - Added a configurable FFT length for sound pressure PSDs - Renamed bin/windowing attributes for clarity - Fixed an incorrectly signed gain correction - MHKiT-Software#433 - Author: @jmcvey3 - Reviewer: @akeeste ### DOLfYN - Refactored PSD calculations to use `scipy.signal.welch` - Replaced DOLfYN's custom Welch-like PSD implementation (built each segment from bin slices, could not overlap FFT segments) with `scipy.signal.welch` - Removed the bin-based segment structure and added a `step` argument to control overlap - Dropped float32 casts in some functions in favor of float64 - Removed `fft.py` and renamed `tools/misc.py` to `tools.py` - Breaking Changes: - NaNs are no longer tolerated in PSD calculations - DOLfYN defaults to 0% overlap to preserve existing array shapes - Acoustics defaults to 50% overlap - PSD output time dimension renamed to `time_psd` - MHKiT-Software#452 - Author: @jmcvey3 - Reviewer: @akeeste, @simmsa - Improved handling of "averaged" profiles - Fixed handling of Nortek Signature dual-profile ADCP data by defaulting to "_avg" velocity variables when untagged ones are absent. - MHKiT-Software#430 - Author: @jmcvey3 - Reviewer: @akeeste - Critical PSD bugfix - Fixed a bug where individual FFTs received a 50% overlap twice, which corrupted the first and last spectrum of a timeseries. - MHKiT-Software#430 - Author: @jmcvey3 - Reviewer: @akeeste ### Examples - Added histograms to ADCP example - MHKiT-Software#448 - Author: @browniea - Reviewer: @akeeste ### River/IO - Fixed Qhull interpolation and D3D coordinate-system errors - MHKiT-Software#448 - Author: @browniea - Reviewer: @akeeste - Fixes: MHKiT-Software#442, MHKiT-Software#444 - Delft3D module updates - Added new Delft3D coordinate names - Added a new grid-convergence-index calculation function - Added support for xarray/netCDF4 input in the D3D module - MHKiT-Software#428 - Author: @browniea - Reviewer: @akeeste ### Wave - NDBC Directional Wave Units - Fixed NDBC directional wave spectrum output to return degrees instead of radians - Updated polar plots so 0 deg is at the top and increases clockwise - MHKiT-Software#437 - Author: @jmcvey3 - Reviewer: @akeeste - Fixes: MHKiT-Software#427 ### Wave/Hindcast - Added a `hindcast_guard` exception-handling decorator (`hindcast_exceptions.py`) that surfaces a clear error on HSDS request failures, distinguishing the known NLR HSDS outage (MHKiT-Software#450) from other failures - MHKiT-Software#449 - Author: @simmsa - Reviewer: @akeeste ## Maintenance - Added Python 3.13 support - MHKiT-Software#445 - Author: @simmsa - Reviewer: @akeeste - Fixes: MHKiT-Software#441 - Added [pandas 3](https://pandas.pydata.org/community/blog/pandas-3.0.html) Support - Updated the pandas dependency to allow pandas 3.x - Added a compatibility shim for NDBC missing-value handling related to pandas 3 object to String dtype api changes - Fixed a deprecated period alias in tests - MHKiT-Software#443 - Author: @simmsa - Reviewer: @akeeste - Fixes: MHKiT-Software#440 - Updated GitHub Actions CI, expanded installation/developer documentation, refreshed dev environment, and trimmed dependencies - Refactored optional dependencies - Standardized conda/conda-forge environment builds - Scoped black linting to changed files - MHKiT-Software#436 - Author: @simmsa - Reviewer: @akeeste - Update `rex` dependency to target pypi package to [`NLR-rex[hsds]>=0.5.0`](https://pypi.org/project/NLR-rex/) - MHKiT-Software#449 - Author: @simmsa - Reviewer: @akeeste
simmsa
added a commit
that referenced
this pull request
Jul 10, 2026
## Pandas 3 Changes * Updated pandas dependency to allow versions >=2.2.2 without an upper limit in both `environment-dev.yml` and `pyproject.toml`, enabling support for pandas 3.x. * Added a new function `replace_pandas_missing_values_with_nan` in `mhkit/wave/io/ndbc.py` to replace missing values with `NaN` in a way that is compatible with both pandas 2.x and 3.x. ## Test Updates * Increased `test_get_buoy_metadata` latitude and longitude assertions delta tolerance to allow for a small amount real world buoy drift. * Fix deprecation in `test_request_parse_workflow_multiyear` (`mhkit/tests/wave/io/test_cdip.py`) by using `"D"` instead of `"d"` in the `floor` method. Pandas deprecated `d` in 3.0+ pandas-dev/pandas#58998. period aliases are here: https://pandas.pydata.org/docs/user_guide/timeseries.html#period-aliases
simmsa
added a commit
that referenced
this pull request
Jul 10, 2026
# MHKiT-Python v1.1.0 ## Additions * Acoustics: Add millicdecade and WISPR instrument support * Added millidecade spectral conversion * Added a WISPR hydrophone reader * Added a voltage-based `export_audio` resampling option * Refactored band-averaging to avoid losing information at frequency-band boundaries * #447 * Author: @jmcvey3 * Reviewer: @simmsa * DOLfYN: Add [Nortek Aquadopp](https://www.nortekgroup.com/oceanography/aquadopp-series) ADCP support * Added DOLfYN support for reading Aquadopp instruments * Cleaned Nortek parsing code * Simplified handling of the non-cabled ADV orientation flag * #434 * Author: @jmcvey3 * Reviewer: @akeeste * Examples: Add ADCP waves example: `example/adcp_waves_example.ipynb` * Added an example notebook showing how to ingest and analyze wave measurements from a dual-profile Nortek Signature 250 deployment at PacWave. * #430 * Author: @jmcvey3 * Reviewer: @akeeste ## Improvements ### Acoustics - Improved flexibility and robustness of the Acoustics module - Added a configurable FFT length for sound pressure PSDs - Renamed bin/windowing attributes for clarity - Fixed an incorrectly signed gain correction - #433 - Author: @jmcvey3 - Reviewer: @akeeste ### DOLfYN - Refactored PSD calculations to use `scipy.signal.welch` - Replaced DOLfYN's custom Welch-like PSD implementation (built each segment from bin slices, could not overlap FFT segments) with `scipy.signal.welch` - Removed the bin-based segment structure and added a `step` argument to control overlap - Dropped float32 casts in some functions in favor of float64 - Removed `fft.py` and renamed `tools/misc.py` to `tools.py` - Breaking Changes: - NaNs are no longer tolerated in PSD calculations - DOLfYN defaults to 0% overlap to preserve existing array shapes - Acoustics defaults to 50% overlap - PSD output time dimension renamed to `time_psd` - #452 - Author: @jmcvey3 - Reviewer: @akeeste, @simmsa - Improved handling of "averaged" profiles - Fixed handling of Nortek Signature dual-profile ADCP data by defaulting to "_avg" velocity variables when untagged ones are absent. - #430 - Author: @jmcvey3 - Reviewer: @akeeste - Critical PSD bugfix - Fixed a bug where individual FFTs received a 50% overlap twice, which corrupted the first and last spectrum of a timeseries. - #430 - Author: @jmcvey3 - Reviewer: @akeeste ### Examples - Added histograms to ADCP example - #448 - Author: @browniea - Reviewer: @akeeste ### River/IO - Fixed Qhull interpolation and D3D coordinate-system errors - #448 - Author: @browniea - Reviewer: @akeeste - Fixes: #442, #444 - Delft3D module updates - Added new Delft3D coordinate names - Added a new grid-convergence-index calculation function - Added support for xarray/netCDF4 input in the D3D module - #428 - Author: @browniea - Reviewer: @akeeste ### Wave - NDBC Directional Wave Units - Fixed NDBC directional wave spectrum output to return degrees instead of radians - Updated polar plots so 0 deg is at the top and increases clockwise - #437 - Author: @jmcvey3 - Reviewer: @akeeste - Fixes: #427 ### Wave/Hindcast - Added a `hindcast_guard` exception-handling decorator (`hindcast_exceptions.py`) that surfaces a clear error on HSDS request failures, distinguishing the known NLR HSDS outage (#450) from other failures - #449 - Author: @simmsa - Reviewer: @akeeste ## Maintenance - Added Python 3.13 support - #445 - Author: @simmsa - Reviewer: @akeeste - Fixes: #441 - Added [pandas 3](https://pandas.pydata.org/community/blog/pandas-3.0.html) Support - Updated the pandas dependency to allow pandas 3.x - Added a compatibility shim for NDBC missing-value handling related to pandas 3 object to String dtype api changes - Fixed a deprecated period alias in tests - #443 - Author: @simmsa - Reviewer: @akeeste - Fixes: #440 - Updated GitHub Actions CI, expanded installation/developer documentation, refreshed dev environment, and trimmed dependencies - Refactored optional dependencies - Standardized conda/conda-forge environment builds - Scoped black linting to changed files - #436 - Author: @simmsa - Reviewer: @akeeste - Update `rex` dependency to target pypi package to [`NLR-rex[hsds]>=0.5.0`](https://pypi.org/project/NLR-rex/) - #449 - Author: @simmsa - Reviewer: @akeeste
simmsa
added a commit
that referenced
this pull request
Jul 10, 2026
# MHKiT-Python v1.1.0 ## Additions * Acoustics: Add millicdecade and WISPR instrument support * Added millidecade spectral conversion * Added a WISPR hydrophone reader * Added a voltage-based `export_audio` resampling option * Refactored band-averaging to avoid losing information at frequency-band boundaries * #447 * Author: @jmcvey3 * Reviewer: @simmsa * DOLfYN: Add [Nortek Aquadopp](https://www.nortekgroup.com/oceanography/aquadopp-series) ADCP support * Added DOLfYN support for reading Aquadopp instruments * Cleaned Nortek parsing code * Simplified handling of the non-cabled ADV orientation flag * #434 * Author: @jmcvey3 * Reviewer: @akeeste * Examples: Add ADCP waves example: `example/adcp_waves_example.ipynb` * Added an example notebook showing how to ingest and analyze wave measurements from a dual-profile Nortek Signature 250 deployment at PacWave. * #430 * Author: @jmcvey3 * Reviewer: @akeeste ## Improvements ### Acoustics - Improved flexibility and robustness of the Acoustics module - Added a configurable FFT length for sound pressure PSDs - Renamed bin/windowing attributes for clarity - Fixed an incorrectly signed gain correction - #433 - Author: @jmcvey3 - Reviewer: @akeeste ### DOLfYN - Refactored PSD calculations to use `scipy.signal.welch` - Replaced DOLfYN's custom Welch-like PSD implementation (built each segment from bin slices, could not overlap FFT segments) with `scipy.signal.welch` - Removed the bin-based segment structure and added a `step` argument to control overlap - Dropped float32 casts in some functions in favor of float64 - Removed `fft.py` and renamed `tools/misc.py` to `tools.py` - Breaking Changes: - NaNs are no longer tolerated in PSD calculations - DOLfYN defaults to 0% overlap to preserve existing array shapes - Acoustics defaults to 50% overlap - PSD output time dimension renamed to `time_psd` - #452 - Author: @jmcvey3 - Reviewer: @akeeste, @simmsa - Improved handling of "averaged" profiles - Fixed handling of Nortek Signature dual-profile ADCP data by defaulting to "_avg" velocity variables when untagged ones are absent. - #430 - Author: @jmcvey3 - Reviewer: @akeeste - Critical PSD bugfix - Fixed a bug where individual FFTs received a 50% overlap twice, which corrupted the first and last spectrum of a timeseries. - #430 - Author: @jmcvey3 - Reviewer: @akeeste ### Examples - Added histograms to ADCP example - #448 - Author: @browniea - Reviewer: @akeeste ### River/IO - Fixed Qhull interpolation and D3D coordinate-system errors - #448 - Author: @browniea - Reviewer: @akeeste - Fixes: #442, #444 - Delft3D module updates - Added new Delft3D coordinate names - Added a new grid-convergence-index calculation function - Added support for xarray/netCDF4 input in the D3D module - #428 - Author: @browniea - Reviewer: @akeeste ### Wave - NDBC Directional Wave Units - Fixed NDBC directional wave spectrum output to return degrees instead of radians - Updated polar plots so 0 deg is at the top and increases clockwise - #437 - Author: @jmcvey3 - Reviewer: @akeeste - Fixes: #427 ### Wave/Hindcast - Added a `hindcast_guard` exception-handling decorator (`hindcast_exceptions.py`) that surfaces a clear error on HSDS request failures, distinguishing the known NLR HSDS outage (#450) from other failures - #449 - Author: @simmsa - Reviewer: @akeeste ## Maintenance - Added Python 3.13 support - #445 - Author: @simmsa - Reviewer: @akeeste - Fixes: #441 - Added [pandas 3](https://pandas.pydata.org/community/blog/pandas-3.0.html) Support - Updated the pandas dependency to allow pandas 3.x - Added a compatibility shim for NDBC missing-value handling related to pandas 3 object to String dtype api changes - Fixed a deprecated period alias in tests - #443 - Author: @simmsa - Reviewer: @akeeste - Fixes: #440 - Updated GitHub Actions CI, expanded installation/developer documentation, refreshed dev environment, and trimmed dependencies - Refactored optional dependencies - Standardized conda/conda-forge environment builds - Scoped black linting to changed files - #436 - Author: @simmsa - Reviewer: @akeeste - Update `rex` dependency to target pypi package to [`NLR-rex[hsds]>=0.5.0`](https://pypi.org/project/NLR-rex/) - #449 - Author: @simmsa - Reviewer: @akeeste
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pandas 3 Changes
environment-dev.ymlandpyproject.toml, enabling support for pandas 3.x.replace_pandas_missing_values_with_naninmhkit/wave/io/ndbc.pyto replace missing values withNaNin a way that is compatible with both pandas 2.x and 3.x.Test Updates
test_get_buoy_metadatalatitude and longitude assertions delta tolerance to allow for a small amount real world buoy drift.test_request_parse_workflow_multiyear(mhkit/tests/wave/io/test_cdip.py) by using"D"instead of"d"in thefloormethod. Pandas deprecateddin 3.0+ DEPR: lowercase stringsw,d,bandcdenoting frequencies inWeek,Day,BusinessDayandCustomBusinessDayclasses pandas-dev/pandas#58998. period aliases are here:https://pandas.pydata.org/docs/user_guide/timeseries.html#period-aliases