Conversation
Update documentation to use National Laboratory of the Rockies per: https://www.energy.gov/eere/articles/energy-department-renames-nrel-national-lab-rockies At this point in time <https://nrel.gov> is still active and <https://nlr.gov> does not work, so we are keeping the nrel.gov links the same. Also the HSDS api endpoints: `/nrel/US_wave/virtual_buoy/{region}/{region}_virtual_buoy_{year}.h5` `/nrel/wtk/{region.lower()}/{region}_*.h5` `/nlr/wtk/{region.lower()}-5min/{region}_*.h5` In ``` mhkit/wave/io/hindcast/hindcast.py mhkit/wave/io/hindcast/wind_toolkit.py ``` Are not updated yet, but may change in the future.
…ves example, critical PSD bugfix (#430) Collection of bugfixes and documentation for Nortek Signature ADCPs. I received a dual-profile datafile from a Nortek Signature250 deployed at PacWave that was collecting both water velocity and wave measurements, and I made some updates to the codebase when things would fail working through the standard ADCP workflow. DOLfYN is currently set up to return individual profiles as individual datasets, so there is one dataset containing the wave-relevant variables, and one containing the water velocity variables. Both datasets contain water velocity information, so to differentiate between the two, the "_avg" tag is added to the dataset containing what Nortek calls the "averaging" profile. However, this dataset no longer comes out of the box "bin-averaged"; individual pings from each duty cycle are now saved in the output file. I've been making updates to dolfyn so that the codebase will recognize "_avg" variables and default to them if untagged variables do not exist. The following updates are the latest: - The "U_mag" and "U_dir" shortcuts will using "vel_avg" to calculate speed and direction if "vel" does not exist in the dataset. - The shear functions in the ADCP turbulence API (`dudz`, `dvdz`, etc) can now utilize "vel_avg" if given as an input. - `calc_declination` will no longer complain if you try to update the magnetic declination in the Nortek-created bin-averaged binary file ("<filename>_avgd.ad2cp") Two, I created an example notebook showing how to calculate wave statistics using DOLfYN's FFT tools. We've gotten questions about this in the past, and now that I have a good dataset, this is a good time to document this. Three, I found a bug in the PSD functions where individual FFTs get a 50% overlap not once, but twice. The core FFT function applies a 50% overlap using a series of "for loops" (the more robust method), while the input FFT function `cpsd` was adding overlap via the "npad" input to the `reshape` function. This latter method pads the first and last FFT with a lot of zeros, which in turn corrupts the first and last spectrum of a timeseries. It appears the latter method was written first and improved upon via the second method and should have been removed. Removing it fixes said bug. --------- Co-authored-by: Adam Keester <72414466+akeeste@users.noreply.github.com>
There are three main changes in this pull request: 1. Added the option to determine the number of datapoints in each FFT when creating the sound pressure PSDs. This was currently hardcoded to the maximum FFT length, i.e. the total number of datapoints in each window, which might be more resolution and require more storage space than desired 2. Changed names of bin and windowing related attributes on the PSDs to make them easier to understand 3. Gain was improperly added - the sign has been corrected Minor changes are refactoring some of the argument type checks to clean them up.
There is a bug where something in requiring pyarrow that is likely related to pandas 3.0. Adding pyarrow as a dependency is a reasonable fix, but managing the pyarrow version should be handled by pandas and not mhkit. This pins pandas below 3.0 to see if pyarrow dependencies are caused by including >= 3.0 somewhere.
Package / Actions Cleanup - Continuation of #435
Updates to `io/d3d.py` code - Added new coordinate names for the latest version of Delft3D - new `calculate_grid_convergence_index` function to calculate the grid convergence index between two grid's results basted on the equation [GCI]( https://www.grc.nasa.gov/WWW/wind/valid/tutorial/spatconv.html) - Allows xarray or netCDF4 input --------- Co-authored-by: akeeste <akeeste@sandia.gov> Co-authored-by: Simms, Andrew <andrew.simms@nlr.gov>
Opening this to solve Issue #427 I changed the source code to return degrees instead of radians, which was a simple fix. I also went ahead and updated the polar plots so that zero degrees is at the top of the figure, and positive degrees runs clockwise. We should update this in other polar plots in MHKiT too. I also updated units to pass CF conventions, whereafter I noticed that our tests for this code check that the units are the same as what NDBC outputs. I'm not sure that's necessary. I also saw that we're not actually testing the value output from the source code, which is important. --------- Co-authored-by: Simms, Andrew <andrew.simms@nlr.gov>
## Pandas 3 Changes * Updated pandas dependency to allow versions >=2.2.2 without an upper limit in both `environment-dev.yml` and `pyproject.toml`, enabling support for pandas 3.x. * Added a new function `replace_pandas_missing_values_with_nan` in `mhkit/wave/io/ndbc.py` to replace missing values with `NaN` in a way that is compatible with both pandas 2.x and 3.x. ## Test Updates * Increased `test_get_buoy_metadata` latitude and longitude assertions delta tolerance to allow for a small amount real world buoy drift. * Fix deprecation in `test_request_parse_workflow_multiyear` (`mhkit/tests/wave/io/test_cdip.py`) by using `"D"` instead of `"d"` in the `floor` method. Pandas deprecated `d` in 3.0+ pandas-dev/pandas#58998. period aliases are here: https://pandas.pydata.org/docs/user_guide/timeseries.html#period-aliases
This PR verifies MHKiT-Python supports Python 3.13. No python code changes were required. README was updated to include Python 3.13 Actions were updated to include 3.13 in test matrices and for notebook tests (single version) Action versions for setup-miniconda and download-artifact were update
Adding visualizations to match the [MHKiT_MATLAB ADCP example](https://github.com/MHKiT-Software/MHKiT-MATLAB/blob/master/examples/acoustics_example.mlx) and addressing several issues: Qhull Error #444 D3D coordinate systems #442
Enhancements: - Adds ability to convert sound spectral densities to millidecade format (https://doi.org/10.1121/10.0003324) - Adds reader for WISPR (Wideband Intelligent Signal Processing and Recording) system for hydrophones - Updates the export_audio function to input voltage timeseries and allow resampling to speed up recordings Bugfixes - Refactors band-averaging for spectral density levels and sound pressure levels. This refactor avoids losing information at frequencies located at band boundaries, and conducts all of the band-averaging immediately after the PSD calculation. The jupyter notebook has been updated with these changes as well.
This refactor solves a DOLfYN structural problem between its binning architecture and the traditional sliding-window architecture of Welch's algorithm. Because DOLfYN was using slices of bins to compute each PSD segment, it was impossible to ensure data overlap between one bin and the next, meaning the industry-standard 50% overlap cannot be accomplished. This PR removes the bin structure from the PSD computation and ensures overlap using a "step" argument, which is also now built into the bin structure to properly average dimensions for the PSD. In doing this, DOLfYN's custom PSD code, which was a custom implementation of Welch's algorithm without the ability to overlap FFT segments, has been replaced with scipy.signal.welch. This has one breaking change, in that NaN's are no longer tolerated in the PSD calculation. To avoid breaking changes, the DOLfYN code will default to an overlap of 0%. This ensures array shapes will remain the same. On the other hand, since it's new, the Acoustics module will default to 50%. The second breaking change is that the PSD code now renames the input time dimension to "time_psd". In this way, if a different overlap is used, and the PSD is saved into dataset with an averaged "time" dimension, the PSD will not follow the xarray default and go to nan. Finally, I removed the float32 datatypes in certain functions since sometimes we do need float64 precision. Removing custom code means fft.py no longer exists, and tools/misc.py was renamed to tools.py
# MHKiT-Python v1.1.0 ## Additions * Acoustics: Add millicdecade and WISPR instrument support * Added millidecade spectral conversion * Added a WISPR hydrophone reader * Added a voltage-based `export_audio` resampling option * Refactored band-averaging to avoid losing information at frequency-band boundaries * #447 * Author: @jmcvey3 * Reviewer: @simmsa * DOLfYN: Add [Nortek Aquadopp](https://www.nortekgroup.com/oceanography/aquadopp-series) ADCP support * Added DOLfYN support for reading Aquadopp instruments * Cleaned Nortek parsing code * Simplified handling of the non-cabled ADV orientation flag * #434 * Author: @jmcvey3 * Reviewer: @akeeste * Examples: Add ADCP waves example: `example/adcp_waves_example.ipynb` * Added an example notebook showing how to ingest and analyze wave measurements from a dual-profile Nortek Signature 250 deployment at PacWave. * #430 * Author: @jmcvey3 * Reviewer: @akeeste ## Improvements ### Acoustics - Improved flexibility and robustness of the Acoustics module - Added a configurable FFT length for sound pressure PSDs - Renamed bin/windowing attributes for clarity - Fixed an incorrectly signed gain correction - #433 - Author: @jmcvey3 - Reviewer: @akeeste ### DOLfYN - Refactored PSD calculations to use `scipy.signal.welch` - Replaced DOLfYN's custom Welch-like PSD implementation (built each segment from bin slices, could not overlap FFT segments) with `scipy.signal.welch` - Removed the bin-based segment structure and added a `step` argument to control overlap - Dropped float32 casts in some functions in favor of float64 - Removed `fft.py` and renamed `tools/misc.py` to `tools.py` - Breaking Changes: - NaNs are no longer tolerated in PSD calculations - DOLfYN defaults to 0% overlap to preserve existing array shapes - Acoustics defaults to 50% overlap - PSD output time dimension renamed to `time_psd` - #452 - Author: @jmcvey3 - Reviewer: @akeeste, @simmsa - Improved handling of "averaged" profiles - Fixed handling of Nortek Signature dual-profile ADCP data by defaulting to "_avg" velocity variables when untagged ones are absent. - #430 - Author: @jmcvey3 - Reviewer: @akeeste - Critical PSD bugfix - Fixed a bug where individual FFTs received a 50% overlap twice, which corrupted the first and last spectrum of a timeseries. - #430 - Author: @jmcvey3 - Reviewer: @akeeste ### Examples - Added histograms to ADCP example - #448 - Author: @browniea - Reviewer: @akeeste ### River/IO - Fixed Qhull interpolation and D3D coordinate-system errors - #448 - Author: @browniea - Reviewer: @akeeste - Fixes: #442, #444 - Delft3D module updates - Added new Delft3D coordinate names - Added a new grid-convergence-index calculation function - Added support for xarray/netCDF4 input in the D3D module - #428 - Author: @browniea - Reviewer: @akeeste ### Wave - NDBC Directional Wave Units - Fixed NDBC directional wave spectrum output to return degrees instead of radians - Updated polar plots so 0 deg is at the top and increases clockwise - #437 - Author: @jmcvey3 - Reviewer: @akeeste - Fixes: #427 ### Wave/Hindcast - Added a `hindcast_guard` exception-handling decorator (`hindcast_exceptions.py`) that surfaces a clear error on HSDS request failures, distinguishing the known NLR HSDS outage (#450) from other failures - #449 - Author: @simmsa - Reviewer: @akeeste ## Maintenance - Added Python 3.13 support - #445 - Author: @simmsa - Reviewer: @akeeste - Fixes: #441 - Added [pandas 3](https://pandas.pydata.org/community/blog/pandas-3.0.html) Support - Updated the pandas dependency to allow pandas 3.x - Added a compatibility shim for NDBC missing-value handling related to pandas 3 object to String dtype api changes - Fixed a deprecated period alias in tests - #443 - Author: @simmsa - Reviewer: @akeeste - Fixes: #440 - Updated GitHub Actions CI, expanded installation/developer documentation, refreshed dev environment, and trimmed dependencies - Refactored optional dependencies - Standardized conda/conda-forge environment builds - Scoped black linting to changed files - #436 - Author: @simmsa - Reviewer: @akeeste - Update `rex` dependency to target pypi package to [`NLR-rex[hsds]>=0.5.0`](https://pypi.org/project/NLR-rex/) - #449 - Author: @simmsa - Reviewer: @akeeste
simmsa
marked this pull request as ready for review
July 8, 2026 22:50
Contributor
Author
|
@akeeste, this is ready for review. I added the changelog above and can make any changes/edits. We can also edit the release notes as necessary. |
Contributor
Author
Contributor
Contributor
Author
Contributor
The release notes looks good to me! All our merged PRs since v1.0.1 are included |
Contributor
Author
|
@akeeste, do you think that a "Squash and merge" is the correct option here (that is the only option available to me. I did not look at the GH settings)? I'm not sure if is correct to merge all of these commits into one commit on main? I think a merge commit would be more correct? 
|
Contributor
Author
|
@akeeste, I think there was a bug on the GH side, I now see all of the options. Per our release notes "Rebase and merge" is the preferred option. Do you agree? 
|
Contributor
|
@simmsa I just updated this under Settings/Pull Requests. Disabling merge commits and rebasing by default makes it less likely we accidentally do anything but squash (this one then has to go into the settings and manually enable another option). In this case, I agree that we don't want to lose the develop history on the main branch. If rebasing is very clean and straightforward (typically if the branches were identical at the previous release and there are no commits on main that aren't on develop), that is a good option. Sometimes it gets conflicted though, in which case I'd just make things easy with a merge commit. |
akeeste
approved these changes
Jul 10, 2026
Contributor
Author
|
@akeeste, thank you! I'm going to go ahead and rebase and merge. |
Contributor
Author
|
@akeeste, for some reason I'm getting this error. 
I am investigating, will follow up when I figure out the cause and any solutions. I verified that my email is correct so I'm not sure if this is my email or some other security feature. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MHKiT-Python v1.1.0
Additions
export_audioresampling optionexample/adcp_waves_example.ipynbImprovements
Acoustics
DOLfYN
scipy.signal.welchscipy.signal.welchstepargument to control overlapfft.pyand renamedtools/misc.pytotools.pytime_psdExamples
River/IO
Wave
Wave/Hindcast
hindcast_guardexception-handling decorator (hindcast_exceptions.py) that surfaces a clear error on HSDS request failures, distinguishing the known NLR HSDS outage (Hindcast/Rex Data Downloads Tests are Failing #450) from other failuresMaintenance
test_get_buoy_metadata#440rexdependency to target pypi package toNLR-rex[hsds]>=0.5.0