Does mhkit work with the Nortek Aquadopp?

Does mhkit work with the Nortek Aquadopp? Specifically, I have data from the legacy (pre-2024) aquadopp. It is an ADCP, but it isn't mentioned in the documentation anywhere.

If mhkit does support aquadopps, which output file extension should I be trying to read?

@ddileonardo

@ddileonardo Looks like I forgot to update the reader docstring... It does support Aquadopps, but I've only had test data for the profiling versions. For those it's the .prf file, for the single point meters it is the .aqd filetype.

Thanks! I have a .prf file. The instrument was set to profile.

I am getting a KeyError when I try to use dolfyn.read.

from mhkit import dolfyn

data_path = r'GS-LMR_CH2ADP_020526.prf'
ds = dolfyn.read(data_path)

KeyError                                  Traceback (most recent call last)
Cell In[3], line 2
      1 data_path = r'GS-LMR_CH2ADP_020526.prf'
----> 2 ds = dolfyn.read(data_path)

File c:\Users\ddileonardo\.conda\envs\qbay\Lib\site-packages\mhkit\dolfyn\io\api.py:114, in read(fname, userdata, nens, **kwargs)
    112     func_map = dict(RDI=read_rdi, nortek=read_nortek, signature=read_signature)
    113     func = func_map[file_type]
--> 114 return func(fname, userdata=userdata, nens=nens, **kwargs)

File c:\Users\ddileonardo\.conda\envs\qbay\Lib\site-packages\mhkit\dolfyn\io\nortek.py:60, in read_nortek(filename, userdata, debug, do_checksum, nens, **kwargs)
     51     logging.basicConfig(
     52         filename=str(logfile),
     53         filemode="w",
     54         level=logging.NOTSET,
     55         format="%(name)s - %(levelname)s - %(message)s",
     56     )
     58 userdata = base._find_userdata(filename, userdata)
---> 60 rdr = _NortekReader(filename, debug=debug, do_checksum=do_checksum, nens=nens)
     61 rdr.readfile()
     62 rdr.cleanup()

File c:\Users\ddileonardo\.conda\envs\qbay\Lib\site-packages\mhkit\dolfyn\io\nortek.py:230, in _NortekReader.__init__(self, fname, endian, debug, do_checksum, bufsize, nens)
    228     self.config["config_type"] = "ADV"
    229 # Initialize the instrument type:
--> 230 self._inst = self.config.pop("config_type")
    231 # This is the position after reading the 'hardware',
    232 # 'head', and 'user' configuration.
    233 pnow = self.pos

KeyError: 'config_type'

@ddileonardo

@ddileonardo Can you rebuild your conda environment with the latest version of MHKiT we just released? Looks like you're running an older version.
If you open up your .prf file in a text editor, you will will see a few ascii characters that spell out the serial number at the very beginning of the file. MHKiT reads this serial number, which will start with "AQD" or "PRF" for Aquadopp profilers, to determine the correct reader to use. If yours doesn't start with either of those, please let me know.

I used the method from the documentation to install
conda install -c conda-forge mhkit

Since it doesn't specify a version, I would have though it should be the most recent version? I installed in late July/early August.

When I try to update with conda, the only package that updates is openssl. I still get the same error.

I have tried creating a whole new environment and installing mhkit in the same way. I'm still getting the same error.

Is there a different conda channel that I need to use?

As far as the top of prf file goes, when I open in text editor I see a couple bits of binary and the AQD with a serial number so that seems right.

Oh, please use pip install mhkit to install the full package, or pip install mhkit[dolfyn] for just the ADCP-related dependencies. We haven't yet updated the conda-forge package for the latest release.

After the pip install I get an immediate import error for bottleneck when trying to import dolfyn or mhkit. Is there a specific python version I need or something else?

My install history in the environment goes:

conda install pythong
conda install pip
pip install mhkit
pip install ipykernel

So all the package version should be defined by pip.
pandas 3.0.5 pypi_0 pypi
numpy 2.5.1 pypi_0 pypi

ImportError                               Traceback (most recent call last)
Cell In[1], [line 1](vscode-notebook-cell:?execution_count=1&line=1)
----> [1](vscode-notebook-cell:?execution_count=1&line=1) from mhkit import dolfyn
      2 from mhkit.dolfyn.adp import api
      3 import numpy as np
      4 from matplotlib import pyplot as plt

File ~\.conda\envs\qbay-pip\Lib\site-packages\mhkit\__init__.py:5
      2 import importlib
      4 # Register datetime converter for a matplotlib plotting methods
----> [5](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/mhkit/__init__.py:5) from pandas.plotting import register_matplotlib_converters as _rmc
      7 _rmc()
      9 # Use targeted warning configuration

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\__init__.py:58
     55 # let init-time option registration happen
     56 import pandas.core.config_init  # pyright: ignore[reportUnusedImport] # noqa: F401
---> [58](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/__init__.py:58) from pandas.core.api import (
     59     # dtype
     60     ArrowDtype,
     61     Int8Dtype,
     62     Int16Dtype,
     63     Int32Dtype,
     64     Int64Dtype,
     65     UInt8Dtype,
     66     UInt16Dtype,
     67     UInt32Dtype,
     68     UInt64Dtype,
     69     Float32Dtype,
     70     Float64Dtype,
     71     CategoricalDtype,
     72     PeriodDtype,
     73     IntervalDtype,
     74     DatetimeTZDtype,
     75     StringDtype,
     76     BooleanDtype,
     77     # missing
     78     NA,
     79     isna,
     80     isnull,
     81     notna,
     82     notnull,
     83     # indexes
     84     Index,
     85     CategoricalIndex,
     86     RangeIndex,
     87     MultiIndex,
     88     IntervalIndex,
     89     TimedeltaIndex,
     90     DatetimeIndex,
     91     PeriodIndex,
     92     IndexSlice,
     93     # tseries
     94     NaT,
     95     Period,
     96     period_range,
     97     Timedelta,
     98     timedelta_range,
     99     Timestamp,
    100     date_range,
    101     bdate_range,
    102     Interval,
    103     interval_range,
    104     DateOffset,
    105     # conversion
    106     to_numeric,
    107     to_datetime,
    108     to_timedelta,
    109     # misc
    110     Flags,
    111     Grouper,
    112     factorize,
    113     unique,
    114     NamedAgg,
    115     array,
    116     Categorical,
    117     set_eng_float_format,
    118     Series,
    119     DataFrame,
    120 )
    121 from pandas.core.col import col
    123 from pandas.core.dtypes.dtypes import SparseDtype

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\core\api.py:[27](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/core/api.py:27)
     16 from pandas.core.dtypes.missing import (
     17     isna,
     18     isnull,
     19     notna,
     20     notnull,
     21 )
     23 from pandas.core.algorithms import (
     24     factorize,
     25     unique,
     26 )
---> 27 from pandas.core.arrays import Categorical
     28 from pandas.core.arrays.boolean import BooleanDtype
     29 from pandas.core.arrays.floating import (
     30     Float32Dtype,
     3[1](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/core/arrays/__init__.py:1)     Float64Dtype,
     32 )

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\core\arrays\__init__.py:1
----> 1 from pandas.core.arrays.arrow import ArrowExtensionArray
      2 from pandas.core.arrays.base import (
      3     ExtensionArray,
      4     ExtensionOpsMixin,
      5     ExtensionScalarOpsMixin,
      6 )
      7 from pandas.core.arrays.boolean import BooleanArray

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\core\arrays\arrow\__init__.py:[5](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/core/arrays/arrow/__init__.py:5)
      1 from pandas.core.arrays.arrow.accessors import (
      2     ListAccessor,
      3     StructAccessor,
      4 )
----> 5 from pandas.core.arrays.arrow.array import ArrowExtensionArray
      7 __all__ = ["ArrowExtensionArray", "ListAccessor", "StructAccessor"]

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\core\arrays\arrow\array.py:78
     73 from pandas.core.arrays._utils import to_numpy_dtype_inference
     74 from pandas.core.arrays.base import (
     75     ExtensionArray,
     76     ExtensionArraySupportsAnyAll,
     77 )
---> [78](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/core/arrays/arrow/array.py:78) from pandas.core.arrays.masked import BaseMaskedArray
     79 from pandas.core.arrays.string_ import StringDtype
     80 import pandas.core.common as com

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\core\arrays\masked.py:56
     45 from pandas.core.dtypes.dtypes import (
     46     ArrowDtype,
     47     BaseMaskedDtype,
     48 )
     49 from pandas.core.dtypes.missing import (
     50     array_equivalent,
     51     is_valid_na_for_dtype,
     52     isna,
     53     notna,
     54 )
---> [56](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/core/arrays/masked.py:56) from pandas.core import (
     57     algorithms as algos,
     58     arraylike,
     59     missing,
     60     nanops,
     61     ops,
     62 )
     63 from pandas.core.algorithms import (
     64     factorize_array,
     65     isin,
   (...)     68     take,
     69 )
     70 from pandas.core.array_algos import (
     71     masked_accumulations,
     72     masked_reductions,
     73 )

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\core\nanops.py:[54](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/core/nanops.py:54)
     51 if TYPE_CHECKING:
     52     from collections.abc import Callable
---> 54 bn = import_optional_dependency("bottleneck", errors="warn")
     55 _BOTTLENECK_INSTALLED = bn is not None
     56 _USE_BOTTLENECK = False

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\compat\_optional.py:173, in import_optional_dependency(name, extra, min_version, errors)
    171 minimum_version = min_version if min_version is not None else VERSIONS.get(parent)
    172 if minimum_version:
--> [173](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/compat/_optional.py:173)     version = get_version(module_to_get)
    174     if version and Version(version) < Version(minimum_version):
    175         msg = (
    176             f"Pandas requires version '{minimum_version}' or newer of '{parent}' "
    177             f"(version '{version}' currently installed)."
    178         )

File ~\.conda\envs\qbay-pip\Lib\site-packages\pandas\compat\_optional.py:80, in get_version(module)
     77 version = getattr(module, "__version__", None)
     79 if version is None:
---> [80](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ddileonardo/Documents/GitHub/GS-LMR/~/.conda/envs/qbay-pip/Lib/site-packages/pandas/compat/_optional.py:80)     raise ImportError(f"Can't determine version for {module.__name__}")
     81 if module.__name__ == "psycopg2":
     82     # psycopg2 appends " (dt dec pq3 ext lo64)" to it's version
     83     version = version.split()[0]

ImportError: Can't determine version for bottleneck

Huh, we haven't seen that pandas error before. What happens if you run pip install bottleneck?

Looking at my package list, it seems like bottleneck wasn't installed before, for some reason. Installing it as above fixed the import issue.

Now though I am getting an error related to the beginning of the file:

WARNING:root:Unrecognized identifier: 0x9d

The first few characters look a bit like this when opened in notepad++

¥�� AQD11219

It looks like a bit of binary and then those ascii characters.

We are using legacy aquadopps, which might be useful information if the aquadopp workflow was developed with the newer gen 2 aquadopps.

Noted, that should've been installed with the package.

Yes, that's the serial number. I've tested with an older instrument than yours (based on the serial number) and one of the new ones.

Can you run ds = dolfyn.read(data_path, debug=True) and upload the resulting log file here? There is no 0x9d ID, looking at Nortek's integrator guide

Here is the log file

GS-LMR_CH2ADP_050726.dolfyn.log

I had a peek in there and it suggested to me that there was an issue with the data file. I tried a different data file and it read successfully.

Thank you very much for all your help.

Is there an odd number of bins in the datafile with the issue? And an even number of bins in the datafile that ran successfully?

There is a note in the docs that states the former will include an extra "fill" byte, but we've never seen that to be true before. But it looks like that might be true for your case.

Yes, there are 5 cells listed in the hdr file. There are 10 cells list in the hdr file of the file that works.

Well this might be really easy. Can you pull PR #459 and try that?

You could also copy-paste the changes in https://github.com/MHKiT-Software/MHKiT-Python/pull/459/changes if that's faster for you

I'm trying, but it's not quite working.

I've cloned the repo, but I couldn't figure out how to pull a PR, so I just made the changes in nortek.py. But when I tried to build it as in the documentation

pip install -e .["all"]

Something didn't quite work. I've got python 3.14, but that wasn't an issue before.

This is the pip error from the build

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for numcodecs
Building wheel for bitshuffle (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for bitshuffle (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [18 lines of output]
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
running bdist_wheel
running build
running build_py
creating build\lib.win-amd64-cpython-314\bitshuffle
copying bitshuffle_init_.py -> build\lib.win-amd64-cpython-314\bitshuffle
running build_ext

  #################################
  # Compiling with OpenMP support #
  #################################
  
  building 'bitshuffle.ext' extension
  error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for bitshuffle
Successfully built mhkit
Failed to build cartopy aiohttp numcodecs bitshuffle
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> cartopy, aiohttp, numcodecs, bitshuffle

Alright, we got there. And it works!

The install of all the required packages is potentially a big mess and MHKit is not properly installed as a package, but I've got al the files with the edits and was able to import them. And my odd bin file now works

Great! But yes, sounds like we need to update our installation page

I think the install files might need a check over, but for testing that fix, it works

Should I expect that the pip install will be updated with this fix or should I keep going with the test environment I just created?

Thanks again for all your help!

Happy to help! Please continue with the test environment, it might take a minute (read: week or two) to release a hotfix. The conda-forge release will also take about that time

If you'd like to have a clean environment, "cd" into the "MHKiT-Python" folder and run, from an anaconda prompt,

conda activate base
conda env create
conda activate mhkit-env
pip install -e ".[all]"

That should install the whole package properly, and then in the future run your python code after activating the "mhkit-env" environment.

That still fails at the same place as before. Not sure what is going on there, but I will likely just go with what I was able to cobble together before, and check back in a couple of weeks. I'm likely to be putting this project down for a couple of weeks starting tomorrow anyway.

Building wheels for collected packages: mhkit, bitshuffle
Building editable for mhkit (pyproject.toml) ... done
Created wheel for mhkit: filename=mhkit-1.1.0-0.editable-py3-none-any.whl size=9137 sha256=66f742b6ea01fc0b6cddeb235df6cc09dc77016190cd3e108a4fe55fae01a6fa
Stored in directory: C:\Users\ddileonardo\AppData\Local\Temp\pip-ephem-wheel-cache-d7k69etx\wheels\ba\e4\a3\133e46a242694d82f1dd6da9ea21e938cca9c013d0b8f1396a
Building wheel for bitshuffle (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for bitshuffle (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [18 lines of output]
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
Can't find hdf5 with pkg-config fallback to static config
running bdist_wheel
running build
running build_py
creating build\lib.win-amd64-cpython-311\bitshuffle
copying bitshuffle_init_.py -> build\lib.win-amd64-cpython-311\bitshuffle
running build_ext

  #################################
  # Compiling with OpenMP support #
  #################################
  
  building 'bitshuffle.ext' extension
  error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for bitshuffle
Successfully built mhkit
Failed to build bitshuffle
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> bitshuffle

This looks like some sort of windows installation problem with python or certain python packages. It sounds like the conda-forge version installed fine for you, and we'll have that out soon.

@jmcvey3

@jmcvey3 and @ddileonardo,

We just updated our experimental desktop application to mhkit 1.1.1 which leverages dolfyn "under the hood":

https://github.com/MHKiT-Software/mhkit-dolfyn-gui

There is a windows executable here: https://github.com/MHKiT-Software/mhkit-dolfyn-gui/releases/tag/v0.3.2

That has the python libraries "inside" and should work for reading Nortek Aquadopp files, but please let us know if that does not work.

I'm also working on the conda-forge release which has some non dolfyn related issues and can report back when that is available

@ddileonardo

@ddileonardo to get the latest mhkit dolfyn (v1.1.1) within a conda environment the steps below should work:

conda env create mhkit-env --python=3.11

conda activate mhkit-env

pip install "mhkit[dolfyn]"

Checking the mhkit version:

python -c "import mhkit; print(mhkit.__version__)"

Should yield:

v1.1.1

Note: pip install mhkit doesn't install any libraries, which is why you get import issues (this is a bit confusing, we should probably make this more obvious). You have to specify a module which is detailed in the documentation here https://github.com/MHKiT-Software/MHKiT-Python#module-specific-install-using-pip.

And if you want all modules in mhkit pass all, ie `pip install "mhkit[all]".

@simmsa

@simmsa Oh no I'm also getting this same error trying to pip install mhkit[all]

Closing this and opening this package build error in #461

Navigation Menu

Repository navigation

Does mhkit work with the Nortek Aquadopp? #458

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Does mhkit work with the Nortek Aquadopp? #458

Description

Activity

jmcvey3 commented on Jul 29, 2026

ddileonardo commented on Jul 30, 2026

jmcvey3 commented on Aug 4, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 5, 2026

ddileonardo commented on Aug 5, 2026

jmcvey3 commented on Aug 7, 2026

simmsa commented on Aug 7, 2026

simmsa commented on Aug 7, 2026

jmcvey3 commented on Aug 7, 2026

jmcvey3 commented on Aug 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions