Unresolved external using static build on Windows

[andrewcorrigan]

(EDIT: Sorry, pressed a keyboard shortcut to post, before finishing)

I'm trying to build NetCDF and its dependencies on Windows as static libraries, for use with Exodus. When I link my application (manually listing the static libraries in many different orders) I'm getting the error messages shown below about unresolved external symbols __imp_H5DS, etc.

I noticed that these symbols are present in netcdf.lib

(base) PS > dumpbin /symbols netcdf.lib | Select-String -Pattern H5DS

05E 00000000 UNDEF  notype       External     | __imp_H5Dset_extent
07A 00000000 UNDEF  notype       External     | __imp_H5DSget_num_scales
07B 00000000 UNDEF  notype       External     | __imp_H5DSget_scale_name
07C 00000000 UNDEF  notype       External     | __imp_H5DSis_scale
07D 00000000 UNDEF  notype       External     | __imp_H5DSiterate_scales
038 00000000 UNDEF  notype       External     | __imp_H5DSdetach_scale
072 00000000 UNDEF  notype       External     | __imp_H5Dset_extent
097 00000000 UNDEF  notype       External     | __imp_H5DSattach_scale
098 00000000 UNDEF  notype       External     | __imp_H5DSdetach_scale
099 00000000 UNDEF  notype       External     | __imp_H5DSset_scale

but lack the __imp_ prefix in libhdf5_hl.lib.

(base) PS > dumpbin /symbols libhdf5_hl.lib | Select-String -Pattern H5DS

056 00000000 UNDEF  notype ()    External     | H5Dset_extent
04B 00000000 SECTF  notype ()    External     | H5DSwith_new_ref
04C 00000000 SECT5  notype ()    External     | H5DSattach_scale
04D 00000000 SECT6  notype ()    External     | H5DSdetach_scale
04E 00000000 SECTE  notype ()    External     | H5DSset_scale
04F 00000000 SECT8  notype ()    External     | H5DSget_num_scales
050 00000000 SECTD  notype ()    External     | H5DSset_label
051 00000000 SECT7  notype ()    External     | H5DSget_label
052 00000000 SECT9  notype ()    External     | H5DSget_scale_name
053 00000000 SECTB  notype ()    External     | H5DSis_scale
054 00000000 SECTC  notype ()    External     | H5DSiterate_scales
055 00000000 SECTA  notype ()    External     | H5DSis_attached
05B 00000000 SECT4  notype ()    Static       | H5DS_is_reserved

Is there something I'm doing obviously wrong? Does anyone have a working example of building + using NetCDF on Windows using Powershell + CMake + MSVC only (trying to avoid MINGW, etc.)

Linker errors

netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSget_num_scales referenced in function get_attached_info
netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSget_scale_name referenced in function read_scale
netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSis_scale referenced in function read_hdf5_obj
netcdf.lib(hdf5open.obj) : error LNK2019: unresolved external symbol __imp_H5DSiterate_scales referenced in function get_attached_info
netcdf.lib(nc4info.obj) : error LNK2019: unresolved external symbol __imp_H5Tset_cset referenced in function NC4_write_ncproperties
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5Tset_cset
netcdf.lib(nc4info.obj) : error LNK2019: unresolved external symbol __imp_H5Tset_strpad referenced in function NC4_write_ncproperties
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5Tset_strpad
netcdf.lib(nc4info.obj) : error LNK2019: unresolved external symbol __imp_H5Acreate1 referenced in function NC4_write_ncproperties
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5Acreate1
netcdf.lib(hdf5internal.obj) : error LNK2019: unresolved external symbol __imp_H5DSdetach_scale referenced in function nc4_reform_coord_var
netcdf.lib(nc4hdf.obj) : error LNK2001: unresolved external symbol __imp_H5DSdetach_scale
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Tcommit1 referenced in function commit_type
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Aget_num_attrs referenced in function NC4_walk
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Aopen_idx referenced in function NC4_walk
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Dopen1 referenced in function NC4_walk
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5Pget_version referenced in function NC4_hdf5get_superblock
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5DSattach_scale referenced in function nc4_rec_write_metadata
netcdf.lib(nc4hdf.obj) : error LNK2019: unresolved external symbol __imp_H5DSset_scale referenced in function nc4_create_dim_wo_var
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLappend referenced in function NC4_hdf5_plugin_path_set
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLremove referenced in function NC4_hdf5_plugin_path_set
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLget referenced in function NC4_hdf5_plugin_path_get
netcdf.lib(hdf5plugins.obj) : error LNK2019: unresolved external symbol __imp_H5PLsize referenced in function NC4_hdf5_plugin_path_get

Script I used to build NetCDF + dependencies as static libraries

# Specify installation path
$Env:INSTALL_PATH="C:/Users/username/Documents/install"
#
#
# libaec
#
curl https://gitlab.dkrz.de/k202009/libaec/-/archive/v1.1.3/libaec-v1.1.3.zip -o libaec.zip
Expand-Archive .\libaec.zip
cd libaec
cd libaec-v1.1.3
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DBUILD_TESTING=OFF ..
cmake --build . --target install --config Release
cd ../../..
#
# zlib
#
curl https://github.com/madler/zlib/archive/refs/tags/v1.3.1.zip -o zlib.zip
Expand-Archive zlib.zip
cd zlib
cd zlib-1.3.1
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DZLIB_BUILD_EXAMPLES=OFF .. 
cmake --build . --target install --config Release
cd ../../..
#
# HDF5
#
curl https://github.com/HDFGroup/hdf5/archive/refs/tags/hdf5_1.14.6.zip -o hdf5.zip
Expand-Archive .\hdf5.zip
cd hdf5
cd hdf5-hdf5_1.14.6
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DBUILD_TESTING=OFF -DHDF5_BUILD_EXAMPLES=OFF -DHDF5_BUILD_UTILS=OFF -DHDF5_BUILD_TOOLS=OFF  -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON -DSZIP_LIBRARY="$Env:INSTALL_PATH"/lib/szip-static.lib -DSZIP_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DZLIB_LIBRARY="$Env:INSTALL_PATH"/lib/zlibstatic.lib -DZLIB_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DHDF5_USE_LIBAEC_STATIC=ON -Dlibaec_DIR="$Env:INSTALL_PATH"/cmake -Dlibaec_INCLUDE_DIR="$Env:INSTALL_PATH"/include -Dlibaec_LIBRARY="$Env:INSTALL_PATH"/lib/aec-static.lib ..
cmake --build . --target install --config Release
cd ../../..
#
# Netcdf
#
curl https://github.com/Unidata/netcdf-c/archive/refs/tags/v4.9.3.zip -o netcdf.zip
Expand-Archive .\netcdf.zip
cd netcdf
cd netcdf-c-4.9.3
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$Env:INSTALL_PATH" -DBUILD_SHARED_LIBS=OFF -DBUILD_TESTING=OFF -DBUILD_TESTSETS=OFF -DNETCDF_ENABLE_TESTS=OFF -DNETCDF_ENABLE_UNIT_TESTS=OFF -DNETCDF_ENABLE_FILTER_TESTING=OFF -DNETCDF_ENABLE_FILTER_ZSTD=OFF -DNETCDF_ENABLE_FILTER_BZ2=OFF -DNETCDF_ENABLE_DAP=OFF -DNETCDF_ENABLE_BYTERANGE=OFF -DNETCDF_ENABLE_NCZARR=OFF -DNETCDF_ENABLE_LIBXML2=OFF -DNETCDF_ENABLE_NCZARR_FILTERS=OFF -DNETCDF_ENABLE_V2_API=OFF -DNETCDF_ENABLE_LIB=OFF -DNETCDF_ENABLE_BASH_SCRIPT_TESTING=OFF -DNETCDF_BUILD_UTILITIES=OFF -DHDF5_DIR="$Env:INSTALL_PATH"/cmake -DHDF5_C_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DHDF5_hdf5_LIBRARY_RELEASE="$Env:INSTALL_PATH"/lib/libhdf5.lib -DHDF5_hdf5_hl_LIBRARY_RELEASE="$Env:INSTALL_PATH"/lib/libhdf5_hl.lib -DZLIB_LIBRARY="$Env:INSTALL_PATH"/lib/zlibstatic.lib -DZLIB_INCLUDE_DIR="$Env:INSTALL_PATH"/include -DSzip_DEBUG_LIBRARY="$Env:INSTALL_PATH"/lib/szip-static.lib -DSzip_RELEASE_LIBRARY="$Env:INSTALL_PATH"/lib/szip-static.lib -DSzip_INCLUDE_DIRS="$Env:INSTALL_PATH"/include ..
cmake --build . --target install --config Release
cd ../../..

[andrewcorrigan]

The issue is that NetCDF is implicitly defining H5_BUILT_AS_DYNAMIC_LIB due to the commented out code here https://github.com/Unidata/netcdf-c/blob/main/cmake/dependencies.cmake#L126

• Can that code be uncommented? As it is now, netcdf includes hdf5 incorrectly.

Once I did that e9325d5, it detected hdf5-static, but I hit the following error

-- Checking for HDF5 config header
-- Looking for H5public.h
-- Looking for H5public.h - found
-- Using H5public.h to check for feature macros
-- Checking for H5_HAVE_ZLIB_H
-- Performing Test _has_H5_HAVE_ZLIB_H
-- Performing Test _has_H5_HAVE_ZLIB_H - Success
-- HDF5 has zlib.
-- Checking for H5_HAVE_FILTER_SZIP
-- Performing Test _has_H5_HAVE_FILTER_SZIP
-- Performing Test _has_H5_HAVE_FILTER_SZIP - Success
-- -- HDF5_UTF8_PATHS (HDF5 version 1.10.6+): ON
-- Looking for H5Pset_all_coll_metadata_ops
CMake Error at C:/netcdf/netcdf-c-4.9.3/build/CMakeFiles/CMakeScratch/TryCompile-537ruh/cmTC_639b9Targets.cmake:31 (set_target_properties):
  The link interface of target "hdf5-static" contains:

    ZLIB::ZLIB

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

I believe that this is an HDF5 issue, since I also saw the same thing with CGNS CGNS/CGNS#834. The difference is that here is that there's already a call to find_package(ZLIB).

I then saw that netcdf bundles its own FindZLIB.cmake, which doesn't define ZLIB::ZLIB, so I deleted that file from cmake/modules e9325d5 and upon reconfiguring, the ZLIB::ZLIB target was defined.

• Does NetCDF need its own custom version or can FindZLIB.cmake be removed? CMake has included support for ZLIB since version 3.1.

Finally, I had to add a call to find_package(libaec) 677e5ca, similar to CGNS/CGNS#847.

With that, I was able to build and install NetCDF statically on windows, and my application linked successfully as well.

[WardF]

Thank you for the report, and the sample script. I'll take a look at this, and see what we need to change on our end; removing the FindZLIB.cmake we bundle, and requiring the cmake version be 3.1+ shouldn't be a problem. I'm setting up to try to replicate this and then I'll make the changes to address this issue. Thanks!

Thank you for your patience!

[DennisHeimbigner]

I built the cmake/modules/FindZLIB.cmake and several other compression libraries.
This is quite a while ago, so it is possible that some of these FindXXX.cmake files
can now be deleted.