bug: error on init

[CXBoyy]

When initializing MatrixDepot, there are two errors appearing. I have tried removing and adding the package again without any success.

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.1 (2024-10-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using MatrixDepot
[ Info: verify download of index files...
[ Info: reading database
EOFError()
┌ Warning: recreating database file
└ @ MatrixDepot ~/.julia/packages/MatrixDepot/4S7Oa/src/download.jl:59
[ Info: reading index files
[ Info: adding metadata...
[ Info: adding svd data...
[ Info: writing database
┌ Warning: exception during initialization: 'KeyError(MatrixDepot)'
└ @ MatrixDepot ~/.julia/packages/MatrixDepot/4S7Oa/src/MatrixDepot.jl:125

julia> MatrixDepot.init()
[ Info: verify download of index files...
[ Info: reading database
EOFError()
┌ Warning: recreating database file
└ @ MatrixDepot ~/.julia/packages/MatrixDepot/4S7Oa/src/download.jl:59
[ Info: reading index files
[ Info: adding metadata...
[ Info: adding svd data...
[ Info: writing database
[ Info: used remote sites are sparse.tamu.edu with MAT index and math.nist.gov with HTML index

When running test MatrixDepot I get the following output:

julia> Pkg.test("MatrixDepot")
     Testing MatrixDepot
      Status `/tmp/jl_lnJgj8/Project.toml`
  [79a69506] ChannelBuffers v0.4.2
  [a93c6f00] DataFrames v1.7.0
  [23992714] MAT v0.10.7
  [b51810bb] MatrixDepot v1.0.13
  [6c6a2e73] Scratch v1.2.1
  [37e2e46d] LinearAlgebra v1.11.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [a63ad114] Mmap v1.11.0
  [9e88b42a] Serialization v1.11.0
  [2f01184e] SparseArrays v1.11.0
  [8dfed614] Test v1.11.0
      Status `/tmp/jl_lnJgj8/Manifest.toml`
  [e1450e63] BufferedStreams v1.2.2
  [79a69506] ChannelBuffers v0.4.2
  [944b1d66] CodecZlib v0.7.6
  [34da2185] Compat v4.16.0
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.16.0
  [a93c6f00] DataFrames v1.7.0
  [864edb3b] DataStructures v0.18.20
  [e2d170a0] DataValueInterfaces v1.0.0
  [f67ccb44] HDF5 v0.17.2
  [842dd82b] InlineStrings v1.4.2
  [41ab1584] InvertedIndices v1.3.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.6.1
  [b964fa9f] LaTeXStrings v1.4.0
  [23992714] MAT v0.10.7
  [3da0fdf6] MPIPreferences v0.1.11
  [b51810bb] MatrixDepot v1.0.13
  [e1d29d7a] Missings v1.2.0
  [bac558e1] OrderedCollections v1.6.3
  [2dfb63ee] PooledArrays v1.4.3
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [08abe8d2] PrettyTables v2.4.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [6c6a2e73] Scratch v1.2.1
  [91c51154] SentinelArrays v1.4.7
  [a2af1166] SortingAlgorithms v1.2.1
  [10745b16] Statistics v1.11.1
  [892a3eda] StringManipulation v0.4.0
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [3bb67fe8] TranscodingStreams v0.11.3
  [0234f1f7] HDF5_jll v1.14.3+3
  [e33a78d0] Hwloc_jll v2.11.2+1
  [7cb0a576] MPICH_jll v4.2.3+0
  [f1f71cc9] MPItrampoline_jll v5.5.1+0
  [9237b28f] MicrosoftMPI_jll v10.1.4+2
⌅ [fe0851c0] OpenMPI_jll v4.1.6+0
  [458c3c95] OpenSSL_jll v3.0.15+1
  [477f73a3] libaec_jll v1.1.2+0
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching v1.11.0
  [9fa8497b] Future v1.11.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.11.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [a63ad114] Mmap v1.11.0
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.11.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [2f01184e] SparseArrays v1.11.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test v1.11.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] LibCURL_jll v8.6.0+0
  [e37daf67] LibGit2_jll v1.7.2+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.6+0
  [14a3606d] MozillaCACerts_jll v2023.12.12
  [4536629a] OpenBLAS_jll v0.3.27+1
  [bea87d4a] SuiteSparse_jll v7.7.0+0
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.11.0+0
  [8e850ede] nghttp2_jll v1.59.0+0
  [3f19e933] p7zip_jll v17.4.0+2
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading.
     Testing Running tests...
[ Info: verify download of index files...
[ Info: creating database file
[ Info: reading index files
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/files/ss_index.mat
[ Info: downloading index file file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/MatrixMarket/matrices.html
[ Info: adding metadata...
[ Info: adding svd data...
[ Info: writing database
┌ Warning: exception during initialization: 'KeyError(MatrixDepot)'
└ @ MatrixDepot ~/.julia/packages/MatrixDepot/4S7Oa/src/MatrixDepot.jl:125
[ Info: verify download of index files...
[ Info: reading database
EOFError()
┌ Warning: recreating database file
└ @ MatrixDepot ~/.julia/packages/MatrixDepot/4S7Oa/src/download.jl:59
[ Info: reading index files
[ Info: adding metadata...
[ Info: adding svd data...
[ Info: writing database
[ Info: used remote sites are sparse.tamu.edu with MAT index and math.nist.gov with HTML index
'magic' passed test...
'cauchy' passed test...
'circul' passed test...
'hadamard' passed test...
'hilb' passed test...
'dingdong' passed test...
'frank' passed test...
'invhilb' passed test...
'forsythe' passed test...
'grcar' passed test...
'triw' passed test...
'moler' passed test...
'pascal' passed test...
'kahan' passed test...
'pei' passed test...
'vand' passed test...
'invol' passed test...
'chebspec' passed test...
'lotkin' passed test...
'clement' passed test...
'fiedler' passed test...
'minij' passed test...
'binomial' passed test...
'tridiag' passed test...
'lehmer' passed test...
'parter' passed test...
'chow' passed test...
'randcorr' passed test...
'poisson' passed test...
'neumann' passed test...
'rosser' passed test...
'sampling' passed test...
'wilkinson' passed test...
'rando' passed test...
'randsvd' passed test...
'rohess' passed test...
'kms' passed test...
'wathen' passed test...
'oscillate' passed test...
'toeplitz' passed test...
'prolate' passed test...
'hankel' passed test...
'golub' passed test...
'companion' passed test...
'erdrey' passed test...
'gilbert' passed test ...
'erdrey' passed test...
MatrixDepot.jl: Error During Test at /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/include_generator.jl:33
  Test threw exception
  Expression: mdinfo("randsym") == Base.Docs.doc(randsym)
  MethodError: no method matching doc(::typeof(randsym))
  The function `doc` exists, but no method is defined for this combination of argument types.
  Stacktrace:
   [1] macro expansion
     @ ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Test/src/Test.jl:676 [inlined]
   [2] top-level scope
     @ ~/.julia/packages/MatrixDepot/4S7Oa/test/include_generator.jl:515
running /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/download.jl ...
[ Info: creating database file
[ Info: reading index files
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/files/ss_index.mat
[ Info: downloading index file file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/MatrixMarket/matrices.html
[ Info: adding metadata...
[ Info: adding svd data...
[ Info: writing database
[ Info: downloading head of file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/HB/1138_bus.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/HB/1138_bus.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/pub/MatrixMarket2/Harwell-Boeing/psadmit/1138_bus.mtx.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/Pajek/Journals.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/pub/MatrixMarket2/Harwell-Boeing/smtape/bp___200.mtx.gz
[ Info: downloading head of file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/pub/MatrixMarket2/Harwell-Boeing/psadmit/662_bus.mtx.gz
[ Info: downloading head of file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/Bates/Chem97Zt.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/Bates/Chem97Zt.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/Bates/Chem97ZtZ.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/Averous/epb0.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/svd/HB/1138_bus_SVD.mat
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/pub/MatrixMarket2/Harwell-Boeing/lanpro/nos5.mtx.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/Bai/dwg961b.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/pub/MatrixMarket2/Harwell-Boeing/cegb/cegb2802.mtx.gz
┌ Warning: download of file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/math.nist.gov/pub/MatrixMarket2/Harwell-Boeing/cegb/cegb2802.mtx.gz failed: TaskFailedException(Task (failed) @0x00007af97dd66590)
└ @ MatrixDepot ~/.julia/packages/MatrixDepot/4S7Oa/src/download.jl:142
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/DRIVCAV/cavity14.tar.gz
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/AG-Monien/bfly.tar.gz
finished /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/download.jl
running /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/common.jl ...
[ Info: downloading: file:///home/cx/.julia/packages/MatrixDepot/4S7Oa/test/data/sparse.tamu.edu/MM/HB/well1033.tar.gz
finished /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/common.jl
running /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/number.jl ...
finished /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/number.jl
running /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/property.jl ...
finished /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/property.jl
running /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/downloadmm.jl ...
finished /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/downloadmm.jl
running remote.jl ...
remote tests not executed
finished remote.jl
Test Summary:                              | Pass  Error  Total     Time
MatrixDepot.jl                             |  677      1    678  1m53.2s
  MatrixDepot generator tests              |  382           382    46.7s
  regularization methods                   |   44            44    23.4s
  MatrixDepot simulate remote matrix tests |  238           238    38.4s
  MatrixDepot real remote matrix tests     |                  0     0.0s
ERROR: LoadError: Some tests did not pass: 677 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/cx/.julia/packages/MatrixDepot/4S7Oa/test/runtests.jl:16
ERROR: Package MatrixDepot errored during testing
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Types.jl:68
 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{…}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
   @ Pkg.Operations ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:2104
 [3] test
   @ ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:1989 [inlined]
 [4] test(ctx::Pkg.Types.Context, pkgs::Vector{…}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{…})
   @ Pkg.API ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:475
 [5] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{})
   @ Pkg.API ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:159
 [6] test(pkgs::Vector{Pkg.Types.PackageSpec})
   @ Pkg.API ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:148
 [7] test
   @ ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:147 [inlined]
 [8] test(pkg::String)
   @ Pkg.API ~/.julia/juliaup/julia-1.11.1+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:146
 [9] top-level scope
   @ REPL[4]:1
Some type information was truncated. Use `show(err)` to see complete types.

Is this a bug or am I doing something wrong?

[KlausC]

This is a compatibility issue, which appears in 1.11. Works as expected in 1.10.6 (lts)

[PatCsudo]

I am having the same issue. I set up a new environment and ran add FundamentalsNumericalComputation as suggested at

https://github.com/fncbook/FundamentalsNumericalComputation.jl

I received the following warning:

┌ Warning: exception during initialization: 'KeyError(MatrixDepot)'
│ └ @ MatrixDepot ~/.julia/packages/MatrixDepot/4S7Oa/src/MatrixDepot.jl:125

I am running 1.11. Attached is the output for add FundamentalsNumericalComputation
FNC_installation.txt

[KlausC]

You can (for the moment) ignore the warning; it is just not possible to write a cache of the internal database. Maybe it is necessary to remove the file MatrixDepot.dbpath(MatrixDepot.MATRIX_DB) before the next run.

[PatCsudo]

Okay. Thanks. My plan was to maybe let just let it be. It is not like I need it straight away.

[CXBoyy]

This is a compatibility issue, which appears in 1.11. Works as expected in 1.10.6 (lts)

Okay, will use Julia 1.10.6 in the mean time then. Will there be a fix for this issue for Julia 1.11 in the future? @KlausC

[KlausC]

Will there be a fix for this issue for Julia 1.11 in the future?

I am working on it.

[willow-ahrens]

This bug appears differently on nightly builds, I think it is related to this PR: JuliaLang/julia@9dbdeb4

[willow-ahrens]

I've filed an issue with julia for this: JuliaLang/julia#56916

[willow-ahrens]

A quick fix for this would be to define a custom serialization to JSON for the MatrixDatabase and MatrixData types.

[jpsamaroo]

An even better alternative might be to do this work on first call to matrixdepot and other functions which access this data. This would reduce how long precompile/package loading takes to finish, which is a win if packages want to depend directly on MatrixDepot.

[willow-ahrens]

In general, is MatrixDepot overengineered? The matrix keys correspond to filenames, the core functionality of the package could run entirely on "isfile", "curl", and "mmread", without the need for a centralized database, right?