Dune package management: submodules

[giltho]

Expected Behavior

Dune should behave identically to when handling pinned packages from git that have submodules.
In particular, opam will always recursively initialise submodules when downloading pin packages from git.

Actual Behavior

Pinned dependencies submodules are not recursively initialised.

Reproduction

• install the dune developer preview
• Checkout branch dune-issue-11606 of this repo.
• run dune pkg lock && dune build

Also, see this github action run

Output is:

File "vendor/jsonrpc/dune", line 1, characters 12-71:
1 | (copy_files %{project_root}/submodules/ocaml-lsp/jsonrpc/src/*.{ml,mli})
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Cannot find directory: submodules/ocaml-lsp/jsonrpc/src
File "vendor/lsp/dune", line 1, characters 12-71:
1 | (copy_files %{project_root}/submodules/ocaml-lsp/lsp/src/*.{ml,mli,mll})
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Cannot find directory: submodules/ocaml-lsp/lsp/src

This stems from the fact that I'm using the version of linol from this PR, which has ocaml-lsp as a submodule.

Specifications

• Version of dune (output of dune --version): "Dune Developer Preview: build 2025-04-03T07:55:38Z, git revision cf611d3"
• Version of ocaml (output of ocamlc --version): 5.3.0
• Operating system (distribution and version): macOS Sequoia 15.3.2

Additional information

Verbose output (dune build --verbose):

~/.local/bin/dune build --verbose
Shared cache: enabled-except-user-rules
Shared cache location: /Users/sacha/.cache/dune/db
Workspace root: /Users/sacha/Prog/bfa-ocaml
Auto-detected concurrency: 16
Dune context:
{ name = "default"
; kind = lock { default = true }
; profile = Dev
; merlin = true
; fdo_target_exe = None
; build_dir = In_build_dir "default"
; instrument_with = []
}
Actual targets:

• alias @@default
  Building ocaml-config.3
  Building linol.dev
  Running[1]: (cd _build/_private/default/.pkg/linol/source && /Users/sacha/Prog/bfa-ocaml/_opam/bin/dune build -p linol)
  Context: _private
  Command [1] exited with code 1:
  $ (cd _build/_private/default/.pkg/linol/source && /Users/sacha/Prog/bfa-ocaml/_opam/bin/dune build -p linol)
  File "vendor/jsonrpc/dune", line 1, characters 12-71:
  1 | (copy_files %{project_root}/submodules/ocaml-lsp/jsonrpc/src/.{ml,mli})
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Error: Cannot find directory: submodules/ocaml-lsp/jsonrpc/src
  File "vendor/lsp/dune", line 1, characters 12-71:
  1 | (copy_files %{project_root}/submodules/ocaml-lsp/lsp/src/.{ml,mli,mll})
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Error: Cannot find directory: submodules/ocaml-lsp/lsp/src

[giltho]

Hello, just confirming that upgrading to this commit, which uses git subtree instead of git submodules allowed the package to be successfully installed

[mdales]

I'd have to investigate using subtrees for Claudius because submodules are not supported/working. However, I find this a suboptimal solution for my particular use case, and would much rather see submodules work (see the PR above this comment).

Claudius is a graphics library for OCaml, and part of the build for that involves taking an open licensed font from GitHub, compiling it with crunch to be inline, and then linking that into Claudius. This way there is a font built into the graphics library which is extremely useful.

By adding the font as a subtree, we end up with the font's history mixed in with the graphics library history - this is not what I want: the font is just a resource that is added, not something that is otherwise meaningful to trace in this context. Submodules provide the correct mechnism to let me both have git fetch the resource and yet not track history.

But because of this limitation of dune not supporting submodules, I've pragmatically had to accept that we'll need to pollute our project history by using subtrees. It'd be great if this feature could be supported, as I anticipate in the future adding more resources to Claudius this way, and having each one pull in that project's history will get messy.