Skip to content

Clangd support and clarification on CMake structure #2408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
amitrahman1026 opened this issue Oct 8, 2024 · 3 comments
Closed

Clangd support and clarification on CMake structure #2408

amitrahman1026 opened this issue Oct 8, 2024 · 3 comments

Comments

@amitrahman1026
Copy link
Contributor

amitrahman1026 commented Oct 8, 2024

Hi! Firstly, thanks for your amazing work!

I am trying to get clangd to play nice, and hoping to get some support.

So I'm generated the compile_commands.json file for clangd to use as a compilation database. No further

mkdir build
cd build
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
..
ln -s build/compile_commands.json ./compile_commands.json

One strange thing I notice is that the symbol definitions are incomplete. For e.g. when I'm working on the silkworm/silkworm/sentry/sentry.hpp file.

and I try to go to the definition of Settings for instance in the snippet

#include "settings.hpp"
// ...
class SentryImpl final {
  public:
    explicit SentryImpl(Settings settings, concurrency::ExecutorPool& executor_pool);

Clangd only traces back up to the #include "settings.hpp" directive and stops there, unable to jump further.

This leads me to suspect that the lsp does not have enough information about the includes.

This led me to poke around in the generated compile_commands.json to see the (truncated) node generated for the file and I notice -I/Users/amit/Cpp_Projects/silkworm is the only include that is relative.

{
  "directory": "/Users/amit/Cpp_Projects/silkworm/build/silkworm/sentry",
  "command": "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DAGRPC_BOOST_ASIO -DBOOST_ASIO_NO_DEPRECATED -DCARES_STATICLIB -DECMULT_GEN_PREC_BITS=4 -DECMULT_WINDOW_SIZE=15 -DENABLE_MODULE_ECDH -DENABLE_MODULE_RECOVERY -DSILKWORM_CORE_USE_ABSEIL -DSPDLOG_COMPILED_LIB -DSPDLOG_FMT_EXTERNAL -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS 
-I/Users/amit/Cpp_Projects/silkworm 
-I/Users/amit/Cpp_Projects/silkworm/third_party/ethash/ethash/include 
// more third_party stuff
-I/Users/amit/Cpp_Projects/silkworm/third_party/cpp-base64/cpp-base64 -isystem /Users/amit/.conan/data/boost/1.83.0/_/_/package/7b157ba7a68c0b8018fa90ec764a3f9052b441c2/include 
// .. more conan stuff 
 -O3 -DNDEBUG -std=c++20 -arch arm64 
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk 
-mmacosx-version-min=14.0 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden 
-stdlib=libc++ -Werror -Wall -Wextra -pedantic -Wshadow -Wimplicit-fallthrough -Wunused -Wsign-compare -Wsign-conversion -Wdouble-promotion -Wold-style-cast -Wnon-virtual-dtor -Woverloaded-virtual -Wtype-limits -Wformat=2 -Wno-missing-field-initializers -Wconversion -Wthread-safety 
-o CMakeFiles/silkworm_sentry.dir/sentry.cpp.o 
-c /Users/amit/Cpp_Projects/silkworm/silkworm/sentry/sentry.cpp",
  "file": "/Users/amit/Cpp_Projects/silkworm/silkworm/sentry/sentry.cpp",
  "output": "silkworm/sentry/CMakeFiles/silkworm_sentry.dir/sentry.cpp.o"
},

Now I understand that relative include directives like #include "settings.hpp" were used and this results in compilation working just fine, so I was hypothesizing that without the flag -I/Users/amit/Cpp_Projects/silkworm/silkworm/{include dir}, clangd would not be able to get the full path of definition.

Now I've tried a few permutations of where to include, below:

diff --git a/cmake/common/targets.cmake b/cmake/common/targets.cmake
index cfaa91a9..d966c02b 100644
--- a/cmake/common/targets.cmake
+++ b/cmake/common/targets.cmake
@@ -54,7 +54,7 @@ function(silkworm_library TARGET)
   list(FILTER SRC EXCLUDE REGEX "_benchmark\\.cpp$")
   add_library(${TARGET} ${ARG_TYPE} ${SRC})
diff --git a/silkworm/sentry/CMakeLists.txt b/silkworm/sentry/CMakeLists.txt
index 8c1816f6..8b152ed3 100644
--- a/silkworm/sentry/CMakeLists.txt
+++ b/silkworm/sentry/CMakeLists.txt
@@ -15,6 +15,7 @@
 ]]
 
 include("${SILKWORM_MAIN_DIR}/cmake/common/targets.cmake")
+include_directories("${SILKWORM_MAIN_DIR}/silkworm/sentry")
 
 find_package(absl REQUIRED)
 find_package(Boost REQUIRED headers)

They do add the desired -I/Users/amit/Cpp_Projects/silkworm/silkworm/sentry flag into the compile_commands.json however its added to ALL of the nodes.

Ultimately, I'm seeking some guidance on whether I'm missing something out and how I should approach this problem. While this is not critical maybe this has to do with the way the CMakeLists are handling include directories?

@battlmonstr
Copy link
Contributor

battlmonstr commented Oct 11, 2024

@amitrahman1026 thanks for this feedback. I need to do a research on this for a definite answer.

My current understanding is that relative includes are automatically found relative to the source file by the compiler therefore there's no need to pass extra -I flags. If it's not the case we'd have to add many more directories to -I.

The previous version of CLion was using clangd as its code understanding backend, and it was able to find these headers somehow so that they were navigatable in the IDE.

Could you say:

  1. why do you need to symlink compile_commands.json ? (that might break relative paths if any)
  2. what command do you use to run clangd and in which working directory?
  3. what's the purpose of clangd? is it running as a part of an IDE/editor plugin? in this case which IDE/editor do you use? (also which version of the IDE/editor and which plugin version)
  4. when you describe the problem:

Clangd only traces back up to the #include "settings.hpp" directive and stops there, unable to jump further.

How does it manifest? How do you try to "jump further"?
5. Is there any log diagnostics from clangd itself about this problem?

@amitrahman1026
Copy link
Contributor Author

amitrahman1026 commented Oct 11, 2024

Thanks for the response @battlmonstr, I'd been fiddling around further with clangd, and I am still in the process of investigating but it may draw to a close soon because I generally have it working and have found a similar lead here!

To respond to your questions,

  1. You're right there was not a need to symlink. Clangd does indeed look at /build path. I did that out of habit from projects where there were multiple subdirectories in the builds 😓
  2. My high level understanding is that nvim makes some rpc calls to whatever language server I provide.
  3. I was running as part of a neovim plugin via the manager mason.nvim. The alternative I had was using vscode's intellisense which was unable to locate things included from the conan files for example. This may be a configuration failure on my part but I did not check further.
  4. clangd is indeed the backend of my lsp support on my editor NVIM v0.10.0. Conceptually this should act the same as how CLion would have probably done for you.
	-- LSP
	{
		"neovim/nvim-lspconfig",
		cmd = { "LspInfo", "LspInstall", "LspStart" },
		event = { "BufReadPre", "BufNewFile" },
		dependencies = {
			{ "hrsh7th/cmp-nvim-lsp" },
			{ "williamboman/mason-lspconfig.nvim" },
		},
		config = function()
			local lsp_zero = require("lsp-zero")
			local lspconfig = require("lspconfig")
-- ... 
			-- Specific setup for clangd
			lspconfig.clangd.setup({
				cmd = { "/opt/homebrew/opt/llvm/bin/clangd" },
				capabilities = require("cmp_nvim_lsp").default_capabilities(),
				on_attach = lsp_attach,
			})
  1. Yup looking at the logs did indeed point me to the best leads, I'll share below.

Further in my investigations, I'd switched around between three different versions of clangd to see what was causing the issue.

The initial was with a version of a clangd binary (unconfirmed what it was at the time) pulled by mason. I was unable to reproduce the backtrace failure however with mason's version of clangd. and I had overwritten the mason.logs.

My first approaches to solve this lsp problem (for my setup because I understand there's probably not a real need for this for the majority of end users) was to throw more includes at it so put together some hacky recursive cmake just for compile_commands.json. Regardless that was a bandaid and not really a sustainable solution.

Then I explored switching to native Apple clangd version 15.0.0 which was ALMOST working fine, except it couldn't really find anything that had origins in third part libraries kind of like vscode's intellisense, which is workable but not the best experience.

Finally, I came around to the probably source of my problem! llvm/llvm-project#109367.

The bug report describes crashes due to certain clang-tidy checks, namely boost-use-ranges.

Here, I have reproduced the (truncated) logs when using Homebrew clangd version 19.1.1. I highly suspect that mason was also using a binary of clang in the 19.x range at the time hence also not letting me "jump" to the full definition due to crashes that I didn't realise at the time.

[START][2024-10-11 16:59:17] LSP logging initiated
[ERROR][2024-10-11 16:59:17] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:59:17.991] Homebrew clangd version 19.1.1\nI[16:59:17.992] Features: mac+xpc\nI[16:59:17.992] PID: 62437\nI[16:59:17.992] Working directory: /Users/amit/Cpp_Projects/silkworm\nI[16:59:17.992] argv[0]: /opt/homebrew/opt/llvm/bin/clangd\nI[16:59:17.992] Starting LSP over stdin/stdout\n"
[ERROR][2024-10-11 16:59:17] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:59:17.995] <-- initialize(1)\n"
[ERROR][2024-10-11 16:59:18] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:59:18.016] --> reply:initialize(1) 21 ms\n"
[ERROR][2024-10-11 16:59:18] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:59:18.022] <-- initialized\nI[16:59:18.022] <-- textDocument/didOpen\n"// ...
[ERROR][2024-10-11 16:42:45] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:45.807] --> reply:textDocument/definition(8) 1 ms\n"
[ERROR][2024-10-11 16:42:45] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:45.837] <-- textDocument/didOpen\n"
[ERROR][2024-10-11 16:42:45] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:45.838] ASTWorker building file /Users/amit/Cpp_Projects/silkworm/silkworm/infra/common/log.hpp version 0 with command inferred from /Users/amit/Cpp_Projects/silkworm/cmd/sentry.cpp\n[/Users/amit/Cpp_Projects/silkworm/build/cmd]\n/Applications/Xcode.a/

// TRUNCATED: Compile flags, unmodified from master branches's cmake.

Wconversion -Wthread-safety -c -resource-dir=/opt/homebrew/Cellar/llvm/19.1.1/lib/clang/19 -std=c++20 -- /Users/amit/Cpp_Projects/silkworm/silkworm/infra/common/log.hpp\n"
[ERROR][2024-10-11 16:42:46] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:45.840] <-- textDocument/semanticTokens/full(9)\n"
[ERROR][2024-10-11 16:42:46] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:46.525] Built preamble of size 14746684 for file /Users/amit/Cpp_Projects/silkworm/silkworm/infra/common/log.hpp version 0 in 0.69 seconds\n"
[ERROR][2024-10-11 16:42:46] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:46.525] --> workspace/semanticTokens/refresh(2)\n"
[ERROR][2024-10-11 16:42:46] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:46.527] <-- $/cancelRequest\nI[16:42:46.527] <-- textDocument/semanticTokens/full(10)\n"
[ERROR][2024-10-11 16:42:46] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:42:46.527] <-- reply(2)\n"
[ERROR][2024-10-11 16:42:46] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	
"PLEASE submit a bug report to https://github.com/Homebrew/homebrew-core/issues and include the crash backtrace.\n #0 0x0000000119474fe0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/homebrew/Cellar/llvm/19.1.1/lib/libLLVM.dylib+0x4a10fe0)\n #1 0x00000001194753b8 SignalHandler(int) (/opt/homebrew/Cellar/llvm/19.1.1/lib/libLLVM.dylib+0x4a113b8)\n #2 0x0000000194c53584 (/usr/lib/system/libsystem_platform.dylib+0x180477584)\n #3 0x000000010060953c clang::tidy::boost::UseRangesCheck::getReplacerMap() const::$_1::operator()(llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>, std::initializer_list<llvm::StringRef>) const 

// TRUNCATED: Backtrace of clang-tidy, similar to the one 

[ERROR][2024-10-11 16:43:57] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:43:57.741] <-- shutdown(4)\nI[16:43:57.741] --> reply:shutdown(4) 0 ms\n"
[ERROR][2024-10-11 16:43:57] .../vim/lsp/rpc.lua:770	"rpc"	"/opt/homebrew/opt/llvm/bin/clangd"	"stderr"	"I[16:43:

Fix-ish(?)

Using the suggested method to remove the problematic check for clangd by adding the following to a .clangd at the root directory. Of course this isn't a permanent fix but the performance on the lsp is extremely good now, way better than what intellisense/apple clangd is able to provide (inablility to locate third party libs for e.g.).

Diagnostics:
  ClangTidy:
    Remove: boost-use-ranges

I'll stay tuned to the fixes, and perhaps downgrade to llvm 18 in the meanwhile to retest as well.


I think we can close this! This may have been due to misconfigurations/ lack of updated language servers on my part, rather than missing include flags!

Appreciate you for taking some time to respond to this rather niche query :)

@battlmonstr
Copy link
Contributor

@amitrahman1026 I'm glad that you've found a workaround. Our recommended IDEs for silkworm are CLion or Xcode, but it is good to ensure that VSCode and NeoVim also work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants