Homebrew Clang does not have correct search path for system include directory on MacOS Catalina

[ssbanerje]

Homebrew Clang does not search the correct include directories in MacOS Catalina.

What you were trying to do (and why)

If I try to compile the following C file after installing LLVM+Clang from homebrew

#include <stdio.h>

int main() {
    printf("hi");
}

What happened

I get the error

» which clang                                                                                                                                                                                             1 ↵
/usr/local/opt/llvm/bin/clang
» clang test.c
In file included from test.c:1:
/Library/Developer/CommandLineTools/usr/include/c++/v1/stdio.h:108:15: fatal error: 'stdio.h' file not found
#include <stdio.h>
              ^~~~~~~~~
1 error generated.

What you expected to happen

The program to compile correctly.

However, if I alias clang to the system clang, everything works!

» alias clang=/usr/bin/clang                                                                                                                                                                              1 ↵
» clang test.c
» ./a.out
hi%

By default the Homebrew clang has the following search paths

» echo | clang -E -Wp,-v -
clang -cc1 version 9.0.0 based upon LLVM 9.0.0 default target x86_64-apple-darwin19.0.0
ignoring nonexistent directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/local/Cellar/llvm/9.0.0/lib/clang/9.0.0/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 352 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

The system clang looks in different folders

» alias clang=/usr/bin/clang
» echo | clang -E -Wp,-v -
clang -cc1 version 11.0.0 (clang-1100.0.33.8) default target x86_64-apple-darwin19.0.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 362 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

I think the problem is that until recently /usr/include would have the header files that the compiler requires, however, that directory does not exist anymore (since Catalina update).

I can currently work around this by explicitly setting CPATH. But I think this is a bug, not sure if it is a homebrew problem or upstream in LLVM.

Step-by-step reproduction instructions (by running brew commands)

brew install llvm
export PATH="/usr/local/opt/llvm/bin:$PATH"
clang test.c

My homebrew is updated to the latest version and doctor does not report any problems.

» brew -v                                                                                                                                                                                                 1 ↵
Homebrew 2.1.13
Homebrew/homebrew-core (git revision 4d523; last commit 2019-10-09)
Homebrew/homebrew-cask (git revision c4f9d3; last commit 2019-10-09)

» brew doctor
Your system is ready to brew.

» brew config
HOMEBREW_VERSION: 2.1.13
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: aa043c1d678afd26f50156a83417c296cb9d5a4b
Last commit: 24 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 4d5233f01f792cb83661ccc1a6abfb139c168951
Core tap last commit: 4 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_VISUAL: nvim
CPU: quad-core 64-bit broadwell
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
Clang: 11.0 build 1100
Git: 2.23.0 => /usr/local/bin/git
Curl: 7.64.1 => /usr/bin/curl
Java: 12.0.1, 1.8.0_131, 1.8.0_40
macOS: 10.15-x86_64
CLT: 11.0.0.0.1.1567737322
Xcode: 11.1
CLT headers: 11.0.0.0.1.1567737322
XQuartz: 2.7.11 => /opt/X11

[jamiesnape]

There is a CMake variable C_INCLUDE_DIRS
https://github.com/llvm/llvm-project/blob/c0da1282fc036908cc721ee74f574fbb99d5e506/clang/CMakeLists.txt#L225-L226
that maybe could be set.

[fxcoudert]

I think @mistydemeo was looking at that issue

[tschoonj]

What I did to fix this is pass -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk to CPPFLAGS...

[sramsay]

@tschoonj After days of trying to figure this out, setting CPPFLAGS finally did it.

I would suggest that appending this to the LLVM build would stifle the sound of people crying all over the world.

[fxcoudert]

The problem with -isysroot, I think, is that it then excludes some paths from its header search. Like /usr/local/include.

[smorimoto]

I opened a PR. and I can't build it in my environment... (don't have enough spec) So if you want to try it, please checkout the branch.

[smorimoto]

I was finally able to build it. and it seems to be working correctly.

[fxcoudert]

With the compiler from this pull request, can you take a simple C file (one line #include <stdio.h> will do), run clang -v a.c, and paste the output here?

With an unpatched llvm, I get:

ignoring nonexistent directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/local/Cellar/llvm/9.0.0/lib/clang/9.0.0/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.

[dulvinw]

The issue is still there? I still get the error after completely uninstalling llvm, updating brew, reinstalling after deleting the install cache.

[smorimoto]

@fxcoudert I added comment on #45304.

[smorimoto]

@dulvinw Yes.

[LeadroyaL]

Since Command-Line-Tools >= 10.14, there is no /usr/include path. And after macOS Catalina, '/' partition is read-only. So I finally fix it by my self.

I diff the /usr/include for 10.13 and /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include for 10.15. The only difference is that:

• Most of them are the same.
• 10.15 has more header files, maybe some new feature.
• 10.13 has c++ directory, which is /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++ in macOS 10.15.

So do the following things:

1. add c++ headers
   sudo ln -s /Library/Developer/CommandLineTools/usr/include/c++ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++

2. reboot to recovery and disable SIP

3. reboot to desktop,

sudo remount -uw /

sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include /usr/include

4. reboot to recovery and enable SIP.

5. result:

➜  /usr lsa include
lrwxr-xr-x  1 root  wheel    63B 10 16 14:31 include -> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include

➜  /tmp clang -v test.c
clang version 8.0.0 (tags/RELEASE_800/final)
Target: x86_64-apple-darwin19.0.0
Thread model: posix
........
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Users/leadroyal/pllvm/r/lib/clang/8.0.0/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.

[smorimoto]

Do NOT disable SIP even if it's temporary. and it's not good solution for many people who work for companies. Some not bad solutions are already listed #45304.