cpp-httplib (ccache dependency) requires SSL_get1_peer_certificate

[orbea]

OS: Gentoo
LibreSSL: 3.9.2
cpp-httplib: 0.16.0

ccache added a new dependency of cpp-httplib which Gentoo installs system-wide and it fails during cmake.

CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
  system variable OPENSSL_ROOT_DIR: Found unsuitable version "2.0.0", but
  required is at least "3.0.0" (found /usr/lib/libcrypto.so, found
  components: Crypto SSL)
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:601 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake/Modules/FindOpenSSL.cmake:689 (find_package_handle_standard_args)
  CMakeLists.txt:121 (find_package)

https://github.com/yhirose/cpp-httplib/blob/v0.16.0/CMakeLists.txt#L82-L84

This is easy to workaround.

sed -i 's/"3.0.0"/"2.0.0"/' CMakeLists.txt

But then it fails because LibreSSL doesn't have SSL_get1_peer_certificate.

[1/3] Building CXX object CMakeFiles/httplib.dir/out/httplib.cc.o
samu: job failed with status 1: /usr/lib/ccache/bin/x86_64-pc-linux-musl-g++ -DCPPHTTPLIB_OPENSSL_SUPPORT -DCPPHTTPLIB_ZLIB_SUPPORT -Dhttplib_EXPORTS -isystem /var/tmp/portage/dev-cpp/cpp-httplib-0.16.0/work/cpp-httplib-0.16.0_build-abi_x86_64.amd64/out  -O2 -pipe -fPIC -MD -MT CMakeFiles/httplib.dir/out/httplib.cc.o -MF CMakeFiles/httplib.dir/out/httplib.cc.o.d -o CMakeFiles/httplib.dir/out/httplib.cc.o -c /var/tmp/portage/dev-cpp/cpp-httplib-0.16.0/work/cpp-httplib-0.16.0_build-abi_x86_64.amd64/out/httplib.cc
In file included from /var/tmp/portage/dev-cpp/cpp-httplib-0.16.0/work/cpp-httplib-0.16.0_build-abi_x86_64.amd64/out/httplib.cc:1:
/var/tmp/portage/dev-cpp/cpp-httplib-0.16.0/work/cpp-httplib-0.16.0_build-abi_x86_64.amd64/out/httplib.h:273:2: error: #error Sorry, OpenSSL versions prior to 3.0.0 are not supported
  273 | #error Sorry, OpenSSL versions prior to 3.0.0 are not supported
      |  ^~~~~
/var/tmp/portage/dev-cpp/cpp-httplib-0.16.0/work/cpp-httplib-0.16.0_build-abi_x86_64.amd64/out/httplib.cc: In lambda function:
/var/tmp/portage/dev-cpp/cpp-httplib-0.16.0/work/cpp-httplib-0.16.0_build-abi_x86_64.amd64/out/httplib.cc:6645:30: error: 'SSL_get1_peer_certificate' was not declared in this scope; did you mean 'SSL_get_peer_certificate'?
 6645 |           auto server_cert = SSL_get1_peer_certificate(ssl2);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~
      |                              SSL_get_peer_certificate
samu: subcommand failed

https://github.com/yhirose/cpp-httplib/blob/v0.16.0/httplib.h#L9032

Is it possible that LibreSSL can add SSL_get1_peer_certificate in the future? Or is there a better way to avoid this?

It seems in OpenBSD the ccache package uses the vendored copy of cpp-httplibwhich doesn't have this problem (Yet?).

[botovq]

https://github.com/yhirose/cpp-httplib/blob/v0.16.0/httplib.h#L9032 Is it possible that LibreSSL can add `SSL_get1_peer_certificate` in the future? Or is there a better way to avoid this?

It would be easy to add (but it's too late for the 4.0.0 release. We generally avoid adding OpenSSL 3 API. SSL_get1_peer_certificate is the OpenSSL 3 name for SSL_get_peer_certificate. Does this build if you pass

…

-DSSL_get1_peer_certificate=SSL_get_peer_certificate in CXXFLAGS? Or is there more missing?

[botovq]

It seems in OpenBSD the `ccache` package uses the vendored copy of `cpp-httplib`which doesn't have this problem (Yet?).

The OpenBSD port does use the bundled cpp-httplib version which appears to have these problems, but it seems it doesn't hit this issue because CPPHTTPLIB_OPENSSL_SUPPORT isn't set for whatever reason. As you can see in WANTLIB it doesn't link against `crypto` or `ssl`: https://github.com/openbsd/ports/blob/d80b8b5d70f0efcdc592035d0990787ccc04386c/devel/ccache/Makefile#L13

[botovq]

Since this is a header only library, CXX flags won't work. I would probably work around this this by adding something like #if defined(LIBRESSL_VERSION_NUMBER) #define SSL_get1_peer_certificate SSL_get_peer_certificate #endif and see if that's enough.

[orbea]

For building cpp-httplib itself it is enough to add -DSSL_get1_peer_certificate=SSL_get_peer_certificate to the CXXFLAGS, but the tests also fail.

 * abi_x86_64.amd64: running multilib-minimal_abi_src_test
make LIBTOOL=rlibtool -j4 -C test CXX=x86_64-pc-linux-musl-g++ 'CXXFLAGS=-O2 -pipe -DSSL_get1_peer_certificate=SSL_get_peer_certificate -I.' 
make: Entering directory '/var/tmp/portage/dev-cpp/cpp-httplib-0.16.0/work/cpp-httplib-0.16.0_build-abi_x86_64.amd64/test'
openssl genrsa 2048 > key.pem
python3 ../split.py -o .
Generating RSA private key, 2048 bit long modulus
......Wrote ./httplib.h and ./httplib.cc
..........................................
...............................
e is 65537 (0x010001)
openssl req -new -batch -config test.conf -key key.pem | openssl x509 -days 3650 -req -signkey key.pem > cert.pem
Signature ok
subject=/C=US/ST=Test State or Province/L=Test Locality/O=Organization Name/OU=Organizational Unit Name/CN=Common Name/emailAddress=test@email.address
openssl req -x509 -config test.conf -key key.pem -sha256 -days 3650 -nodes -out cert2.pem -extensions SAN
unable to load X509 request
140597270309672:error:09FFF06C:PEM routines:CRYPTO_internal:no start line:/var/tmp/portage/dev-libs/libressl-3.9.2/work/libressl-3.9.2/crypto/pem/pem_lib.c:699:Expecting: CERTIFICATE REQUEST
make: *** [Makefile:68: cert.pem] Error 1

[orbea]

This patch is enough to build httplib-cpp, but if possible it would be nice to have this working more out of the box in the future.

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -81,7 +81,7 @@ project(httplib
 
 # Change as needed to set an OpenSSL minimum version.
 # This is used in the installed Cmake config file.
-set(_HTTPLIB_OPENSSL_MIN_VER "3.0.0")
+set(_HTTPLIB_OPENSSL_MIN_VER "2.0.0")
 
 # Lets you disable C++ exception during CMake configure time.
 # The value is used in the install CMake config file.
--- a/httplib.h
+++ b/httplib.h
@@ -269,9 +269,13 @@ using socket_t = int;
 #include <iostream>
 #include <sstream>
 
+#ifdef LIBRESSL_VERSION_NUMBER
+#define SSL_get1_peer_certificate SSL_get_peer_certificate
+#else
 #if OPENSSL_VERSION_NUMBER < 0x30000000L
 #error Sorry, OpenSSL versions prior to 3.0.0 are not supported
 #endif
+#endif
 
 #endif
 

[botovq]

subject=/C=US/ST=Test State or Province/L=Test Locality/O=Organization Name/OU=Organizational Unit Name/CN=Common ***@***.*** openssl req -x509 -config test.conf -key key.pem -sha256 -days 3650 -nodes -out cert2.pem -extensions SAN unable to load X509 request

Not sure where that message comes from, but it smells like an incompatibility in the horrible openssl command line utility.

[orbea]

I should of checked more recent versions than the version marked as stable by Gentoo, with 0.16.2 also fails because OPENSSL_thread_stop is missing. Gentoo doesn't yet have any of the newer 0.17.x tags.

[1/3] Building CXX object CMakeFiles/httplib.dir/out/httplib.cc.o
samu: job failed with status 1: /usr/lib/ccache/bin/x86_64-pc-linux-musl-g++ -DCPPHTTPLIB_OPENSSL_SUPPORT -DCPPHTTPLIB_ZLIB_SUPPORT -Dhttplib_EXPORTS -isystem /var/tmp/portage/dev-cpp/cpp-httplib-0.16.3/work/cpp-httplib-0.16.3_build-abi_x86_64.amd64/out  -O2 -pipe -fPIC -MD -MT CMakeFiles/httplib.dir/out/httplib.cc.o -MF CMakeFiles/httplib.dir/out/httplib.cc.o.d -o CMakeFiles/httplib.dir/out/httplib.cc.o -c /var/tmp/portage/dev-cpp/cpp-httplib-0.16.3/work/cpp-httplib-0.16.3_build-abi_x86_64.amd64/out/httplib.cc
In file included from /var/tmp/portage/dev-cpp/cpp-httplib-0.16.3/work/cpp-httplib-0.16.3_build-abi_x86_64.amd64/out/httplib.cc:1:
/var/tmp/portage/dev-cpp/cpp-httplib-0.16.3/work/cpp-httplib-0.16.3_build-abi_x86_64.amd64/out/httplib.h: In member function 'void httplib::ThreadPool::worker::operator()()':
/var/tmp/portage/dev-cpp/cpp-httplib-0.16.3/work/cpp-httplib-0.16.3_build-abi_x86_64.amd64/out/httplib.h:736:7: error: 'OPENSSL_thread_stop' was not declared in this scope
  736 |       OPENSSL_thread_stop();
      |       ^~~~~~~~~~

Note that with more recent versions they are using the previous solution for BoringSSL already.

https://github.com/yhirose/cpp-httplib/blob/80fb03628bb57ca9d3ab855a7feec2876249bb61/httplib.h#L280

The OpenBSD port does use the bundled cpp-httplib version which appears to have these problems, but it seems it doesn't hit this issue because CPPHTTPLIB_OPENSSL_SUPPORT isn't set for whatever reason.

I suppose for ccache its not required to use SSL.

[botovq]

On Fri, Sep 06, 2024 at 09:30:16AM -0700, orbea wrote: This patch is enough to build httplib-cpp, but if possible it would be nice to have this working more out of the box in the future.

It's not the first time this specific API comes up. As mentioned, it's trivial to add. In this case we might even consider it if it doesn't have ill effects in our ports tree. As also mentioned, it's not possible to do this before 4.1. It seems a weirdly aggressive decision by the upstream dev to make using alternatives to OpenSSL 3 harder than necessary. There are still plenty of LTS distros that provide 1.1.1 libraries (and these will remain supported for many years to come). The version check will also bite BoringSSL who announce themselves as OpenSSL 1.1.1.

[botovq]

What apart form ccache uses this and needs the OpenSSL support? BoringSSL also doesn't currently have OPENSSL_thread_stop.

[orbea]

I could not find how they avoid the version check for BoringSSL even if they have checks for it in httplib.h where they also disable OPENSSL_thread_stop.

https://github.com/yhirose/cpp-httplib/blob/80fb03628bb57ca9d3ab855a7feec2876249bb61/httplib.h#L779-L781

Also I am not sure there is anything in Gentoo that needs ssl support in cpp-httplib even though its enabled by default. Its a dependency for openrgb, ccache and indilib as well as an optional dependency for lldb and llvm. However if any of them require ssl support its undocumented by the ebuilds.

[orbea]

This patch works for 0.16.3.

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -81,7 +81,7 @@ project(httplib
 
 # Change as needed to set an OpenSSL minimum version.
 # This is used in the installed Cmake config file.
-set(_HTTPLIB_OPENSSL_MIN_VER "3.0.0")
+set(_HTTPLIB_OPENSSL_MIN_VER "2.0.0")
 
 # Lets you disable C++ exception during CMake configure time.
 # The value is used in the install CMake config file.
--- a/httplib.h
+++ b/httplib.h
@@ -269,7 +269,7 @@ using socket_t = int;
 #include <iostream>
 #include <sstream>
 
-#if defined(OPENSSL_IS_BORINGSSL)
+#if defined(OPENSSL_IS_BORINGSSL) || defined(LIBRESSL_VERSION_NUMBER)
 #if OPENSSL_VERSION_NUMBER < 0x1010107f
 #error Please use OpenSSL or a current version of BoringSSL
 #endif
@@ -732,7 +732,7 @@ private:
         fn();
       }
 
-#if defined(CPPHTTPLIB_OPENSSL_SUPPORT) && !defined(OPENSSL_IS_BORINGSSL)
+#if defined(CPPHTTPLIB_OPENSSL_SUPPORT) && !defined(OPENSSL_IS_BORINGSSL) && !defined(LIBRESSL_VERSION_NUMBER)
       OPENSSL_thread_stop();
 #endif
     }

[botovq]

ok, thanks for the report and the info. The patch for 0.16.3 looks good. I assume that OpenSSL support is enabled simply because it's the upstream's default.

[orbea]

With the cpp-httplib git repo (yhirose/cpp-httplib@80fb036) the tests do not fail, they just hang on an openssl command.

openssl genrsa 2048 > key.pem
python3 ../split.py -o .
Generating RSA private key, 2048 bit long modulus
................Wrote ./httplib.h and ./httplib.cc
............................................................................................
........
e is 65537 (0x010001)
openssl req -new -batch -config test.conf -key key.pem | openssl x509 -days 3650 -req -signkey key.pem > cert.pem
Signature ok
subject=/C=US/ST=Test State or Province/L=Test Locality/O=Organization Name/OU=Organizational Unit Name/CN=Common Name/emailAddress=test@email.address
openssl req -x509 -config test.conf -key key.pem -sha256 -days 3650 -nodes -out cert2.pem -extensions SAN

This is test.conf.

[req]
default_bits           = 2048
distinguished_name     = req_distinguished_name
attributes             = req_attributes
prompt                 = no
output_password        = mypass

[req_distinguished_name]
C                      = US
ST                     = Test State or Province
L                      = Test Locality
O                      = Organization Name
OU                     = Organizational Unit Name
CN                     = Common Name
emailAddress           = test@email.address

[req_attributes]
challengePassword              = 1234

[SAN]
subjectAltName=IP:127.0.0.1

[botovq]

This diff makes the hanging openssl command work:

--- apps/openssl/req.c.orig	Fri Sep  6 13:24:38 2024
+++ apps/openssl/req.c	Fri Sep  6 15:42:00 2024
@@ -586,6 +586,9 @@ req_main(int argc, char **argv)
 	req_conf = NULL;
 	cipher = EVP_aes_256_cbc();
 
+	if (cfg.infile == NULL && cfg.x509)
+		cfg.newreq = 1;
+
 	if (!app_passwd(bio_err, cfg.passargin, cfg.passargout, &passin, &passout)) {
 		BIO_printf(bio_err, "Error getting passwords\n");
 		goto end;

[orbea]

Thanks, that patch allows the tests to run, a few fail.

[  FAILED  ] 3 tests, listed below:
[  FAILED  ] ConnectionErrorTest.InvalidHost
[  FAILED  ] ConnectionErrorTest.InvalidHost2
[  FAILED  ] ConnectionErrorTest.InvalidHostCheckResultErrorToString

test.cc:890: Failure
Expected equality of these values:
  Error::Connection
    Which is: Could not establish connection (2)
  res.error()
    Which is: Connection timed out (13)
[  FAILED  ] ConnectionErrorTest.InvalidHost (2047 ms)

test.cc:905: Failure
Expected equality of these values:
  Error::Connection
    Which is: Could not establish connection (2)
  res.error()
    Which is: Connection timed out (13)
[  FAILED  ] ConnectionErrorTest.InvalidHost2 (2048 ms)

test.cc:922: Failure
Expected equality of these values:
  "error code: Could not establish connection (2)"
  s.str()
    Which is: "error code: Connection timed out (13)"
[  FAILED  ] ConnectionErrorTest.InvalidHostCheckResultErrorToString (2049 ms)

I tried to send some of the fixes upstream, but they weren't interested.

yhirose/cpp-httplib#1922 (comment)