Add snappy extension

[stloyd]

This PR tries to introduce a new PHP extension: https://github.com/kjdev/php-ext-snappy for http://google.github.io/snappy/

Currently, I have a problem with linking the library with an extension (the bundled one is older and is not compiling at all on OSX).

---

Latest library linking error:

Configuring extensions checking for io.h... no checking for strtoll... yes checking for atoll... yes checking whether to build with LIBXML support... no checking for OpenSSL support... no checking for Kerberos support... no checking whether to use system default cipher list instead of hardcoded value... no checking for PCRE library to use... bundled checking whether to enable PCRE JIT functionality... yes checking whether Intel CET is enabled... no checking whether to enable the SQLite3 extension... no checking for ZLIB support... no checking whether to enable bc style precision math functions... no checking for BZip2 support... no checking whether to enable calendar conversion support... no checking whether to enable ctype functions... no checking for cURL support... no checking for QDBM support... no checking for GDBM support... no checking for NDBM support... no checking for TCADB support... no checking for LMDB support... no checking for Berkeley DB4 support... no checking for Berkeley DB3 support... no checking for Berkeley DB2 support... no checking for DBMaker support... no checking for any ODBC driver support... no checking whether to enable Zend OPcache support... no checking whether to enable copying PHP CODE pages into HUGE PAGES... yes checking whether to enable JIT... yes checking whether to enable pcntl support... no checking whether to enable PDO support... no checking for PDO_DBLIB support via FreeTDS... no checking for Firebird support for PDO... no checking for MySQL support for PDO... no checking for the location of libz... no checking Oracle OCI support for PDO... no checking for ODBC v3 support for PDO... no checking for PostgreSQL support for PDO... no checking for sqlite 3 support for PDO... no checking for PostgreSQL support... no checking for phar archive support... no checking whether to enable POSIX-like functions... no checking for PSPELL support... no checking whether arc4random_buf is declared... yes checking for CommonCrypto/CommonRandom.h... yes checking for libedit readline replacement... no checking for readline support... no checking whether to enable PHP sessions... no checking for mm support... no checking whether to enable shmop support... no checking whether to enable SimpleXML support... no checking PHP version... 8.2.10 checking whether to enable snappy support... yes checking for snappy header... /Users/runner/work/static-php-cli/static-php-cli/buildroot/lib checking searching for libsnappy... found in /Users/runner/work/static-php-cli/static-php-cli/buildroot/lib checking for snappy_compress in -lsnappy... no configure: error: could not find usable libsnappy [13:56:10] [ERRO] Uncaught SPC\exception\RuntimeException: Command run failed with code[1]: cd '/Users/runner/work/static-php-cli/static-php-cli/source/php-src' && ./configure --prefix= --with-valgrind=no --enable-shared=no --enable-static=yes CFLAGS='--target=x86_64-apple-darwin -Werror=unknown-warning-option' --disable-all --disable-cgi --disable-phpdbg --enable-cli --enable-fpm --enable-micro --enable-snappy --with-snappy-includedir="/Users/runner/work/static-php-cli/static-php-cli/buildroot/lib" PKG_CONFIG='/Users/runner/work/static-php-cli/static-php-cli/buildroot/bin/pkg-config' PKG_CONFIG_PATH='/Users/runner/work/static-php-cli/static-php-cli/buildroot/lib/pkgconfig/' CC='clang' CXX='clang++' CFLAGS='--target=x86_64-apple-darwin -Wimplicit-function-declaration -Os' PATH='/Users/runner/work/static-php-cli/static-php-cli/buildroot/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/usr/local/opt/[email protected]/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin:/Users/runner/hostedtoolcache/stack/2.11.1/x64' at /Users/runner/work/static-php-cli/static-php-cli/src/globals/functions.php(88) [13:56:10] [ERRO] #0 /Users/runner/work/static-php-cli/static-php-cli/src/SPC/util/UnixShell.php(44): f_passthru('cd '/Users/runn...') #1 /Users/runner/work/static-php-cli/static-php-cli/src/SPC/builder/macos/MacOSBuilder.php(155): SPC\util\UnixShell->exec('cd '/Users/runn...') #2 /Users/runner/work/static-php-cli/static-php-cli/src/SPC/command/BuildCliCommand.php(87): SPC\builder\macos\MacOSBuilder->buildPHP(1) #3 /Users/runner/work/static-php-cli/static-php-cli/src/SPC/command/BaseCommand.php(83): SPC\command\BuildCliCommand->handle() #4 /Users/runner/work/static-php-cli/static-php-cli/vendor/symfony/console/Command/Command.php(326): SPC\command\BaseCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #5 /Users/runner/work/static-php-cli/static-php-cli/vendor/symfony/console/Application.php(1063): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #6 /Users/runner/work/static-php-cli/static-php-cli/vendor/symfony/console/Application.php(320): Symfony\Component\Console\Application->doRunCommand(Object(SPC\command\BuildCliCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #7 /Users/runner/work/static-php-cli/static-php-cli/vendor/symfony/console/Application.php(174): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #8 /Users/runner/work/static-php-cli/static-php-cli/bin/spc(15): Symfony\Component\Console\Application->run() #9 {main}

Bundled library error:

In file included from /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:31: /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:212:10: error: unknown type name 'constexpr' static constexpr int kBlockLog = 16; ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:213:10: error: unknown type name 'constexpr' static constexpr size_t kBlockSize = 1 << kBlockLog; ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:213:26: error: expected ';' after top level declarator static constexpr size_t kBlockSize = 1 << kBlockLog; ^ ; /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:215:10: error: unknown type name 'constexpr' static constexpr int kMinHashTableBits = 8; ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:216:10: error: unknown type name 'constexpr' static constexpr size_t kMinHashTableSize = 1 << kMinHashTableBits; ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:216:26: error: expected ';' after top level declarator static constexpr size_t kMinHashTableSize = 1 << kMinHashTableBits; ^ ; /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:218:10: error: unknown type name 'constexpr' static constexpr int kMaxHashTableBits = 14; ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:219:10: error: unknown type name 'constexpr' static constexpr size_t kMaxHashTableSize = 1 << kMaxHashTableBits; ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.h:219:26: error: expected ';' after top level declarator static constexpr size_t kMaxHashTableSize = 1 << kMaxHashTableBits; ^ ; /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:92:1: error: unknown type name 'constexpr' constexpr int kSlopBytes = 64; ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:94:17: error: no member named 'char_table' in namespace 'snappy::internal' using internal::char_table; ~~~~~~~~~~^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:120:8: error: unknown type name 'constexpr' inline constexpr int16_t MakeEntry(int16_t len, int16_t offset) { ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:120:1: warning: inline variables are a C++17 extension [-Wc++17-extensions] inline constexpr int16_t MakeEntry(int16_t len, int16_t offset) { ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:120:25: error: expected ';' after top level declarator inline constexpr int16_t MakeEntry(int16_t len, int16_t offset) { ^ ; /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:139:26: error: unknown type name 'size_t' template ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:139:32: warning: variadic templates are a C++11 extension [-Wc++11-extensions] template ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:142:11: error: unknown type name 'size_t' template ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:142:17: warning: variadic templates are a C++11 extension [-Wc++11-extensions] template ^ /Users/runner/work/static-php-cli/static-php-cli/source/php-src/ext/snappy/snappy/snappy.cc:143:40: error: no template named 'index_sequence' struct make_index_sequence<0, Is...> : index_sequence {}; ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 12 warnings and 20 errors generated. make: *** [ext/snappy/snappy/snappy.lo] Error 1 [14:08:00] [ERRO] Uncaught SPC\exception\RuntimeException: Command run failed with code[2]: cd '/Users/runner/work/static-php-cli/static-php-cli/source/php-src' && make -j3 EXTRA_CFLAGS='-g -Os' EXTRA_LIBS='-lc++ -lresolv' cli at /Users/runner/work/static-php-cli/static-php-cli/src/globals/functions.php(88) [14:08:00] [ERRO] #0 /Users/runner/work/static-php-cli/static-php-cli/src/SPC/util/UnixShell.php(44): f_passthru('cd '/Users/runn...')

[crazywhalecc]

Thank you very much for contributing a new extension!

At present, I tried it locally and made the following modifications to make the compilation successful:

1. Patch ext-snappy.

<?php

declare(strict_types=1);

namespace SPC\builder\extension;

use SPC\builder\Extension;
use SPC\builder\macos\MacOSBuilder;
use SPC\exception\FileSystemException;
use SPC\store\FileSystem;
use SPC\util\CustomExt;

#[CustomExt('snappy')]
class snappy extends Extension
{
    /**
     * @throws FileSystemException
     */
    public function patchBeforeConfigure(): bool
    {
        FileSystem::replaceFileRegex(
            SOURCE_PATH . '/php-src/configure',
            '/-lsnappy/',
            $this->getLibFilesString() . $this->builder instanceof MacOSBuilder ? ' -lc++' : ' -lstdc++'
        );
        return true;
    }

    public function getUnixConfigureArg(): string
    {
        return '--enable-snappy --with-snappy-includedir="' . BUILD_ROOT_PATH . '"';
    }
}

2. Modify the install prefix of snappy, because I am not sure whether it is affected by PREFIX and cannot be found.

protected function build(): void
{
    FileSystem::resetDir($this->source_dir . '/cmake/build');

    shell()->cd($this->source_dir . '/cmake/build')
        ->exec(
            "{$this->builder->configure_env} cmake " .
            "-DCMAKE_TOOLCHAIN_FILE={$this->builder->cmake_toolchain_file} " .
            '-DCMAKE_BUILD_TYPE=Release ' .
            '-DCMAKE_INSTALL_PREFIX=' . escapeshellarg(BUILD_ROOT_PATH) . ' ' .
            '-DSNAPPY_BUILD_TESTS=OFF ' .
            '-DSNAPPY_BUILD_BENCHMARKS=OFF ' .
            '../..'
        )
        ->exec("cmake --build . -j {$this->builder->concurrency}")
        ->exec('make install');
}

3. Modify dependencies, set it as c++ extension and add zlib.

{"snappy": {
        "type": "external",
        "source": "ext-snappy",
        "cpp-extension": true,
        "arg-type": "custom",
        "lib-depends": [
            "snappy"
        ],
        "ext-suggest": [
            "apcu"
        ]
    }}

{
"snappy": {
        "source": "snappy",
        "static-libs-unix": [
            "libsnappy.a"
        ],
        "headers-unix": [
            "snappy-c.h",
            "snappy-sinksource.h",
            "snappy.h",
            "snappy-stubs-internal.h",
            "snappy-stubs-public.h"
        ],
        "lib-depends": [
            "zlib"
        ]
    }
}

I only tested on macOS and it's just a simple experiment. For Linux, because it is written in C++, it should only be fully statically compiled under Alpine.

When you're done with it, I'd do a full local compilation test. 👍

[stloyd]

@crazywhalecc Thanks for quick check, adjusted to what you posted, but unfortunately it's still not correctly linking ext to lib.

Configuring extensions checking for io.h... no checking for strtoll... yes checking for atoll... yes checking whether to build with LIBXML support... no checking for OpenSSL support... no checking for Kerberos support... no checking whether to use system default cipher list instead of hardcoded value... no checking for PCRE library to use... bundled checking whether to enable PCRE JIT functionality... yes checking whether Intel CET is enabled... no checking whether to enable the SQLite3 extension... no checking for ZLIB support... no checking whether to enable APCu support... no checking if APCu should be allowed to use rwlocks... yes checking if APCu should be built in debug mode... no checking if APCu should clear on SIGUSR1... no checking if APCu will use mmap (or shm)... yes checking if APCu should utilize spinlocks before flocks... no checking whether the target compiler supports builtin atomics... yes checking whether to include code coverage symbols... no checking whether to enable bc style precision math functions... no checking for BZip2 support... no checking whether to enable calendar conversion support... no checking whether to enable ctype functions... no checking for cURL support... no checking for QDBM support... no checking for GDBM support... no checking for NDBM support... no checking for TCADB support... no checking for LMDB support... no checking for Berkeley DB4 support... no checking for Berkeley DB3 support... no checking for Berkeley DB2 support... no checking for DB1 support... no checking for DBM support... no checking for CDB support... no checking for INI File support... no checking for FlatFile support... no checking whether to enable DBA interface... no checking whether to enable dl-test extension... no checking whether to enable DOM support... no checking whether to build with Enchant support... no checking whether to enable EXIF (metadata from images) support... no checking for FFI support... no checking for fileinfo support... no checking whether to enable input filter support... no checking whether to enable FTP support... no checking whether to explicitly enable FTP SSL support... no checking for GD support... no checking for external libgd... no checking for libavif... no checking for libwebp... no checking for libjpeg... no checking for libXpm... no checking for FreeType 2... no checking whether to enable JIS-mapped Japanese font support in GD... no checking for GNU gettext support... no checking for GNU MP support... no checking for mhash support... no checking size of long... (cached) 8 checking if we're at 64-bit platform... yes checking for iconv support... no checking for IMAP support... no checking for IMAP Kerberos support... no checking for IMAP SSL support... no checking whether to enable internationalization support... no checking for LDAP support... no checking whether to build with LDAP Cyrus SASL support... no checking whether to enable multibyte string support... no checking whether to enable multibyte regex support (requires oniguruma)... yes checking for MySQLi support... no checking for specified location of the MySQL UNIX socket... no checking for Oracle Database OCI8 support... no checking for Adabas support... no checking for SAP DB support... no checking for Solid support... no checking for IBM DB2 support... no checking for Empress support... no checking for Empress local access support... no checking for a custom ODBC support... no checking whether to build with iODBC support... no checking for Easysoft ODBC-ODBC Bridge support... no checking whether to build with unixODBC support... no checking for DBMaker support... no checking for any ODBC driver support... no checking whether to enable Zend OPcache support... no checking whether to enable copying PHP CODE pages into HUGE PAGES... yes checking whether to enable JIT... yes checking whether to enable pcntl support... no checking whether to enable PDO support... no checking for PDO_DBLIB support via FreeTDS... no checking for Firebird support for PDO... no checking for MySQL support for PDO... no checking for the location of libz... no checking Oracle OCI support for PDO... no checking for ODBC v3 support for PDO... no checking for PostgreSQL support for PDO... no checking for sqlite 3 support for PDO... no checking for PostgreSQL support... no checking for phar archive support... no checking whether to enable POSIX-like functions... no checking for PSPELL support... no checking for libedit readline replacement... no checking for readline support... no checking whether to enable PHP sessions... no checking for mm support... no checking whether to enable shmop support... no checking whether to enable SimpleXML support... no checking PHP version... 8.1.23 checking whether to enable snappy support... yes checking for snappy header... /Users/stloyd/Documents/static-php-cli/buildroot checking searching for libsnappy... found in /Users/stloyd/Documents/static-php-cli/buildroot checking for snappy_compress in -lc++... no configure: error: could not find usable libsnappy

SPC\exception\RuntimeException

Command run failed with code[1]: cd '/Users/stloyd/Documents/static-php-cli/source/php-src' && ./configure --prefix= --with-valgrind=no --enable-shared=no --enable-static=yes CFLAGS='--target=x86_64-apple-darwin -Werror=unknown-warning-option' --disable-all --disable-cgi --disable-phpdbg --enable-cli --enable-fpm --enable-micro --enable-snappy --with-snappy-includedir="/Users/stloyd/Documents/static-php-cli/buildroot" PKG_CONFIG='/Users/stloyd/Documents/static-php-cli/buildroot/bin/pkg-config' PKG_CONFIG_PATH='/Users/stloyd/Documents/static-php-cli/buildroot/lib/pkgconfig/' CC='clang' CXX='clang++' CFLAGS='--target=x86_64-apple-darwin -Wimplicit-function-declaration -Os' PATH='/Users/stloyd/Documents/static-php-cli/buildroot/bin:/usr/local/Cellar/[email protected]/8.1.23/bin:/usr/local/opt/[email protected]/sbin:/usr/local/opt/[email protected]/bin:/usr/local/opt/[email protected]/sbin:/usr/local/opt/[email protected]/bin:/usr/local/sbin:/usr/local/opt/unzip/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin'

at src/globals/functions.php:88
84▕ logger()->debug('Running command with direct output: ' . $cmd);
85▕ }
86▕ $ret = passthru($cmd, $code);
87▕ if ($code !== 0) {
➜ 88▕ throw new \SPC\exception\RuntimeException('Command run failed with code[' . $code . ']: ' . $cmd, $code);
89▕ }
90▕ return $ret;
91▕ }
92▕

1 src/SPC/util/UnixShell.php:44

2 src/SPC/builder/macos/MacOSBuilder.php:171
SPC\util\UnixShell::exec("cd '/Users/stloyd/Documents/static-php-cli/source/php-src' && ./configure --prefix= --with-valgrind=no --enable-shared=no --enable-static=yes CFLAGS='--target=x86_64-apple-darwin -Werror=unknown-warning-option' --disable-all --disable-cgi --disable-phpdbg --enable-cli --enable-fpm --enable-micro --enable-snappy --with-snappy-includedir="/Users/stloyd/Documents/static-php-cli/buildroot" PKG_CONFIG='/Users/stloyd/Documents/static-php-cli/buildroot/bin/pkg-config' PKG_CONFIG_PATH='/Users/stloyd/Documents/static-php-cli/buildroot/lib/pkgconfig/' CC='clang' CXX='clang++' CFLAGS='--target=x86_64-apple-darwin -Wimplicit-function-declaration -Os' PATH='/Users/stloyd/Documents/static-php-cli/buildroot/bin:/usr/local/Cellar/[email protected]/8.1.23/bin:/usr/local/opt/[email protected]/sbin:/usr/local/opt/[email protected]/bin:/usr/local/opt/[email protected]/sbin:/usr/local/opt/[email protected]/bin:/usr/local/sbin:/usr/local/opt/unzip/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Appl...")


Tested a few variations (opposite compiler, with just bundled lib), etc. but either it was above or previous errors.

[crazywhalecc]

Oh, I made a mistake:

// Operator precedence bothers me
$this->getLibFilesString() . ($this->builder instanceof MacOSBuilder ? ' -lc++' : ' -lstdc++')

[stloyd]

You're fix was correct, I guess I'm wasted that I didn't notice the precedence problem.

btw. both of versions allowed to compile the code:

$this->getLibFilesString() . ($this->builder instanceof MacOSBuilder ? ' -lc++' : ' -lstdc++')

[17:30:16] [INFO] Build complete, used 205.435 s !
[17:30:16] [INFO] Static php binary path: /Users/stloyd/Documents/static-php-cli/buildroot/bin/php
[17:30:16] [DEBU] Removing path recursively: "/Users/stloyd/Documents/static-php-cli/buildroot/license"
[17:30:16] [DEBU] scanning directory /Users/stloyd/Documents/static-php-cli/buildroot/license
[17:30:16] [DEBU] Making new directory recursive: /Users/stloyd/Documents/static-php-cli/buildroot/license
[17:30:16] [INFO] License path: /Users/stloyd/Documents/static-php-cli/buildroot/license/

---

$this->getLibFilesString() . (!$this->builder instanceof MacOSBuilder ? ' -lc++' : ' -lstdc++')

[17:21:24] [INFO] Build complete, used 205.135 s !
[17:21:24] [INFO] Static php binary path: /Users/stloyd/Documents/static-php-cli/buildroot/bin/php
[17:21:24] [DEBU] Removing path recursively: "/Users/stloyd/Documents/static-php-cli/buildroot/license"
[17:21:24] [DEBU] scanning directory /Users/stloyd/Documents/static-php-cli/buildroot/license
[17:21:24] [DEBU] Making new directory recursive: /Users/stloyd/Documents/static-php-cli/buildroot/license
[17:21:24] [INFO] License path: /Users/stloyd/Documents/static-php-cli/buildroot/license/

[crazywhalecc]

Test done. I will merge it after arm64 test.

btw I want to know is there a reason to use the source code of the snappy library and snappy extensions from the master branch instead of the release version?

[stloyd]

The bundled version of Snappy was not possible to compile for me locally, probably it's doable with some hacks/changes that I was not aware of but also I used zstd extension as a base which also has a bundled library in the extension, but locally compiles a different one.