Bug: Elasticsearch tests fail on machine with ARM CPU (Apple M1, M2)

[max-pfeiffer]

Describe the bug

Elasticsearch tests fail, see screenshots:

To Reproduce

To Reproduce

1. Clone project and install project dependencies with Poetry
2. Run test with pytest: pytest modules/elasticsearch/tests/test_elasticsearch.py

Runtime environment
OS: Darwin 22.6.0 Darwin Kernel Version 22.6.0: Sun Dec 17 22:13:25 PST 2023; root:xnu-8796.141.3.703.2~2/RELEASE_ARM64_T6020 arm64

Python: 3.11.6

Poetry show:

alabaster                     0.7.16       A light, configurable Sphinx theme
anyio                         4.3.0        High level compatibility layer for multiple asynchronous event loop implementations
argon2-cffi                   23.1.0       Argon2 for Python
argon2-cffi-bindings          21.2.0       Low-level CFFI bindings for Argon2
asn1crypto                    1.5.1        Fast ASN.1 parser and serializer with definitions for private keys, public keys, certificates, CRL, OCSP, CMS, PKCS#3, PKCS#7, PKCS#8, PKCS#12, PKCS#5, X.509 an...
attrs                         23.2.0       Classes Without Boilerplate
azure-core                    1.29.7       Microsoft Azure Core Library for Python
azure-storage-blob            12.19.0      Microsoft Azure Blob Storage Client Library for Python
babel                         2.14.0       Internationalization utilities
boto3                         1.34.28      The AWS SDK for Python
botocore                      1.34.28      Low-level, data-driven core of boto 3.
cachetools                    5.3.2        Extensible memoizing collections and decorators
certifi                       2023.11.17   Python package for providing Mozilla's CA Bundle.
cffi                          1.16.0       Foreign Function Interface for Python calling C code.
cfgv                          3.4.0        Validate configuration and produce human readable error messages.
charset-normalizer            3.3.2        The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet.
clickhouse-driver             0.2.6        Python driver with native interface for ClickHouse
coverage                      7.4.0        Code coverage measurement for Python
cryptography                  42.0.1       cryptography is a package which provides cryptographic recipes and primitives to Python developers.
cx-oracle                     8.3.0        Python interface to Oracle
deprecation                   2.1.0        A library to handle automated deprecations
distlib                       0.3.8        Distribution utilities
dnspython                     2.5.0        DNS toolkit
docker                        7.0.0        A Python library for the Docker Engine API.
docutils                      0.20.1       Docutils -- Python Documentation Utilities
ecdsa                         0.18.0       ECDSA cryptographic signature library (pure python)
filelock                      3.13.1       A platform independent file lock.
google-api-core               2.15.0       Google API client core library
google-auth                   2.27.0       Google Authentication Library
google-cloud-pubsub           2.19.0       Google Cloud Pub/Sub API client library
googleapis-common-protos      1.62.0       Common protobufs used in Google APIs
grpc-google-iam-v1            0.13.0       IAM API client library
grpcio                        1.60.0       HTTP/2-based RPC framework
grpcio-status                 1.60.0       Status proto mapping for gRPC
h11                           0.14.0       A pure-Python, bring-your-own-I/O implementation of HTTP/1.1
identify                      2.5.35       File identification library for Python
idna                          3.6          Internationalized Domain Names in Applications (IDNA)
imagesize                     1.4.1        Getting image size from png/jpeg/jpeg2000/gif file
importlib-metadata            7.0.1        Read metadata from Python packages
iniconfig                     2.0.0        brain-dead simple config-ini parsing
isodate                       0.6.1        An ISO 8601 date/time/duration parser and formatter
jaraco-classes                3.3.1        Utility functions for Python class constructs
jinja2                        3.1.3        A very fast and expressive template engine.
jmespath                      1.0.1        JSON Matching Expressions
kafka-python                  2.0.2        Pure Python client for Apache Kafka
keyring                       24.3.0       Store and access your passwords safely.
kubernetes                    29.0.0       Kubernetes python client
markdown-it-py                3.0.0        Python port of markdown-it. Markdown parsing, done right!
markupsafe                    2.1.4        Safely add untrusted strings to HTML/XML markup.
mdurl                         0.1.2        Markdown URL utilities
minio                         7.2.3        MinIO Python SDK for Amazon S3 Compatible Cloud Storage
more-itertools                10.2.0       More routines for operating on iterables, beyond itertools
mypy                          1.7.1        Optional static typing for Python
mypy-extensions               1.0.0        Type system extensions for programs checked with the mypy type checker.
neo4j                         5.16.0       Neo4j Bolt driver for Python
nh3                           0.2.15       Python bindings to the ammonia HTML sanitization library.
nodeenv                       1.8.0        Node.js virtual environment builder
oauthlib                      3.2.2        A generic, spec-compliant, thorough implementation of the OAuth request-signing logic
opensearch-py                 2.4.2        Python client for OpenSearch
outcome                       1.3.0.post0  Capture the outcome of Python function calls.
packaging                     23.2         Core utilities for Python packages
pg8000                        1.30.4       PostgreSQL interface library
pika                          1.3.2        Pika Python AMQP Client Library
pkginfo                       1.9.6        Query metadata from sdists / bdists / installed packages.
platformdirs                  4.2.0        A small Python package for determining appropriate platform-specific dirs, e.g. a "user data dir".
pluggy                        1.4.0        plugin and hook calling mechanisms for python
pre-commit                    3.6.2        A framework for managing and maintaining multi-language pre-commit hooks.
proto-plus                    1.23.0       Beautiful, Pythonic protocol buffers.
protobuf                      4.25.2       
psycopg2-binary               2.9.9        psycopg2 - Python-PostgreSQL Database Adapter
pyasn1                        0.5.1        Pure-Python implementation of ASN.1 types and DER/BER/CER codecs (X.208)
pyasn1-modules                0.3.0        A collection of ASN.1-based protocols modules
pycparser                     2.21         C parser in Python
pycryptodome                  3.20.0       Cryptographic library for Python
pygments                      2.17.2       Pygments is a syntax highlighting package written in Python.
pyjwt                         2.8.0        JSON Web Token implementation in Python
pymongo                       4.6.1        Python driver for MongoDB <http://www.mongodb.org>
pymssql                       2.2.11       DB-API interface to Microsoft SQL Server for Python. (new Cython-based version)
pymysql                       1.1.0        Pure Python MySQL Driver
pysocks                       1.7.1        A Python SOCKS client module. See https://github.com/Anorov/PySocks for more information.
pytest                        7.4.3        pytest: simple powerful testing with Python
pytest-cov                    4.1.0        Pytest plugin for measuring coverage.
python-arango                 7.9.1        Python Driver for ArangoDB
python-dateutil               2.8.2        Extensions to the standard Python datetime module
python-jose                   3.3.0        JOSE implementation in Python
python-keycloak               3.7.0        python-keycloak is a Python package providing access to the Keycloak API.
pytz                          2023.3.post1 World timezone definitions, modern and historical
pyyaml                        6.0.1        YAML parser and emitter for Python
readme-renderer               42.0         readme_renderer is a library for rendering readme descriptions for Warehouse
redis                         5.0.1        Python client for Redis database and key-value store
requests                      2.31.0       Python HTTP for Humans.
requests-oauthlib             1.3.1        OAuthlib authentication support for Requests.
requests-toolbelt             1.0.0        A utility belt for advanced users of python-requests
rfc3986                       2.0.0        Validating URI References per RFC 3986
rich                          13.7.0       Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal
rsa                           4.9          Pure-Python RSA implementation
s3transfer                    0.10.0       An Amazon S3 Transfer Manager
scramp                        1.4.4        An implementation of the SCRAM protocol.
selenium                      4.17.2       
setuptools                    69.0.3       Easily download, build, install, upgrade, and uninstall Python packages
six                           1.16.0       Python 2 and 3 compatibility utilities
sniffio                       1.3.0        Sniff out which async library your code is running under
snowballstemmer               2.2.0        This package provides 29 stemmers for 28 languages generated from Snowball algorithms.
sortedcontainers              2.4.0        Sorted Containers -- Sorted List, Sorted Dict, Sorted Set
sphinx                        7.2.6        Python documentation generator
sphinxcontrib-applehelp       1.0.8        sphinxcontrib-applehelp is a Sphinx extension which outputs Apple help books
sphinxcontrib-devhelp         1.0.6        sphinxcontrib-devhelp is a sphinx extension which outputs Devhelp documents
sphinxcontrib-htmlhelp        2.0.5        sphinxcontrib-htmlhelp is a sphinx extension which renders HTML help files
sphinxcontrib-jsmath          1.0.1        A sphinx extension which renders display math in HTML via JavaScript
sphinxcontrib-qthelp          1.0.7        sphinxcontrib-qthelp is a sphinx extension which outputs QtHelp documents
sphinxcontrib-serializinghtml 1.1.10       sphinxcontrib-serializinghtml is a sphinx extension which outputs "serialized" HTML files (json and pickle)
sqlalchemy                    2.0.25       Database Abstraction Library
trio                          0.24.0       A friendly Python library for async concurrency and I/O
trio-websocket                0.11.1       WebSocket library for Trio
twine                         4.0.2        Collection of utilities for publishing packages on PyPI
typing-extensions             4.9.0        Backported and Experimental Type Hints for Python 3.8+
tzlocal                       5.2          tzinfo object for the local timezone
urllib3                       2.0.7        HTTP library with thread-safe connection pooling, file post, and more.
virtualenv                    20.25.1      Virtual Python Environment builder
websocket-client              1.7.0        WebSocket client for Python with low level API options
wrapt                         1.16.0       Module for decorators, wrappers and monkey patching.
wsproto                       1.2.0        WebSockets state-machine based protocol implementation
zipp                          3.17.0       Backport of pathlib-compatible object wrapper for zip files

Docker info:

docker info 
Client:
 Version:    25.0.3
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1-desktop.4
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.6-desktop.1
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container. (Docker Inc.)
    Version:  0.0.24
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.22
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.0.1
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.5.0
    Path:     /Users/maxpfeiffer/.docker/cli-plugins/docker-scout
WARNING: Plugin "/Users/maxpfeiffer/.docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /Users/maxpfeiffer/.docker/cli-plugins/docker-scan: no such file or directory

Server:
 Containers: 123
  Running: 0
  Paused: 0
  Stopped: 123
 Images: 259
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.16-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 15.6GiB
 Name: docker-desktop
 ID: b8a8253b-450b-42f1-9f23-f14f67094b44
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

[vinicinbgs]

This image elastichsearch:6.8.23 that you used doesn't have the version linux/arm64 thats why your receive 404 error.

You can check here: https://hub.docker.com/layers/library/elasticsearch/6.8.23/images/sha256-71d4f7d890b90d277a1677d53f100139dcb39a34e9aac1138bfbf272c878a9cf?context=explore

You can fix your problem using an image with version 7.8.0 or greater

[max-pfeiffer]

@vinicinbgs Thanks for your help. I quickly wrote a fix for it.