GDAL installation added to python env.

[thavaahariharangit]

What are you trying to accomplish?

Providing fixes for gdal related issues in this workflow

Solution provide

1. Installing GDAL dependencies
2. SSL verification disabled for get_dependency_hash function

Anything you want to highlight for special attention from reviewers?

SSL verification disabled for get_dependency_hash function, As we are working in a trusted environment : https://pypi.org/simple

How will you know you've accomplished your goal?

Ran the cli in local and ensured below output

2025/01/28 21:59:59 INFO Finished job processing
2025/01/28 21:59:59 INFO Results:
+-----------------------------------------------------+
|         Changes to Dependabot Pull Requests         |
+---------+-------------------------------------------+
| created | attrs ( from 24.3.0 to 25.1.0 )           |
| created | gdal ( from 3.9.3 to 3.10.1 )             |
| created | imageio ( from 2.36.1 to 2.37.0 )         |
| created | numpy ( from 2.2.1 to 2.2.2 )             |
| created | psycopg ( from 3.2.3 to 3.2.4 )           |
| created | scikit-image ( from 0.25.0 to 0.25.1 )    |
| created | scikit-learn ( from 1.6.0 to 1.6.1 )      |
| created | scipy ( from 1.14.1 to 1.15.1 )           |
| created | tifffile ( from 2024.12.12 to 2025.1.10 ) |
| created | tzdata ( from 2024.2 to 2025.1 )          |
| created | filelock ( from 3.16.1 to 3.17.0 )        |
| created | isort ( from 5.13.2 to 6.0.0 )            |
+---------+-------------------------------------------+

Checklist

• I have run the complete test suite to ensure all tests and linters pass.
• I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
• I have written clear and descriptive commit messages.
• I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
• I have ensured that the code is well-documented and easy to understand.

[abdulapopoola]

@jeffwidman or @jakecoffman or @pavera , is there another way to get gdal that doesn't require going to grab it via wget? Just in case you all know

[thavaahariharangit]

I have updated wget to git clone, Still I would like to know your thought on this implementation pls.

[paulschreiber]

Why are you compiling GDAL instead of installing the existing .deb packages?

[thavaahariharangit]

The problem that I am trying to resolve here is a compatibility issue

Exception: Python bindings of GDAL 3.9.3 require at least libgdal 3.9.3, but 3.0.4 was found

Steps to reproduce:

1. Go to https://github.com/techmatters/soil-id-algorithm
2. Create new codespace for this repo
3. In terminal run make install
4. Error : gdal_config_error: [Errno 2] No such file or directory: 'gdal-config'
5. In order to fix this you need to install libgdal-dev

• sudo apt-get update
• sudo apt-get install libgdal-dev

7. Once that's done, run make install again
8. Error: Exception: Python bindings of GDAL 3.9.3 require at least libgdal 3.9.3, but 3.0.4 was found

So I need to ensure that all the packages that I am installing is compatible with each other. That is why I chose 3.10.1 latest gdal version and installing all the compatible packages to that version here.

[thavaahariharangit]

@paulschreiber Please feel free to share your thoughts on any better way to resolve this, I am happy take that to my team's discussion. and make that available on dependabot.

[paulschreiber]

We ended up adjusting the debian sources list to testing to get current GDAL, since bookworm is behind:
https://github.com/techmatters/terraso-backend/blob/main/Dockerfile#L11-L13

RUN sed 's/bookworm/testing/g' /etc/apt/sources.list.d/debian.sources >  /etc/apt/sources.list.d/testing.sources

RUN echo 'Package: libgdal-dev gdal-bin\nPin: release a=testing\nPin-Priority: 900' > /etc/apt/preferences.d/gdal-testing

[robaiken]

Building gdal from source seems excessive here. This adds complexity to our builds and requires us to manually maintain the gdal version. Could we find a simpler solution that doesn't involve building gis libraries from source?

[thavaahariharangit]

Other approaches to install gdal has failed with Exception: Python bindings of GDAL 3.10.1 require at least libgdal 3.10.1, but 3.4.1 was found exception

Command that gave above exception: pyenv exec pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host=files.pythonhosted.org gdal

even apt-get also failed with the same exception (version conflicts, 3.4.1 comes with python installation by default and 3.10.1 is the latest)

And this gdal installation from the source ensures the presence of all the necessary gdal native libraries, which is needed to build a gdal based application.

And I have used version checkout that will ensure that we are using stable release. or the tested and released version is being used.

Note: There are other sources for installation such as Anaconda and Mamba but those are commercial products needs approval from the team to go ahead.

[robaiken]

My concern isn't just about the implementation method, but rather about the fundamental decision to add gdal to our container at all. This raises several questions:

1. Why do we need gdal in the container?
2. If we add gdal for one Python version, shouldn't we maintain consistency across all our ecosystems?
3. Are we setting a precedent for adding large, specialized libraries to our base container?

While I understand the issue you are trying to fix, I'm worried about the implications of this approach for our architecture.

[thavaahariharangit]

As we have decided consider this as a feature request, I am closing this PR.