java.util.zip.ZipException: Not in GZIP format when using a ndv datafeed mirror created with vulnz

[JoKoT3]

Precondition

• I checked the issues list for existing open or closed reports of the same problem.

Describe the bug
Downloading nvd datafeed from a mirror created with vulnz fail with the following error :
java.util.zip.ZipException: Not in GZIP format

In detail :
I created a bitbucket page that mirror the nvd feed. The mirror is created with the vulnz docker image (steps: create cache with vulnz, git commit and push content, mirror is served by bitbucket page).
I then added nvdDataFeedUrl in the another project pom.xml pointing to the page.
I am able to check the urls are indeed correct, as I can download the json.gz with my browser.
dependency-check-maven (and I suppose others too) fail with the error above

if I check the http header I get from the server when I used my browser, I can see that
content-encoding is gzip
content-type is application/json

My best guess is that both client and server agree on handling gzip and the client ends up with an uncompressed json and then try to uncompress it again

Version of dependency-check used
The problem occurs using version 12.1.0 of the maven plugin (docker: maven:3.9.6-eclipse-temurin-17-alpine)

Log file
https://gist.github.com/JoKoT3/5a077a3a5d2d50c3abc17eb04c96f35d

To Reproduce
Steps to reproduce the behavior:

1. create a mirror with vulnz
2. serve said mirror with HTTP/2 with gzip content-encoding (bitbucket pages appear to use cloudfront)
3. set the URL for the mirror in nvdDataFeedUrl
4. run dependency-check in a way that the database has to be populated
5. see error

Expected behavior
Fix may be to implement a catch on the java.util.zip.ZipException and try to load it as an uncompressed json

Additional context
I can provide my personnal mirror URL in private for testing purposes

[aikebah]

@JoKoT3 Do you have .gz files stored in your bitbucket, or did you decompress them and stored them as json files?

[JoKoT3]

Hello, sorry for the delay.
The content of the bitbucket is indeed gzip compressed files

downloading the file through the bitbucket interface results in the same http headers (content-type: application/json, content-encoding: gzip, content-disposition: attachment

[jeremylong]

@aikebah the thought behind my PR is that something was going on header wise that was making the client decompress the GZ file...

[JoKoT3]

Since it's bugging me, triple checked locally the repo content :

joko@GIMLI:/mnt/c/Users/John/Code/nvd-mirror/content$ file nvdcve-2002.json.gz
nvdcve-2002.json.gz: gzip compressed data, original size modulo 2^32 16195365

[jeremylong]

Any chance you could grab the response headers and post them here?

[jeremylong]

Another thing that you could do that would be helpful - build the scratch/decompression branch used in my PR. mvn package -DskipTests and then use the CLI to download from your gitlab mirror. If that works - we have a solution.