Description
Describe the bug
Node.js version: Node 14, 16, 18, 20
OS version: Mac Ventura 3.0.1
Description: It seems like its not possible to receive a response body when the content is compressed using Brotli compression.
Actual behavior
SyntaxError: Unexpected token in JSON at position 0 since the body isn't decompressed
Expected behavior
Body decompressed from Brotli and JSON is returned
Code to reproduce
I tried this simple code:
const superagent = require('superagent');
superagent
.get('http://httpbin.org/brotli')
.set('Accept-Encoding', 'br, gzip, deflate')
.then(response => {
console.log('Response Headers:', response.headers);
console.log('Response:', response.text);
})
.catch(error => {
console.error('Failed getting Brotli response');
console.error('Error:', error);
});
And got SyntaxError: Unexpected token in JSON at position 0 since the body isn't decompressed
When I tried to manually decompress it like that:
const brotli = require('brotli');
const superagent = require('superagent');
superagent
.get('http://httpbin.org/brotli')
.set('Accept-Encoding', 'br, gzip, deflate')
.responseType('arraybuffer') // Get the response as a buffer
.then(response => {
console.log('Response Headers:', response.headers);
const contentEncoding = response.headers['content-encoding'];
const decompressedData = brotli.decompress(response.body);
const responseText = new TextDecoder().decode(decompressedData);
console.log('Response Data:', responseText);
})
.catch(error => {
console.error('Failed getting Brotli response');
console.error('Error:', error);
});
It worked and I got the proper result:
Response Headers: {
date: 'Sun, 14 Jul 2024 06:21:12 GMT',
'content-type': 'application/json',
'content-length': '173',
connection: 'close',
server: 'gunicorn/19.9.0',
'content-encoding': 'br',
'access-control-allow-origin': '*',
'access-control-allow-credentials': 'true'
}
Response Data: {
"brotli": true,
"headers": {
"Accept-Encoding": "br, gzip, deflate",
"Host": "httpbin.org",
},
"method": "GET",
}
I tried using node-fetch and axios and it worked without a need to manually intercept it and Im guessing that behind the scenes you are using Node's http module that have the zlib module that handles gzip and etc... so what am I missing here?
Checklist
- [V] I have searched through GitHub issues for similar issues.
- [V] I have completely read through the README and documentation.
- [V] I have tested my code with the latest version of Node.js and this package and confirmed it is still not working.