writeJson causes "RangeError: Invalid string length"

[maarre]

There is a dependency to jsonfile. Jsonfile calls JSON.stringify. This will cause "RangeError: Invalid string length" for large objects.

I don't know if this is a bug or a feature request. For me this is a showstopper, I will have to find some streaming library implementing streaming json serialization instead.

[RyanZim]

I believe this is a duplicate of #1014; does --max-old-space-size help?

[maarre]

It is a limitation on V8. I cannot rely on code having such limitations.

I am writing my own streams based serialization. Would you be intrested?

[RyanZim]

An actual streaming JSON stringifier/parser would be well outside the scope of fs-extra

[maarre]

Implementation is streaming. You can wrap it in your file stuff. I have someting working now.

[maarre]

It is possible to create very large objects from file in fs-extra. The objects created cannot be written to file. It is not symmetric.

[maarre]

The same problem is for reading json from files. The whole file is read to a string by jsonfile. I will have to roll my own streams based parser as well.

Failed processing xxxxx.json : Error: Cannot create a string longer than 0x1fffffe8 characters
node:buffer:621
    slice: (buf, start, end) => buf.utf8Slice(start, end),
                                    ^

Error: Cannot create a string longer than 0x1fffffe8 characters
    at Object.slice (node:buffer:621:37)
    at Buffer.toString (node:buffer:840:14)
    at stripBom (/tttttt/node_modules/jsonfile/utils.js:10:51)
    at _readFile (/yyyyyy/node_modules/jsonfile/index.js:21:10)
    at async Command.<anonymous> (file:///wwwww/dist/main.js:31:81)
    at async Command.parseAsync (/yyyyy/command.js:1104:5)
    at async main (file://zzzzz/dist/main.js:52:3)
    at async file:///qqqqq/dist/main.js:55:1 {
  code: 'ERR_STRING_TOO_LONG'
}

Node.js v20.17.0