Type mismatch: Property IntValue.Value should be Int32, but it was Int64

[ThatTimothy]

A bit of background: I've been using Lune to convert over some old files into a more usable file system format rather than arbitrary binary files. It seems like some of my older files aren't able to be deserialized.

I can reopen them and save again, and it seems to work fine after that, but I really don't want to do that another 2,000 times. If this is a non-issue, I totally understand lol. It's a very old file from early 2018.

Lune log with RUST_LOG=trace:

local inputFile = fs.readFile("repo.rbxl")
local game = roblox.deserializePlace(inputFile)
print(game.ClassName)

[TRACE rbx_binary::chunk] Chunk "PROP" (compressed: 24, len: 22, reserved: 0)
[TRACE rbx_binary::deserializer::state] PROP chunk (IntValue.Value, instance type 29, prop type 29
[TRACE rbx_binary::deserializer::state] Known prop, canonical name Value and type Int64, with None migration
Failed to read document from buffer - Type mismatch: Property IntValue.Value should be Int32, but it was Int64

Remodel log with RUST_LOG=trace:

local game = remodel.readPlaceFile("repo.rbxl")
print(game.ClassName)

[TRACE] Known prop, canonical name Tags and type Tags
[TRACE] Chunk "PROP" (compressed: 24, len: 22, reserved: 0)
[TRACE] PROP chunk (IntValue.Value, instance type 29, prop type 29
[TRACE] Known prop, canonical name Value and type Int64
[ERROR] callback error
        stack traceback:
                [C]: in field 'readPlaceFile'
                [string "remodel.lua"]:1: in main chunk
        caused by: Type mismatch: Property IntValue.Value should be Int32, but it was Int64

Repo file: https://github.com/ThatTimothy/rbx-binary-issue-repo/blob/main/repo.rbxl (didn't let me upload directly, sorry)

[kennethloeffler]

Given all the information here, it looks like somewhere in between early 2018 and when rbx-dom started supporting the Roblox binary format (late 2020), the serialized type for the IntValue.Value property changed from Int32 to Int64. This can probably be patched by adding an arm for VariantType::Int64 in the match canonical_type here:

rbx-dom/rbx_binary/src/deserializer/state.rs

Line 482 in bc1f154

Type::Int32 => match canonical_type {

You'll want to read_interleaved_i64_array (much like the VariantType::Int32 arm, except this time reading i64s), and convert the values to i32, adding them to their respective instances.

I'm not sure if rbx-dom would take this change back upstream - does anyone else want to chime in?

[ThatTimothy]

Thanks for looking into this! I'll try implementing my own fork and see if I can get it working.

[kennethloeffler]

You'll want to read_interleaved_i64_array (much like the VariantType::Int32 arm, except this time reading i64s), and convert the values to i32, adding them to their respective instances.

Yeah... so I'm wrong here. I think you'll want exactly the same code as in the VariantType::Int32 arm, just cast to i64 when you call add_property, sorry!

[ThatTimothy]

Okay so this is really weird - it's actually the reverse I had to do. I had to, when given Int64, and canonical type is Int32, read it as i32? I would've expected to read it as i64, but that gave a buffer not filled issue (Failed to read document from buffer - failed to fill whole buffer), and reading it as i32 fixed it? Further reads and writes perform normally on the current version after it is fixed.

I looked into how migrations currently work and I don't see a easy way to change types, but someone more experienced might be able to figure that out as a better solution.

For what it's worth, here's my quick and dirty solution: https://github.com/ThatTimothy/rbx-dom/tree/int-32-to-int-64-patch
Diff: Removed, see below

I don't understand why this works, did Roblox decrease the size? That just doesn't make sense that this would fix it.

[ThatTimothy]

Nevermind, I think I got it?

IntValue.Value is showing up as Int64 in xml now instead of int, so I think that's good?

<Item class="IntValue" referent="1406">
                  <Properties>
                    <string name="Name">GlowRange</string>
                    <BinaryString name="Tags"></BinaryString>
                    <int64 name="Value">6</int64>
                  </Properties>
                </Item>

I understand now: canonical type is what we expect from the reflection database (Int64 in this case), vs the binary_type (Int32 in this case). So I had to define a way to get the Int64 type from the Int32 binary type.

Newest diffs: master...ThatTimothy:rbx-dom:int-32-to-int-64-patch

[Dekkonot]

That diff looks good to me, so feel free to open a PR with it.

This needs to be resolved with rbx_xml too, but I've gone ahead and opened #303 since I'm in the process of rewriting that crate.

[ThatTimothy]

Cleaned it up a bit and also did a float change as well, I've opened #305 for the rbx_binary version.