Skip to content

Buffer underflow while deserializing with Collection type field removed (with reproducible test case) #834

Closed
@BrotherJing

Description

@BrotherJing

Describe the bug
Got KryoException: Buffer underflow when I try to deserialize data after remove a field with Collection type from the class.

To Reproduce

First, serialize the original object.

public static class A {
    public List<String> aaa = new ArrayList<>();
    public String bbb = "bbb";
    public String ddd = bbb;
}

@Test
public void test_kryo5() {
    A a = new A();
    a.aaa.add("a");
    byte[] bytes = new KryoSerializeUtil().serialize(a);
    String s = Base64.getEncoder().encodeToString(bytes);
    System.out.println(s);
}

Then, comment field aaa, and run

@Test
public void test_kryo5Deserialize() {
    // the serialized data from above
    String s = "AQBjb20ucGRkLmJpZ2RhdGEucmlzay5wdWJsaXNoLmNvbnRyYWN0LnV0aWxzLkNoZWNrc3VtVXRpbFRlc3QkwQEDYWHhYmLiZGTkGgEBamF2YS51dGlsLkFycmF5TGlz9AECAYJhAAUDAWJi4gACAwUA";
    A a = deserialize(Base64.getDecoder().decode(s));
    System.out.println(a.ddd);
}

Got

00:00 DEBUG: [kryo] Unable to read unknown data, type: java.util.ArrayList (com.pdd.bigdata.risk.publish.contract.utils.ChecksumUtilTest$A#null)
com.esotericsoftware.kryo.kryo5.KryoException: Buffer underflow.
	at com.esotericsoftware.kryo.kryo5.io.Input.require(Input.java:219)
	at com.esotericsoftware.kryo.kryo5.io.Input.readVarIntFlag(Input.java:480)
	at com.esotericsoftware.kryo.kryo5.io.Input.readString(Input.java:775)

FYI, The kryo I use:

private static Kryo kryo = new Kryo();
static {
    kryo.setReferences(true);
    kryo.setRegistrationRequired(false);
    CompatibleFieldSerializer.CompatibleFieldSerializerConfig config =
            new CompatibleFieldSerializer.CompatibleFieldSerializerConfig();
    config.setChunkedEncoding(true);
    config.setReadUnknownFieldData(true);
    kryo.setDefaultSerializer(new SerializerFactory.CompatibleFieldSerializerFactory(config));
    kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
    Log.set(LEVEL_DEBUG);
}

private <T> T deserialize(byte[] bytes) {
    ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
    Input input = new Input(byteArrayInputStream);
    input.close();
    return (T) kryo.readClassAndObject(input);
}

private byte[] serialize(Object obj) {
    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
    Output output = new Output(byteArrayOutputStream);
    kryo.writeClassAndObject(output, obj);
    output.close();
    return byteArrayOutputStream.toByteArray();
}

Environment:

  • OS: Windows 10
  • JDK Version: 8
  • Kryo Version: 5.1.0

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions