Invalid json: Type descriminator is output in json array literals when adding unrelated polymophic serialized property to class

[spand]

Describe the bug

This is output in the following "tags":["type":"kotlin.collections.ArrayList","2323"]

{"body":{"foo1":{"number":1,"stringOrInt":5},"name":"2","tags":["type":"kotlin.collections.ArrayList","2323"],"foo2":{"number":1,"stringOrInt":5}}}

Expected behavior
If I remove the stringOrInt property this is output:

{"body":{"foo1":{"number":1},"name":"2","tags":["2323"],"foo2":{"number":1}}}

It is not visible in my reduced case here but in my real case the equivalent of Foo instances also got a type descriminator in the json even though it should not be needed.

To Reproduce

import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.*
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.serializer
import java.io.ByteArrayOutputStream
import kotlin.test.Test

class JsonTest {

    private val format = Json {
        prettyPrint = false
        serializersModule = SerializersModule {
            polymorphic(SealedBase::class) {
                subclass(SealedInt::class, SealedIntSerializer)
                subclass(SealedString::class, SealedStringSerializer)
            }
        }
    }

    object SealedIntSerializer : JsonTransformingSerializer<SealedInt>(serializer()) {
        override fun transformSerialize(element: JsonElement): JsonElement {
            if (element is JsonObject) {
                val imageElement = element.getValue("image")
                return super.transformSerialize(imageElement)
            }
            return super.transformSerialize(element)
        }
    }

    object SealedStringSerializer : JsonTransformingSerializer<SealedString>(serializer()) {
        override fun transformSerialize(element: JsonElement): JsonElement {
            if (element is JsonObject) {
                val imageElement = element.getValue("image")
                return super.transformSerialize(imageElement)
            }
            return super.transformSerialize(element)
        }
    }

    @Test
    fun foo() {
        val card = BodyType.Foo(
            1,
            SealedInt(5),
        )
        val apiSquadResponseSuccess = Envelope(BodyType(card, "2", listOf("2323"), card))

        val baos = ByteArrayOutputStream()
        format.encodeToStream(Envelope.serializer(), apiSquadResponseSuccess, baos)
        println(baos.toString(Charsets.UTF_8))
    }
}

@Serializable
data class Envelope(
    val body: BodyType,
)

@Serializable
data class BodyType(
    val foo1: Foo,
    val name: String?,
    val tags: List<String>,
    val foo2: Foo,
) {
    @Serializable
    data class Foo(
        val number: Int,
        val stringOrInt: SealedBase,
    )

}

@Serializable
abstract class SealedBase

@Serializable
data class SealedString(val image: String) : SealedBase()

@Serializable
data class SealedInt(val image: Int) : SealedBase()

Environment

• Kotlin version: 1.7.10
• Library version: 1.4.1
• Kotlin platforms: JVM

[sandwwraith]

You can workaround this by changing SealedBase from abstract to sealed and removing SerializersModule from Json (as it is provided automatically for sealed classes)

I've just realized that you're using custom transformers, and they're likely the problem here: polymorphism should be able to add type key to the object, but your transformer return JsonPrimitive, and thus type key goes to the wrong place.

Why do you need to transform polymorphic objects into primitives? Do you realize that you won't be able to deserialize them back, as type info is lost?

[sandwwraith]

Somewhat related: #2049 (as value classes also have incorrect polymorphic type discriminator)

[spand]

Why do you need to transform polymorphic objects into primitives? Do you realize that you won't be able to deserialize them back, as type info is lost?

Sorry I missed this your question here. Yes we know but would like to be able to! What we are trying to do is similar to content-based-polymorphic-deserialization except that we want it to apply to a single json value that may be either a string or a number as you can see in the example. In our system this SealedBase is a widely used type and in order to get this behavior with content-based-polymorphic-deserialization we would have to create this hierarchy for every type that just use this type. That is a poor tradeoff and since we have to be format compatible with existing software our hands are a bit tied here.

Obviously it would be nice to have first class support for content-based-polymorphic-deserialization that changes the type of the value itself and not just of the enclosing class. They kind of taste similar and I cant see any real reason why one should not be able to perform content-based-polymorphic-(de)serialization on primitive values. Now that I think of it, it seems to be exactly what is needed to support enums as strings ?

[sandwwraith]

I think it's possible to use JsonContentPolymorphicSerializer now to deserialize primitive values polymorphically, as there's no additional shape checks. You'd still need JsonTransformingSerializer with both transformSerialize and transformDeserialize though. But at this point it is much easier just to write fully-cusom base class serializer like this: https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/json.md#under-the-hood-experimental

[sgammon]

@sandwwraith We just ran into this on latest (1.6.3 / 2.0.0-Beta4 as of this writing), and we are not using any custom transformers.

Output example:

  "conventions": {
    "@class": "kotlin.collections.LinkedHashMap",
    "JsSettings": {
      "@class": "dont.v1.JsSettings",
      "runtime": null,
      "sources": [
        "@class": "kotlin.collections.ArrayList"
      ]
    },

The kotlin.collections.ArrayList and kotlin.collections.LinkedHashMap seems to be another bug. Those are typed as simple List<...> and Map<...>. I will file these as separate bugs just in case this is an unrelated issue.

Edit: The bug is present on 1.9.22 / 1.6.2 as well.