Context

versions in use

• kotlin: 2.0.20
• kotlinx-serialization: 2.0.20
• kotlinx-io: 0.5.4

I’ve been following the kotlinx-serialization Custom formats guide to implement a custom binary protocol and have run into a wall …

The Decoder in question reads from a kotlinx-io Buffer and creates data class instances

The Issue

The decoder works for the most part except optional fields.

I’m trying to decode the Buffer into this type:

@Serializable
data class Foo(
    val one: Int,
    val two: Int,
    val three: Short,
    val four: Short = 10, // notice the default value
)

So the Buffer can sequentially contain either:

• 12 bytes / 4 fields

|int  |int  |short |short |
|bbbb |bbbb |bb    |bb    |

• or 10 bytes / 3 fields

|int  |int  |short |
|bbbb |bbbb |bb    |

The 12 bytes case is successfully decoded but the 10 bytes case fails with the following issue:

Buffer doesn't contain required number of bytes (size: 0, required: 2)
java.io.EOFException: Buffer doesn't contain required number of bytes (size: 0, required: 2)
    at kotlinx.io.Buffer.throwEof(Buffer.kt:163)
    at kotlinx.io.Buffer.readShort(Buffer.kt:103)
    at com.acme.corp.serde.BufferDecoder.decodeShort(BufferDecoder.kt:<line_number>)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeShortElement(AbstractDecoder.kt:52)
    at com.acme.corp.serde.OptionalFieldTest$Foo$$serializer.deserialize(OptionalFieldTest.kt:<line_number>)

Testcase

class OptionalFieldTest {
    @Test
    fun `options fields default when not enough bytes are available`() {
        val buffer =
            Buffer().apply {
                writeInt(1)
                writeInt(2)
                writeShort(3)
                // notice missing writeShort() here
            }
        val decoder = BufferDecoder(buffer)
        val decoded = decoder.decodeSerializableValue(serializer(), null)
        println("decoded into value=$decoded")
    }
}

Decode Implementation

class BufferDecoder(
    private val buffer: Buffer
    private var elementsCount: Int = 0,
) : AbstractDecoder() {
    private var elementIndex = 0
    override val serializersModule: SerializersModule = EmptySerializersModule()

    override fun decodeBoolean(): Boolean = buffer.readByte().toInt() != 0

    override fun decodeByte(): Byte = buffer.readByte()

    override fun decodeShort(): Short = buffer.readShort()

    override fun decodeInt(): Int = buffer.readInt()

    override fun decodeLong(): Long = buffer.readLong()

    override fun decodeFloat(): Float = buffer.readFloat()

    override fun decodeDouble(): Double = buffer.readDouble()

    override fun decodeChar(): Char = buffer.readByte().toInt().toChar()

    override fun decodeString(): String = buffer.readString()

    override fun decodeEnum(enumDescriptor: SerialDescriptor): Int = decodeByte().toInt()

    override fun decodeSequentially(): Boolean = true

    override fun decodeElementIndex(descriptor: SerialDescriptor): Int =
        if (elementIndex == elementsCount) {
            CompositeDecoder.DECODE_DONE
        } else {
            elementIndex++
        }

    override fun beginStructure(descriptor: SerialDescriptor): CompositeDecoder =
        BufferDecoder(
            buffer = buffer,
            elementsCount = descriptor.elementsCount,
        )
}

Question

I’m not sure how to tell kotlinx-serialization to use the default value for the last/remaining fields if the Buffer has been exhuasted