-
Notifications
You must be signed in to change notification settings - Fork 626
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add a way to reorder elements during encoding #2722
Open
Chuckame
wants to merge
1
commit into
Kotlin:dev
Choose a base branch
from
Chuckame:reordering-encoder
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
247 changes: 247 additions & 0 deletions
247
core/commonMain/src/kotlinx/serialization/encoding/ReorderingCompositeEncoder.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,247 @@ | ||
/* | ||
* Copyright 2017-2024 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license. | ||
*/ | ||
|
||
package kotlinx.serialization.encoding | ||
|
||
import kotlinx.serialization.* | ||
import kotlinx.serialization.descriptors.* | ||
import kotlinx.serialization.modules.* | ||
|
||
|
||
/** | ||
* Encodes elements in a user-defined order managed by [mapElementIndex]. | ||
* | ||
* This encoder will replicate the behavior of a standard encoding, but calling the `encode*Element` methods in | ||
* the order defined by [mapElementIndex]. It first buffers each `encode*Element` calls in an array following | ||
* the given indexes using [mapElementIndex], then when [endStructure] is called, it encodes the buffered calls | ||
* in the expected order by replaying the previous calls on the given [compositeEncoderDelegate]. | ||
* | ||
* This encoder is stateful and not designed to be reused. | ||
* | ||
* @param compositeEncoderDelegate the [CompositeEncoder] to be used to encode the given descriptor's elements in the expected order. | ||
* @param encodedElementsCount The final number of elements to encode, which could be smaller than the original descriptor when [mapElementIndex] returns [SKIP_ELEMENT_INDEX] or when the index mapper has returned the same index twice. | ||
* @param mapElementIndex maps the element index to a new positional zero-based index. | ||
* The mapped index just helps to reorder the elements, | ||
* but the reordered `encode*Element` method calls will still pass the original element index. | ||
* If this mapper returns [SKIP_ELEMENT_INDEX] or -1, the element will be ignored and not encoded. | ||
* If this mapper provides the same index for multiple elements, | ||
* only the last one will be encoded as the previous ones will be overridden. | ||
*/ | ||
@ExperimentalSerializationApi | ||
public class ReorderingCompositeEncoder( | ||
encodedElementsCount: Int, | ||
private val compositeEncoderDelegate: CompositeEncoder, | ||
private val mapElementIndex: (SerialDescriptor, Int) -> Int, | ||
) : CompositeEncoder { | ||
private val bufferedCalls = Array<BufferedCall?>(encodedElementsCount) { null } | ||
|
||
public companion object { | ||
@ExperimentalSerializationApi | ||
public const val SKIP_ELEMENT_INDEX: Int = -1 | ||
} | ||
|
||
override val serializersModule: SerializersModule | ||
// No need to return a serializers module as it's not used during buffering | ||
get() = EmptySerializersModule() | ||
|
||
private data class BufferedCall( | ||
val originalElementIndex: Int, | ||
val encoder: () -> Unit, | ||
) | ||
|
||
private fun bufferEncoding( | ||
descriptor: SerialDescriptor, | ||
index: Int, | ||
encoder: () -> Unit | ||
) { | ||
val newIndex = mapElementIndex(descriptor, index) | ||
if (newIndex != SKIP_ELEMENT_INDEX) { | ||
bufferedCalls[newIndex] = BufferedCall(index, encoder) | ||
} | ||
} | ||
|
||
override fun endStructure(descriptor: SerialDescriptor) { | ||
bufferedCalls.forEach { fieldToEncode -> | ||
// In case of skipped fields, overridden fields (mapped to same index) or too big [encodedElementsCount], | ||
// the fieldToEncode may be null as no element was encoded for that index | ||
fieldToEncode?.encoder?.invoke() | ||
} | ||
compositeEncoderDelegate.endStructure(descriptor) | ||
} | ||
|
||
override fun encodeBooleanElement(descriptor: SerialDescriptor, index: Int, value: Boolean) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeBooleanElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeByteElement(descriptor: SerialDescriptor, index: Int, value: Byte) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeByteElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeCharElement(descriptor: SerialDescriptor, index: Int, value: Char) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeCharElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeDoubleElement(descriptor: SerialDescriptor, index: Int, value: Double) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeDoubleElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeFloatElement(descriptor: SerialDescriptor, index: Int, value: Float) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeFloatElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeIntElement(descriptor: SerialDescriptor, index: Int, value: Int) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeIntElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeLongElement(descriptor: SerialDescriptor, index: Int, value: Long) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeLongElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun <T : Any> encodeNullableSerializableElement( | ||
descriptor: SerialDescriptor, | ||
index: Int, | ||
serializer: SerializationStrategy<T>, | ||
value: T? | ||
) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeNullableSerializableElement(descriptor, index, serializer, value) | ||
} | ||
} | ||
|
||
override fun <T> encodeSerializableElement( | ||
descriptor: SerialDescriptor, | ||
index: Int, | ||
serializer: SerializationStrategy<T>, | ||
value: T | ||
) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeSerializableElement(descriptor, index, serializer, value) | ||
} | ||
} | ||
|
||
override fun encodeShortElement(descriptor: SerialDescriptor, index: Int, value: Short) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeShortElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeStringElement(descriptor: SerialDescriptor, index: Int, value: String) { | ||
bufferEncoding(descriptor, index) { | ||
compositeEncoderDelegate.encodeStringElement(descriptor, index, value) | ||
} | ||
} | ||
|
||
override fun encodeInlineElement(descriptor: SerialDescriptor, index: Int): Encoder { | ||
return BufferingInlineEncoder(descriptor, index) | ||
} | ||
|
||
override fun shouldEncodeElementDefault(descriptor: SerialDescriptor, index: Int): Boolean { | ||
return compositeEncoderDelegate.shouldEncodeElementDefault(descriptor, index) | ||
} | ||
|
||
private inner class BufferingInlineEncoder( | ||
private val descriptor: SerialDescriptor, | ||
private val elementIndex: Int, | ||
) : Encoder { | ||
private var encodeNotNullMarkCalled = false | ||
|
||
override val serializersModule: SerializersModule | ||
get() = [email protected] | ||
|
||
private fun bufferEncoding(encoder: Encoder.() -> Unit) { | ||
bufferEncoding(descriptor, elementIndex) { | ||
compositeEncoderDelegate.encodeInlineElement(descriptor, elementIndex).apply { | ||
if (encodeNotNullMarkCalled) { | ||
encodeNotNullMark() | ||
} | ||
encoder() | ||
} | ||
} | ||
} | ||
|
||
override fun encodeNotNullMark() { | ||
encodeNotNullMarkCalled = true | ||
} | ||
|
||
override fun <T : Any> encodeNullableSerializableValue(serializer: SerializationStrategy<T>, value: T?) { | ||
bufferEncoding { encodeNullableSerializableValue(serializer, value) } | ||
} | ||
|
||
override fun <T> encodeSerializableValue(serializer: SerializationStrategy<T>, value: T) { | ||
bufferEncoding { encodeSerializableValue(serializer, value) } | ||
} | ||
|
||
override fun encodeBoolean(value: Boolean) { | ||
bufferEncoding { encodeBoolean(value) } | ||
} | ||
|
||
override fun encodeByte(value: Byte) { | ||
bufferEncoding { encodeByte(value) } | ||
} | ||
|
||
override fun encodeChar(value: Char) { | ||
bufferEncoding { encodeChar(value) } | ||
} | ||
|
||
override fun encodeDouble(value: Double) { | ||
bufferEncoding { encodeDouble(value) } | ||
} | ||
|
||
override fun encodeEnum(enumDescriptor: SerialDescriptor, index: Int) { | ||
bufferEncoding { encodeEnum(enumDescriptor, index) } | ||
} | ||
|
||
override fun encodeFloat(value: Float) { | ||
bufferEncoding { encodeFloat(value) } | ||
} | ||
|
||
override fun encodeInt(value: Int) { | ||
bufferEncoding { encodeInt(value) } | ||
} | ||
|
||
override fun encodeLong(value: Long) { | ||
bufferEncoding { encodeLong(value) } | ||
} | ||
|
||
@ExperimentalSerializationApi | ||
override fun encodeNull() { | ||
bufferEncoding { encodeNull() } | ||
} | ||
|
||
override fun encodeShort(value: Short) { | ||
bufferEncoding { encodeShort(value) } | ||
} | ||
|
||
override fun encodeString(value: String) { | ||
bufferEncoding { encodeString(value) } | ||
} | ||
|
||
override fun beginStructure(descriptor: SerialDescriptor): CompositeEncoder { | ||
unexpectedCall(::beginStructure.name) | ||
} | ||
|
||
override fun encodeInline(descriptor: SerialDescriptor): Encoder { | ||
unexpectedCall(::encodeInline.name) | ||
} | ||
|
||
private fun unexpectedCall(methodName: String): Nothing { | ||
// This method is normally called from within encodeSerializableValue or encodeNullableSerializableValue which is buffered, so we should never go here during buffering as it will be delegated to the concrete CompositeEncoder | ||
throw UnsupportedOperationException("Non-standard usage of ${CompositeEncoder::class.simpleName}: $methodName should be called from within encodeSerializableValue or encodeNullableSerializableValue") | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that the implementation of this should be equivalent to:
thisEncoder. encodeSerializableElement(MyData. serializer. descriptor, 0, MyInt. serializer(), myInt)
Including the support of nested structs (nested structs always go through
encodeSerializableElement
)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is working like this after reordering, or maybe I misunderstood
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about:
The way I read it, this at least can be implemented using
encodeInlineElement(d, 0)
on the composite encoder for outer. Which would then mean that the inner value is serialized by callingencodeSerializableValue()
on that (BufferingInlineEncoder
) encoder.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test has been successful with your example, so I still don't get your point https://github.com/Kotlin/kotlinx.serialization/pull/2722/files#r1648164815
No, here are the steps:
encodeInlineElement(d, 0)
-> returnsBufferingInlineEncoder
with descriptor ofMiddle
and index 0, but does not buffer anything for the moment (this is the role ofBufferingInlineEncoder
)BufferingInlineEncoder.encodeSerializableValue
is called, so nothing more is done, no nested call is done even forInner
thanks tobufferEncoding
bufferingcompositeEncoderDelegate.encodeInlineElement(descriptor, elementIndex).encodeSerializableValue(serializer, value)
endStructure
is called, replays the elements encoding, so theInner
will be reached out only at this moment