Deserializing ResponseInputItem results in unknown state

[winterbe]

Hey there! 👋

I'm working with the OpenAI Java SDK for a couple of days now and really like it. The general design of the library is fantastic and I'm having quite a lot fun working with it. Thanks! 👍

I stumbled upon an issue which I think is a bug. Let me first clarify about my goals: I'm using the Responses API part of the library to implement Chat-Bot like conversations. My requirement is to use ResponseCreateParams.builder().store(false) and instead serialize conversation state locally by myself.

Here's an example of what I do:

val jsonMapper = jsonMapper()
val inputItems: List<ResponseInputItem> = ...
val jsonData = jsonMapper.writeValueAsString(inputItems)
// store jsonData locally

// later:
val deserializedInputItems = jsonMapper.readValue(nativeData, object : TypeReference<MutableList<ResponseInputItem>>() {})
// attach new user message and continue conversation

This works pretty well. However I've noticed the following behavior:

When a ResponseInputItem from a ResponseInputItem.Message gets serialized and deserialized again, it's in an unknown state. The message field is null but the _json() field still holds the correct structured data from the original message.

Here's are a Kotlin Test to demonstrate the behavior:

❌ This test fails with v3.0.3

@Test
fun testSerializeInputItem() {
    // given
    val message = ResponseInputItem.Message.builder()
        .role(ResponseInputItem.Message.Role.USER)
        .addInputTextContent("Test")
        .build()
    val inputItem1 = ResponseInputItem.ofMessage(message)

    // when
    val jsonMapper = jsonMapper()
    val inputItemJson = jsonMapper.writeValueAsString(inputItem1)
    val inputItem2 = jsonMapper.readValue(inputItemJson, ResponseInputItem::class.java)

    // then
    Assertions.assertEquals(inputItem1, inputItem2)
}

Test fails:

Expected :ResponseInputItem{message=Message{content=[ResponseInputContent{inputText=ResponseInputText{text=Test, type=input_text, additionalProperties={}}}], role=user, status=, type=, additionalProperties={}}}
Actual   :ResponseInputItem{_unknown={content=[{text=Test, type=input_text}], role=user}}

I couldn't reproduce this issue with other types of ResponseInputItems, e.g. for ResponseOutputMessage the test succeeds:

🟢 This test succeeds with v3.0.3

@Test
fun testSerializeOutputItem() {
    val message = ResponseOutputMessage.builder()
        .id("my_id")
        .status(ResponseOutputMessage.Status.COMPLETED)
        .addContent(
            ResponseOutputText.builder()
                .text("Test")
                .annotations(emptyList())
                .build()
        )
        .build()
    val inputItem1 = ResponseInputItem.ofResponseOutputMessage(message)

    val jsonMapper = jsonMapper()
    val inputItemJson = jsonMapper.writeValueAsString(inputItem1)

    val inputItem2 = jsonMapper.readValue(inputItemJson, ResponseInputItem::class.java)
    Assertions.assertEquals(inputItem1, inputItem2)
}

Could you please clarify if this a bug? Or am I doing something wrong / unintended here?

[winterbe]

Turns out I've found a solution to the problem right after writing this issue. 🙄

When setting status and type via Message.builder(), the test succeeds.

@Test
fun testSerializeInputItem() {
    val message = ResponseInputItem.Message.builder()
        .role(ResponseInputItem.Message.Role.USER)
        .addInputTextContent("Test")
        .status(ResponseInputItem.Message.Status.COMPLETED)
        .type(ResponseInputItem.Message.Type.MESSAGE)
        .build()
    val inputItem1 = ResponseInputItem.ofMessage(message)

    val jsonMapper = jsonMapper()
    val inputItemJson = jsonMapper.writeValueAsString(inputItem1)

    val inputItem2 = jsonMapper.readValue(inputItemJson, ResponseInputItem::class.java)
    Assertions.assertEquals(inputItem1, inputItem2)
}

Dunno if setting those fields should be mandatory?

However, I close the issue again.

Have a nice weekend everyone! 🙂

[TomerAberbach]

We do have a lot of roundtripping tests, but looks like we don't test this specific scenario.

We do test the latter scenario you described:

openai-java/openai-java-core/src/test/kotlin/com/openai/models/responses/ResponseInputItemTest.kt

Lines 107 to 127 in 94de167

	@Test
	fun ofMessageRoundtrip() {
	val jsonMapper = jsonMapper()
	val responseInputItem =
	ResponseInputItem.ofMessage(
	ResponseInputItem.Message.builder()
	.addInputTextContent("text")
	.role(ResponseInputItem.Message.Role.USER)
	.status(ResponseInputItem.Message.Status.IN_PROGRESS)
	.type(ResponseInputItem.Message.Type.MESSAGE)
	.build()
	)
	val roundtrippedResponseInputItem =
	jsonMapper.readValue(
	jsonMapper.writeValueAsString(responseInputItem),
	jacksonTypeRef<ResponseInputItem>(),
	)
	assertThat(roundtrippedResponseInputItem).isEqualTo(responseInputItem)
	}

It actually seems like potentially a bug that we even require this type. I think it should be set for the user automatically. I think the reason this happens is because the OpenAPI spec specifies the type property is optional, which seems odd because the description says "Always set to message".