Skip to content
This repository was archived by the owner on Sep 11, 2025. It is now read-only.

Conversation

@mattjohnsonpint
Copy link
Contributor

Fixes #903

@mattjohnsonpint mattjohnsonpint requested review from a team and Copilot June 24, 2025 15:53
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR ensures that any byte sequence produced for inference history is valid UTF-8 by sanitizing it before storage.

  • Refactored getInferenceDataJson to collect serialized bytes in a single variable.
  • Added a call to utils.SanitizeUTF8 before returning the result.
  • Updated CHANGELOG with the new fix entry.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
runtime/db/inferencehistory.go Unified byte handling in getInferenceDataJson and applied UTF-8 sanitization
CHANGELOG.md Added entry for the UTF-8 sanitization fix for inference history
Comments suppressed due to low confidence (3)

runtime/db/inferencehistory.go:164

  • [nitpick] Consider renaming the local variable bytes to avoid shadowing the bytes package and improve readability (e.g., data or buf).
	var bytes []byte

runtime/db/inferencehistory.go:178

  • Add unit tests to verify that invalid UTF-8 byte sequences are correctly sanitized by SanitizeUTF8, ensuring coverage for this new behavior.
	return utils.SanitizeUTF8(bytes), nil

runtime/db/inferencehistory.go:160

  • Update the function comment to mention that output is now sanitized for valid UTF-8 sequences using SanitizeUTF8.
	// If the value is a byte slice or string, it must already have been serialized as JSON.

@mattjohnsonpint mattjohnsonpint enabled auto-merge (squash) June 24, 2025 15:54
@mattjohnsonpint mattjohnsonpint merged commit 5ef3fd4 into main Jun 24, 2025
33 checks passed
@mattjohnsonpint mattjohnsonpint deleted the mjp/utf8 branch June 24, 2025 15:58
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

ERROR: invalid byte sequence for encoding \"UTF8\": 0xf8 (SQLSTATE 22021)

3 participants