Exception of Invalid UTF-8

[0pow0]

Discussed in #7117

^{Originally posted by tzyrq February 22, 2022}
I'm using flatbuffers to serialize data through TCP socket from a cpp program to a java program.
What I'm doing on cpp side:

...
std::string str;
auto str_ = builder.CreateString (str);
auto foo = CreateFooTable(builder, str_, ...);
builder.Finish(foo);
uint8_t* buf = builder.GetBufferPointer();
SocketSend(buf);

What I'm doing on Java side:

byte[] bytes = ....;
SocketRead(bytes);
java.nio.ByteBuffer buf = java.nio.ByteBuffer.wrap(bytes);
FooTable foo = FooTable.getRootAsFooTable(buf);
int type = foo.infoType();
if (type == FooType.Type1) {
	UnionElement1 ele = (UnionElement1) foo.info(new UnionElement1());
	System.out.println(ele.Id()); /*<-- java.lang.IllegalArgumentException: Invalid UTF-8*/
}

Caused by: java.lang.IllegalArgumentException: Invalid UTF-8
	at com.google.flatbuffers.Utf8$DecodeUtil.handleThreeBytes(Utf8.java:185)
	at com.google.flatbuffers.Utf8Safe.decodeUtf8Array(Utf8Safe.java:175)
	at com.google.flatbuffers.Utf8Safe.decodeUtf8(Utf8Safe.java:290)
	at com.google.flatbuffers.Table.__string(Table.java:113)
	at com.google.flatbuffers.Table.__string(Table.java:94)

My .fbs file:

table UnionElement1 {
  filed1: string;
  filed2: string;
}

table UnionElement2 {
  filed1: string;
  filed2: string;
  filed3: string;
}

union FooType {
  Type1: UnionElement1,
  Type2: UnionElement2
}

table FooTable {
  delay: string;
  info: FooType;
}

root_type FooTable;

This whole procedure will be repeated by thousands of times, it works correctly at the begin but after that raise this exception.

flatbuffers version: 2.0.0

Any idea? 🥲 Thanks a lot!

[dbaileychess]

What is the string you are encoding?

Does SocketSend(buf) know the length of the buffer to send? Usually you would have to explicitly pass the length when sending over the socket.

Also, feel free to debug print the characters heres:

flatbuffers/java/com/google/flatbuffers/Utf8.java

Lines 176 to 189 in 5d101af

	static void handleThreeBytes(
	byte byte1, byte byte2, byte byte3, char[] resultArr, int resultPos)
	throws IllegalArgumentException {
	if (isNotTrailingByte(byte2)
	// overlong? 5 most significant bits must not all be zero
	|| (byte1 == (byte) 0xE0 && byte2 < (byte) 0xA0)
	// check for illegal surrogate codepoints
	|| (byte1 == (byte) 0xED && byte2 >= (byte) 0xA0)
	|| isNotTrailingByte(byte3)) {
	throw new IllegalArgumentException("Invalid UTF-8");
	}
	resultArr[resultPos] = (char)
	(((byte1 & 0x0F) << 12) | (trailingByteValue(byte2) << 6) | trailingByteValue(byte3));
	}