quarkus-oidc and scala extension integrations #37979
-
|
I am trying the https://quarkus.io/guides/security-keycloak-authorization guide on my quarkus scala project. I get the below exception when starting the quarkus application and I would like input for how to proceed in hunting the cause for this. I have no knowledge of vertx. I am using the io.quarkus:quarkus-resteasy-reactive-jackson dependency and have configured the io.quarkus.jackson.ObjectMapperCustomizer with the com.fasterxml.jackson.module.scala.DefaultScalaModule, so my rest endpoints worked (before adding the oidc extension). |
Beta Was this translation helpful? Give feedback.
Replies: 9 comments 26 replies
-
|
/cc @pedroigor (oidc), @sberyozkin (oidc) |
Beta Was this translation helpful? Give feedback.
-
|
@trym-moeller AFAIK Vert.x JsonObject is using Jackson internally and looks like the Scala mapper is interfering. My guess that having a reproducer will help, please create the one. CC @tsegismont @cescoffier |
Beta Was this translation helpful? Give feedback.
-
|
@trym-moeller I don't know what can be done here. Perhaps you should checkout the Vert.x source and see why I can't think of anything else that can resolve it. |
Beta Was this translation helpful? Give feedback.
-
|
@trym-moeller can you please also add instructions for configuring and running the Keycloak instance in a container please? |
Beta Was this translation helpful? Give feedback.
-
|
@tsegismont Hi Thomas, if you'd like to try the reproducer, you can start with empty properties and |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tsegismont Did you succeed reproducing the behavior or can I be of further assistence? Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
@sberyozkin The issue seems to be with vertx and jackson-module-scala as you said. I have tried to add the BodyCodecImpl class to my project (overriding the original jar file) and it is clear that an objectmapper using the DefaultScalaModule has already decoded the json object to a scala hashmap before the vertx BodyCodecImpl class gets to do its job. Why vertx ask BodyCodecImpl to do the decoding in this case and not when executing normal rest endpoints, I do not know... Anyone with a little vertx knowledge please? |
Beta Was this translation helpful? Give feedback.
-
|
@trym-moeller You can get the vertx source and debug and see what is going on. My guess is the Scala mapper affects the client side Jackson processing of Vertx Web Client. |
Beta Was this translation helpful? Give feedback.
-
|
I have gotten a little more knowledge after debugging... It seems that io.quarkus.vertx.runtime.jackson.QuarkusJacksonJsonCodec makes a copy of the quarkus managed objectmapper and adds its own (de)serializer modules, thus ensuring that the io.vertx.ext.web.codec.impl.BodyCodecImpl behaves. But when I add other deserializers (DefaultScalaModule), then the BodyCodecImpl misbehaves. Seems fragile, but what do I know. Any suggestions for further investigations? |
Beta Was this translation helpful? Give feedback.
-
|
@trym-moeller thanks, looks like you have a workaround which is a good news. |
Beta Was this translation helpful? Give feedback.
-
|
Well, not quite. Because when I then make a curl to the rest endpoint, I get the following exception (sorry for not having the full stack stace, the logger goes to std out, and it seems to be cut off). |
Beta Was this translation helpful? Give feedback.
-
|
@trym-moeller So it is obvious the Scala serializer is conflicting with what Vertx |
Beta Was this translation helpful? Give feedback.
-
|
The public static final Function<Buffer, JsonObject> JSON_OBJECT_DECODER = buff -> {
Object val = Json.decodeValue(buff);
if (val == null) {
return null;
}
if (val instanceof JsonObject) {
return (JsonObject) val;
}
throw new DecodeException("Invalid Json Object decoded as " + val.getClass().getName());
};The contract for this method is to return a /**
* Decode a given JSON buffer.
*
* @param buf the JSON buffer.
*
* @return a JSON element which can be a {@link JsonArray}, {@link JsonObject}, {@link String}, ...etc if the buffer contains an array, object, string, ...etc
* @throws DecodeException when there is a parsing or invalid mapping.
*/
public static Object decodeValue(Buffer buf) throws DecodeException {
return decodeValue(buf, Object.class);
}
/**
* Decode a given JSON buffer to a POJO of the given class type.
* @param buf the JSON buffer.
* @param clazz the class to map to.
* @param <T> the generic type.
* @return an instance of T
* @throws DecodeException when there is a parsing or invalid mapping.
*/
public static <T> T decodeValue(Buffer buf, Class<T> clazz) throws DecodeException {
return CODEC.fromBuffer(buf, clazz);
}To do so, it invokes the This works in all cases with the I'll try to make a reproducer test in Vert.x core and file an issue. I'll keep you informed. |
Beta Was this translation helpful? Give feedback.
-
|
Great analysis, thanks @tsegismont |
Beta Was this translation helpful? Give feedback.
-
|
I've created this simple Scala program: import com.fasterxml.jackson.module.scala.DefaultScalaModule
import io.vertx.core.json.Json
import io.vertx.core.json.jackson.DatabindCodec
object Hello {
def main(args: Array[String]): Unit = {
DatabindCodec.mapper().registerModule(new DefaultScalaModule)
val value = Json.decodeValue("{\"foo\": \"baz\"}")
println(value.getClass)
}
}It prints After applying this patch to Vert.x core: Index: src/main/java/io/vertx/core/json/jackson/DatabindCodec.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/main/java/io/vertx/core/json/jackson/DatabindCodec.java b/src/main/java/io/vertx/core/json/jackson/DatabindCodec.java
--- a/src/main/java/io/vertx/core/json/jackson/DatabindCodec.java (revision 74a8cc6d673b6485cc87b300f434d0ba53b5b9d0)
+++ b/src/main/java/io/vertx/core/json/jackson/DatabindCodec.java (date 1704477174862)
@@ -16,20 +16,14 @@
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
-import com.fasterxml.jackson.databind.module.SimpleModule;
import io.netty.buffer.ByteBufInputStream;
import io.vertx.core.buffer.Buffer;
import io.vertx.core.buffer.impl.BufferInternal;
import io.vertx.core.json.DecodeException;
import io.vertx.core.json.EncodeException;
-import io.vertx.core.json.JsonArray;
-import io.vertx.core.json.JsonObject;
import java.io.IOException;
import java.io.InputStream;
-import java.time.Instant;
-import java.util.List;
-import java.util.Map;
/**
* @author <a href="mailto:[email protected]">Julien Viet</a>
@@ -69,21 +63,21 @@
return prettyMapper;
}
+ @SuppressWarnings("unchecked")
@Override
public <T> T fromValue(Object json, Class<T> clazz) {
- T value = DatabindCodec.mapper.convertValue(json, clazz);
if (clazz == Object.class) {
- value = (T) adapt(value);
+ return (T) fromBuffer(toBuffer(json), Object.class);
}
- return value;
+ return DatabindCodec.mapper.convertValue(json, clazz);
}
+ @SuppressWarnings("unchecked")
public <T> T fromValue(Object json, TypeReference<T> type) {
- T value = DatabindCodec.mapper.convertValue(json, type);
if (type.getType() == Object.class) {
- value = (T) adapt(value);
+ return (T) fromBuffer(toBuffer(json), Object.class);
}
- return value;
+ return DatabindCodec.mapper.convertValue(json, type);
}
@Override
@@ -120,7 +114,11 @@
}
}
+ @SuppressWarnings("unchecked")
public static <T> T fromParser(JsonParser parser, Class<T> type) throws DecodeException {
+ if (type == Object.class) {
+ return (T) JacksonCodec.fromParser(parser, Object.class);
+ }
T value;
JsonToken remaining;
try {
@@ -134,25 +132,21 @@
if (remaining != null) {
throw new DecodeException("Unexpected trailing token");
}
- if (type == Object.class) {
- value = (T) adapt(value);
- }
return value;
}
+ @SuppressWarnings("unchecked")
private static <T> T fromParser(JsonParser parser, TypeReference<T> type) throws DecodeException {
- T value;
+ if (type.getType() == Object.class) {
+ return (T) JacksonCodec.fromParser(parser, Object.class);
+ }
try {
- value = DatabindCodec.mapper.readValue(parser, type);
+ return DatabindCodec.mapper.readValue(parser, type);
} catch (Exception e) {
throw new DecodeException("Failed to decode:" + e.getMessage(), e);
} finally {
close(parser);
}
- if (type.getType() == Object.class) {
- value = (T) adapt(value);
- }
- return value;
}
@Override
@@ -174,20 +168,4 @@
throw new EncodeException("Failed to encode as JSON: " + e.getMessage());
}
}
-
- private static Object adapt(Object o) {
- try {
- if (o instanceof List) {
- List list = (List) o;
- return new JsonArray(list);
- } else if (o instanceof Map) {
- @SuppressWarnings("unchecked")
- Map<String, Object> map = (Map<String, Object>) o;
- return new JsonObject(map);
- }
- return o;
- } catch (Exception e) {
- throw new DecodeException("Failed to decode: " + e.getMessage());
- }
- }
}The program prints This confirms the assumption in my previous comment. Note that if Vert.x core is patched, Quarkus must be patched too (because it uses its own JsonCodec implementation). |
Beta Was this translation helpful? Give feedback.
-
|
This seems like the correct solution, thanks for your investigation and work! Should I create an issue in https://github.com/eclipse-vertx/vert.x/issues ? @sberyozkin Can I create a feature request for the io.quarkus.vertx.runtime.jackson.QuarkusJacksonJsonCode to support the jackson DefaultScalaModule json (de)serializer? |
Beta Was this translation helpful? Give feedback.
-
|
I would wait before doing that. We could have a solution entirely upstream
Le lun. 8 janv. 2024, 09:50, Trym R. Møller ***@***.***> a
écrit :
… @tsegismont <https://github.com/tsegismont> ,
This seems like the correct solution, thanks for your investigation and
work!
@sberyozkin <https://github.com/sberyozkin> Can I create a feature
request for the io.quarkus.vertx.runtime.jackson.QuarkusJacksonJsonCode to
support the jackson DefaultScalaModule json (de)serializer?
—
Reply to this email directly, view it on GitHub
<#37979 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AALOLNV4H337CUGZEGPJ2A3YNOXL5AVCNFSM6AAAAABBKERBPOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DANBVGQYTI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
That is quite allright. Then I believe that you suggest to wait for a fix in vert.x that quarkus can utilize when upgrading its dependency to vert.x (and fix the QuarkusJacksonJsonCodec at update time as well). Do you need me to create an issue in the vert.x github issue tracker? |
Beta Was this translation helpful? Give feedback.
-
|
Hi Sergey @sberyozkin I am unsure how to proceed from here. Should I create an issue regarding the QuarkusJacksonJsonCodec class or? Best regards Trym |
Beta Was this translation helpful? Give feedback.
-
|
Please don't. Let me finish figuring out what we can do upstream. |
Beta Was this translation helpful? Give feedback.
-
|
Ok, good to know, that you are working on a solution. |
Beta Was this translation helpful? Give feedback.
-
|
Filed eclipse-vertx/vert.x#5058 and sent PR eclipse-vertx/vert.x#5057 When we come to an agreement on the solution upstream, I will file an issue in Quarkus and send a PR |
Beta Was this translation helpful? Give feedback.
-
|
@trym-moeller can you please try using this module instead of the default Scala module: import com.fasterxml.jackson.module.scala._
import com.fasterxml.jackson.module.scala.deser._
import com.fasterxml.jackson.module.scala.introspect._
class MyScalaModule
extends JacksonModule
with IteratorModule
with EnumerationModule
with OptionModule
with SeqModule
with IterableModule
with TupleModule
with MapModule
with SetModule
with ScalaNumberDeserializersModule
with ScalaObjectDeserializerModule
with ScalaAnnotationIntrospectorModule
with EitherModule
with SymbolModule {
override def getModuleName: String = "MyScalaModule"
}
object MyScalaModuleThe difference is it doesn't add |
Beta Was this translation helpful? Give feedback.
-
|
Hi Thomas @tsegismont I have tried your suggestion (added it to the reproducer in https://github.com/trym-moeller/quarkus-reproducer-37979), and it fails as originally with the io.vertx.core.json.DecodeException: Invalid Json Object decoded as scala.collection.immutable.HashMap exception. Thanks for your try! Best regards Trym |
Beta Was this translation helpful? Give feedback.
-
|
I gave it a try again, this is because the Quarkus Scala extension auto loads |
Beta Was this translation helpful? Give feedback.
-
|
Hi Thomas @tsegismont , Good catch! I am unsure what this indicates? Best regards Trym |
Beta Was this translation helpful? Give feedback.
-
|
Perhaps your custom Scala processor is missing this piece: |
Beta Was this translation helpful? Give feedback.
-
|
@tsegismont, sorry for not reading my last message through. I tried to json serialize the QuarkusIdentityProviderManagerImpl class (and there were no json serializer for that class). I have now modified my reproducer, so it does not serialize this class... and everything (until now) is working as expected. I am unsure what consequences to two hacks have, but I will try to include them in my original project and see. Thanks for you help until now. I am unsure, if the hacks should belong to me alone or if we can contribute them to the quarkus code base? Best regards Trym |
Beta Was this translation helpful? Give feedback.
-
I don't know about the consequences of overriding a processor. Ideally, a contribution to the Scala extension would be better. But then the Scala extension you are using is deprecated, so I don't believe an enhancement PR would be accepted. Maybe @gsmet has some thoughts about this? On using a custom Jackson Scala module, the "This class aggregates all of the feature modules into a single concrete class. Its use is recommended for new users and users who want things to "just work". If more customized support is desired, consult each of the constituent traits." The So I believe you should be fine if you deserialize JSON payloads to your own Scala types (like your |
Beta Was this translation helpful? Give feedback.
-
|
@tsegismont, major thanks for working with @trym-moeller and finding a solution, apologies I was not replying, I was on PTO |
Beta Was this translation helpful? Give feedback.
-
|
You're welcome |
Beta Was this translation helpful? Give feedback.
I don't know about the consequences of overriding a processor. Ideally, a contribution to the Scala extension would be better. But then the Scala extension you are using is deprecated, so I don't believe an enhancement PR would be accepted.
Maybe @gsmet has some thoughts about this?
On using a custom Jackson Scala module, the
DefaultScalaModuledoc reads:"This class aggregates all of the feature modules into a single concrete class. Its use…