[BUG] Exception in native build with jackson deserialization

[altro3]

Expected Behavior

Everything should work without errors :-). Must be next logs:

16:30:48.030 INFO  [main] i.m.c.e.DefaultEnvironment [DefaultEnvironment.java:159] - Established active environments: [local]
16:30:48.200 INFO  [main] c.m.b.c.MyEntityProperties [MyEntityProperties.java:19] - prop = test property
16:30:48.363 INFO  [main] c.m.b.c.MyEntityConfig [MyEntityConfig.java:68] - Read ConfigData[variant=VAR1, params=[val1, val2], params2=[122.23, 2222.56], items=[ConfigItem[prop1=true, prop2=3434.34], ConfigItem[prop1=false, prop2=3434.34]]]
16:30:48.379 INFO  [main] c.m.b.c.MyEntityConfig [MyEntityConfig.java:68] - Read ConfigData[variant=VAR2, params=[val1, val2], params2=[122.23, 2222.56], items=[ConfigItem[prop1=true, prop2=3434.34], ConfigItem[prop1=false, prop2=3434.34]]]
16:30:48.379 INFO  [main] c.m.b.s.MyEntityService [MyEntityService.java:52] - Entity MyEntity(id=0, field=field, status=ACTIVE) created
16:30:48.385 INFO  [main] c.m.b.s.MyEntityService [MyEntityService.java:59] - Test property: test property
16:30:48.385 INFO  [main] c.m.b.s.MyEntityService [MyEntityService.java:63] - Found [] entities
16:30:48.601 INFO  [main] i.m.l.PropertiesLoggingLevelsConfigurer [PropertiesLoggingLevelsConfigurer.java:107] - Setting log level 'INFO' for logger: 'root'
16:30:48.833 INFO  [main] i.m.runtime.Micronaut [Micronaut.java:95] - Startup completed in 964ms. Server Running: http://localhost:8080

Actual Behaviour

Look at the example: now everything works fine if you start the service in the standard way and an error occurs when the service starts in the native build.

Now, when trying to deserialize objects in a native build, such an exception:

15:41:36.675 ERROR [main] c.m.b.c.MyEntityConfig [MyEntityConfig.java:71] - Can't read config for variant VAR1
com.fasterxml.jackson.databind.JsonMappingException: Failed to access RecordComponents of type `com.micronaut.bug.config.data.ConfigData`
 at [Source: (URL); line: 1, column: 1]
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:268)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
        at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
        at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:591)
        at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4733)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4594)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3479)
        at com.micronaut.bug.config.MyEntityConfig.myConfigs(MyEntityConfig.java:67)
        at com.micronaut.bug.config.$MyEntityConfig$MyConfigs0$Definition.build(Unknown Source)
        at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2336)
        at io.micronaut.context.DefaultBeanContext.getScopedBeanForDefinition(DefaultBeanContext.java:2922)
        at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2822)
        at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2782)
        at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1638)
        at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:1933)
        at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1189)
        at com.micronaut.bug.service.$MyEntityService$Definition.build(Unknown Source)
        at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2336)
        at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:3281)
        at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:3267)
        at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2820)
        at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2782)
        at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1638)
        at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:1933)
        at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1189)
        at com.micronaut.bug.controller.$MyEntityController$Definition.build(Unknown Source)
        at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2336)
        at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:3281)
        at io.micronaut.context.DefaultBeanContext.loadContextScopeBean(DefaultBeanContext.java:2664)
        at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1932)
        at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:237)
        at io.micronaut.context.DefaultBeanContext.readAllBeanDefinitionClasses(DefaultBeanContext.java:3453)
        at io.micronaut.context.DefaultBeanContext.finalizeConfiguration(DefaultBeanContext.java:3883)
        at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:329)
        at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:183)
        at io.micronaut.runtime.Micronaut.start(Micronaut.java:72)
        at io.micronaut.runtime.Micronaut.run(Micronaut.java:313)
        at io.micronaut.runtime.Micronaut.run(Micronaut.java:299)
        at com.micronaut.bug.Application.main(Application.java:23)
Caused by: java.lang.IllegalArgumentException: Failed to access RecordComponents of type `com.micronaut.bug.config.data.ConfigData`
        at com.fasterxml.jackson.databind.jdk14.JDK14Util$RecordAccessor.recordComponents(JDK14Util.java:124)
        at com.fasterxml.jackson.databind.jdk14.JDK14Util$RecordAccessor.getRecordFieldNames(JDK14Util.java:78)
        at com.fasterxml.jackson.databind.jdk14.JDK14Util.getRecordFieldNames(JDK14Util.java:27)
        at com.fasterxml.jackson.databind.introspect.DefaultAccessorNamingStrategy$RecordNaming.<init>(DefaultAccessorNamingStrategy.java:531)
        at com.fasterxml.jackson.databind.introspect.DefaultAccessorNamingStrategy$Provider.forRecord(DefaultAccessorNamingStrategy.java:448)
        at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:185)
        at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forDeserialization(BasicClassIntrospector.java:104)
        at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forDeserialization(BasicClassIntrospector.java:11)
        at com.fasterxml.jackson.databind.DeserializationConfig.introspect(DeserializationConfig.java:907)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:324)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
        ... 38 common frames omitted

Steps To Reproduce

I used the json format for some additional settings in my project. To read the settings, I create my own custom JsonMapper.

ConfigData:

package com.micronaut.bug.config.data;

import io.micronaut.core.annotation.Introspected;

import java.util.List;

@Introspected
public record ConfigData(
        VariantEnum variant,
        List<String> params,
        List<Double> params2,
        List<ConfigItem> items
) {

    @Introspected
    public record ConfigItem(
            boolean prop1,
            double prop2
    ) {
    }
}

VariantEnum:

package com.micronaut.bug.config.data;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonValue;
import lombok.Getter;

import java.util.HashMap;
import java.util.Map;

@Getter
public enum VariantEnum {

    VAR1(1, "var1"),
    VAR2(2, "var2"),
    ;

    public static final Map<Byte, VariantEnum> BY_ID;
    public static final Map<String, VariantEnum> BY_CODE;

    static {

        var byId = new HashMap<Byte, VariantEnum>();
        var byCode = new HashMap<String, VariantEnum>();
        for (var value : values()) {
            byId.put(value.id, value);
            byCode.put(value.code, value);
        }

        BY_ID = Map.copyOf(byId);
        BY_CODE = Map.copyOf(byCode);
    }

    @Getter(onMethod_ = @JsonValue)
    public final byte id;
    public final String code;

    VariantEnum(int id, String code) {
        this.id = (byte) id;
        this.code = code;
    }

    public static VariantEnum byCode(String code) {
        var value = BY_CODE.get(code);
        if (value == null) {
            throw new IllegalArgumentException("Unknown code: " + code);
        }
        return value;
    }

    @JsonCreator(mode = JsonCreator.Mode.DELEGATING)
    public static VariantEnum byId(byte id) {
        var value = BY_ID.get(id);
        if (value == null) {
            throw new IllegalArgumentException("Unknown id: " + id);
        }
        return value;
    }
}

MyEntityConfig:

package com.micronaut.bug.config;

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.MapperFeature;
import com.fasterxml.jackson.databind.PropertyNamingStrategies;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.util.StdDateFormat;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import com.fasterxml.jackson.module.paramnames.ParameterNamesModule;
import com.micronaut.bug.config.data.ConfigData;
import com.micronaut.bug.config.data.VariantEnum;
import io.micronaut.context.annotation.Bean;
import io.micronaut.context.annotation.Factory;
import io.micronaut.core.util.StringUtils;
import jakarta.inject.Named;
import lombok.extern.slf4j.Slf4j;

import javax.annotation.PostConstruct;
import java.util.EnumMap;
import java.util.Locale;
import java.util.Map;

@Slf4j
@Factory
public class MyEntityConfig {

    @Bean
    @Named("myConfigs")
    public Map<VariantEnum, ConfigData> myConfigs() {

        var mapper = JsonMapper.builder()
                .defaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_NULL, JsonInclude.Include.NON_NULL))
                .defaultDateFormat(StdDateFormat.getDateTimeInstance())
                .propertyNamingStrategy(PropertyNamingStrategies.LOWER_CAMEL_CASE)
                .visibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY)
                .visibility(PropertyAccessor.CREATOR, JsonAutoDetect.Visibility.ANY)
                .defaultLocale(Locale.US)
                .enable(MapperFeature.ACCEPT_CASE_INSENSITIVE_ENUMS)
                .enable(JsonParser.Feature.ALLOW_COMMENTS)
                .disable(
                        SerializationFeature.INDENT_OUTPUT,
                        SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS,
                        SerializationFeature.WRITE_DATES_AS_TIMESTAMPS
                )
                .disable(
                        DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES,
                        DeserializationFeature.READ_DATE_TIMESTAMPS_AS_NANOSECONDS
                )
                .addModule(new JavaTimeModule())
                .addModule(new ParameterNamesModule(JsonCreator.Mode.PROPERTIES))
                .build();

        var myConfigs = new EnumMap<VariantEnum, ConfigData>(VariantEnum.class);

        for (var variant : VariantEnum.values()) {
            try {
                var fileUrl = ClassLoader.getSystemResource("configs/" + variant.code + ".json");
                if (fileUrl == null) {
                    continue;
                }
                var configData = mapper.readValue(fileUrl, ConfigData.class);
                log.info("Read {}", configData);
                myConfigs.put(variant, configData);
            } catch (Throwable t) {
                log.error("Can't read config for variant {}", variant, t);
            }
        }

        return myConfigs;
    }


    @PostConstruct
    public void init() {
        System.setProperty("system.property", StringUtils.EMPTY_STRING);
    }
}

Environment Information

• Windows 10
• JDK 17

Example Application

https://github.com/altro3/micronaut3-bug

Version

3.3.4

[altro3]

By the way, if you refuse records and use the standard @JsonCreator, then another error occurs.

Steps to repsroduce:

In class ConfigData change code from records to classes:

@AllArgsConstructor
@Data
@Introspected
public class ConfigData {

    private VariantEnum variant;
    private List<String> params;
    private List<Double> params2;
    private List<ConfigItem> items;

    @AllArgsConstructor
    @Data
    @Introspected
    public static class ConfigItem {
        private boolean prop1;
        private double prop2;
    }
}

And again, everything works fine with a standard launch, but when I run it in a native build, I see this error (different error):

16:45:53.735 ERROR [main] c.m.b.c.MyEntityConfig [MyEntityConfig.java:71] - Can't read config for variant VAR1
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `com.micronaut.bug.config.data.ConfigData` (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (URL); line: 2, column: 3]
        at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1764)
        at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400)
        at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1209)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1415)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3479)
        at com.micronaut.bug.config.MyEntityConfig.myConfigs(MyEntityConfig.java:67)
        at com.micronaut.bug.config.$MyEntityConfig$MyConfigs0$Definition.build(Unknown Source)
        at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2336)
        at io.micronaut.context.DefaultBeanContext.getScopedBeanForDefinition(DefaultBeanContext.java:2922)
        at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2822)
        at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2782)
        at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1638)
        at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:1933)
        at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1189)
        at com.micronaut.bug.service.$MyEntityService$Definition.build(Unknown Source)
        at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2336)
        at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:3281)
        at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:3267)
        at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2820)
        at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2782)
        at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1638)
        at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:1933)
        at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1189)
        at com.micronaut.bug.controller.$MyEntityController$Definition.build(Unknown Source)
        at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2336)
        at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:3281)
        at io.micronaut.context.DefaultBeanContext.loadContextScopeBean(DefaultBeanContext.java:2664)
        at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1932)
        at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:237)
        at io.micronaut.context.DefaultBeanContext.readAllBeanDefinitionClasses(DefaultBeanContext.java:3453)
        at io.micronaut.context.DefaultBeanContext.finalizeConfiguration(DefaultBeanContext.java:3883)
        at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:329)
        at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:183)
        at io.micronaut.runtime.Micronaut.start(Micronaut.java:72)
        at io.micronaut.runtime.Micronaut.run(Micronaut.java:313)
        at io.micronaut.runtime.Micronaut.run(Micronaut.java:299)
        at com.micronaut.bug.Application.main(Application.java:23)

[lost22git]

Maybe it's caused by not registering BeanIntrospectionModule

[altro3]

@test210820 Hi! Thank you for your answer, but no, problem not in this. As you can see, if I do not use GraalVM, then everything works great. That is, the problem occurs only with native build. The second moment - I still checked your version and added this module to my custom mapper:

                .addModules(new JavaTimeModule(), new ParameterNamesModule(JsonCreator.Mode.PROPERTIES), new BeanIntrospectionModule())

But the miracle did not happen, the behavior of the program did not change

[yawkat]

You can't use your own ObjectMapper with native-image right now due to a jackson bug. The workaround for the bug is in #6770, but it only works if you use the ObjectMapper provided by micronaut. Also, as mentioned, you need the BeanIntrospectionModule.

@ReflectiveAccess on the bean should work around this issue.

[altro3]

@yawkat thanks! I made some chnages in my code after your comment:

1. Use default micronaut objectMapper as a base:

• added some common settings to application.yml

jackson:
  property-naming-strategy: LOWER_CAMEL_CASE
  locale: en_US
  date-format: yyyy-MM-dd'T'HH:mm:ss.SSS
  mapper:
    ACCEPT_CASE_INSENSITIVE_ENUMS: true
  serialization:
    INDENT_OUTPUT: false
    WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS: false
    WRITE_DATES_AS_TIMESTAMPS: false
  deserialization:
    FAIL_ON_UNKNOWN_PROPERTIES: false
    READ_DATE_TIMESTAMPS_AS_NANOSECONDS: false
  serialization-inclusion: non_null

• create my own object mapper as a copy of default objectMapper (because I don't want to allow comments anywhere else, only in my config):

  @Bean
  @Named("myConfigs")
  public Map<VariantEnum, ConfigData> myConfigs(ObjectMapper objectMapper) {

      var myMapper = objectMapper.copy();
      myMapper.enable(JsonParser.Feature.ALLOW_COMMENTS);
   ....

2. Add @ReflectiveAccess annotation to my VariantEnum

After these changes both variants works fine. Thanks for help!

I think it's worth describing in the documentation the bug of the standard object mapper and the fact that you always need to create your custom object mapper only as a copy of the default Micronaut, otherwise problems may arise in the native assembly.

[yawkat]

not sure, if you're using your own object mapper with native image, the presence of micronaut is practically irrelevant. The error itself is a jackson bug though that I plan to fix

[altro3]

Understood. Then we are waiting for the Jackson fix

[yawkat]

FasterXML/jackson-databind#3417

[yawkat]

3.8 will use jackson 2.14