Possibility to nullify config used in ConfigMapping

[manofthepeace]

This heavily relates to; #845

I have some ConfigMapping that are feature related. Those feature are enabled/disabled via .enabled=[true|false]

those ConfigMapping are used "lazily" in ApplicationScoped bean that are using Instance<> and LookupIfProperty() mechanism in quarkus.

I wish that if the enabled flag is false, all of feature.**.* get nullified to I don't have a startup error. I do not know / want to specify manually all the configs to be nullyfied, nor I want to force people to go and remove all their feature related config, they should just toggle the enabled/disable so they don't have to fully reconfigure the feature.

I tried using an interceptor, but it does not work, as the ConfigMapping does not "load" values "/ usees interceptor..

but in soft the logic I had taht I would kike somehow to apply to the config mapping is the following;

public class SubSystemDisabler implements ConfigSourceInterceptor {
    private static final long serialVersionUID = 1L;
  
    private static final List<String> subSystems = List.of("feature", "feature_b", "feature_b");
    private static final UnaryOperator<String> ENABLED = name -> name.concat(".enabled");
    private static final UnaryOperator<String> STARTS_WITH_ANY = name -> subSystems.stream()
            .filter(name::startsWith).findFirst().orElse(null);
    @Override
    public ConfigValue getValue(ConfigSourceInterceptorContext context, String name) {
        String handle = STARTS_WITH_ANY.apply(name);
        if (handle != null) {
            String enabled = ENABLED.apply(handle);
            if (enabled.equals(name)) {
                return context.proceed(name);
            }
            ConfigValue subsystemEnabled = context.proceed(enabled);
            if (subsystemEnabled != null && !Boolean.parseBoolean(subsystemEnabled.getValue())) {
                log.debug("{} is false, disabling {}", enabled, name);
                return null;
            }
        }
        return context.proceed(name);
    }
}

Hope it is somewhat clear, I have created a little app here https://github.com/manofthepeace/opt-configmapping to show the structure, with a super simple mapping, in my app the config are far more complex with lots of nesting.

Obviously the above interceptor is wrong for the job, but its what I would like to achieve in a nutshell. Options welcome :) thanks

[radcortez]

In Quarkus you could wrap the mapping in a CDI Bean and use the annotations @IfBuildProfile or @IfBuildProperty to enable / disable the bean. This will also affect the declared mapping as:

@IfBuildProperty(name = "please.enable", stringValue = "yes")
@ApplicationScoped
public class MyMappingBean {
    @Inject
    MyMapping myMapping;
}

@ConfigMapping(prefix = "my.mapping")
public interface MyMapping {
    String value();
}

Would this work for you?

Another option is to use a SmallRyeConfigBuilderCustomizer and directly control which mappings are available in

smallrye-config/implementation/src/main/java/io/smallrye/config/SmallRyeConfigBuilder.java

Lines 830 to 832 in fc6de87

	public Map<Class<?>, Set<String>> getMappings() {
	return mappings;
	}

, but you are not going to have access to the full config context, because sources are not yet available here. We would require some sort of hook here:

smallrye-config/implementation/src/main/java/io/smallrye/config/ConfigMappingContext.java

Lines 47 to 62 in cd2e20d

	public ConfigMappingContext(
	final SmallRyeConfig config,
	final Map<Class<?>, Set<String>> mappings) {
	this.config = config;
	matchPropertiesWithEnv(mappings);
	for (Map.Entry<Class<?>, Set<String>> mapping : mappings.entrySet()) {
	Map<String, Object> mappingObjects = new HashMap<>();
	for (String prefix : mapping.getValue()) {
	applyPrefix(prefix);
	mappingObjects.put(prefix, constructMapping(mapping.getKey(), prefix));
	}
	this.mappings.put(mapping.getKey(), mappingObjects);
	}
	}

[manofthepeace]

I am using quarkus, and I cannot use build time stuff, as I cannot start supporting X builds with X different combinations of feature enablement.

Another thing is that when let say feature X is enabled, a ConfigSourceFactory loads other configs that fills the config mapping (+ user stuff configs), hence why when feature gets disabled, the ConfigMapping becomes partially filled, and complains about the missing stuff.

That's why my idea was to wrap all the configs in an optional, like in the example repo, and have a way to sort of mark them all as empty, so the config looks "ok" i.e. optional is empty, and feature.enabled=false.

[manofthepeace]

In relation to #845 I wonder if there could be something like this;

@ConfigMapping(prefix="conf", optional=true)

and a api sort of Optional<Server> serverConfig = config.getOptionalConfigMapping(Server.class);

Unsure how that work work for quarkus/cdi cases though..

[radcortez]

I am using quarkus, and I cannot use build time stuff, as I cannot start supporting X builds with X different combinations of feature enablement.

The customizer is SmallRye Config only, so it would work independently of Quarkus.

@ConfigMapping(prefix="conf", optional=true)

This may not be a good idea. We use the declared types to determine the behaviour of the mapping. CDI would be a problem, because you cannot use Optional at the injection point, so you could only retrieve these optional mappings with the programmatic API.

Also, when a declaration is reused in a tree as:

@ConfigMapping(optional = true)
interface OptionalMapping {

}

@ConfigMapping
interface NonOptionalMapping {
  OptionalMapping optionalMapping();

  Optional<OptionalMapping> optionalMappingAsOptional();
}

This looks a bit weird to me. Would the type declaration override the annotation or we have to validate that you always wrap the mapping in an Optional.

[manofthepeace]

Thanks for your answer.

The customizer is SmallRye Config only, so it would work independently of Quarkus.

I actually missed that from your previous message. That seems like it could be a really good solution, but as you mentioned if I do not have access to the raw config even, I cannot really know if a certain feature is available or not. When you said some hook in ConfigMappingContext.java I am not sure what it actually means though, does it mean it could be a way to get the config? A change in SR Config?

about the optional I said above;

CDI would be a problem, because you cannot use Optional at the injection point

Agree, no idea on that. I was naively thinking about something like Instance and use something like isResolvable or use something like Provider<> with it.

This looks a bit weird to me. Would the type declaration override the annotation or we have to validate that you always wrap the mapping in an Optional.

it does look weird. and I do not exactly know how it work work in a fully logical way. My goal is just to prevent half-provided config to break the app, i.e. make a half present optional be simply sort of ignored even if some configSource provides some values.

From what I understand and read, looks like the customizer is quite close of being able to make this work, and actually ignore configMappings.

[radcortez]

I actually missed that from your previous message. That seems like it could be a really good solution, but as you mentioned if I do not have access to the raw config even, I cannot really know if a certain feature is available or not. When you said some hook in ConfigMappingContext.java I am not sure what it actually means though, does it mean it could be a way to get the config? A change in SR Config?

Yes, that does not exist now. We would need to add it.

How is the configuration to enable / disable is defined? Is it the user that provides the configuration? Or it is some other logic that sets that configuration to enable / disable? Couldn't that logic be written in the customizer to add / remove mappings?

[manofthepeace]

It's a little convoluted but let me try to explain my flow;

1- there are features, feature can be enabled/disabled via a <feature>.enabled=<true|false> via env var in k8s
2- via a ConfigSourceFactory, check enabled feature isEnabled (via context.getValue and load their correspondent config from outside (or just external featureX.properties). If feature is disabled we dont bother loading the configs for the feature
3-User can override any properties via env_var or provide extra ones, or disable the feature altogether

My goal is that if the user wants to temporarily disable a feature he can jsut change the env var to featureX.disabled=false. Without bothering removing his extra featureX.customizaions="XXX". so when ready it can just be toggled back to true.

Currently if the feature is toggled FALSE, the ConfigSourceFactory will skip getting the config for that service, and the extra params that remain in a user conf will fill some of the things in the Optional (see linked project in OP) then startup will fail, until the user redeploys by commenting all the FeatureX properties in its env var.

Its not the end of the world obviously, but it would be nice to not have the risk of a feature changing because params were not reapplied properly, besides the basic enabled true false. Basically having a single feature flag, and not 1+ whatever is manually configured and/or overridden

[radcortez]

1- there are features, feature can be enabled/disabled via a <feature>.enabled=<true|false> via env var in k8s

Can you only enable / disable from env var?

[manofthepeace]

Yes, some are by default to "true" in the application.properties. Some needs to be toggled via env var.

[radcortez]

Ok, for now, you can use the same validation trick we discussed in Quarkus. Create your customizer and internally, create a new SmallRyeConfig instance from SmallRyeConfigBuilder. You just need to be careful with the internal instance and not discover / use the current customizer, or you will get in an endless loop.

With the config, you can now query for the expected configuration and add / remove mappings from the original builder. The downside is that you create some additional objects (and read the sources multiple times), but it is usually fast and maybe good enough until we have a proper solution.

[manofthepeace]

I tried the above and unfortunately I cannot remove the Customizer. I tried a little something like this

public class CustomConfigBuilder implements SmallRyeConfigBuilderCustomizer {
    @Override
    public void configBuilder(SmallRyeConfigBuilder builder) {
        var test = builder.isAddDiscoveredCustomizers();
        var conf = builder.build();
        boolean isEnabled = conf.getValue("feature.enabled", boolean.class);
        builder.withMappingIgnore("feature");
    }
}

compare to other the customizer does not have a setAddDiscoveredCustomizers() and the variable above it "true" so I get into the stackoverflow.

[manofthepeace]

I tried a bit more by modifying SR Config without much success;;

changes in SR config
1- customizers ArrayList to CopyOnWriteArrayList (customizer called 6 times and looks like it triggers a CoucurrentModification exception)
2- add api setAddDiscoveredCustomizers and withCustomizerIgnore

end code looks like this;

    public void configBuilder(SmallRyeConfigBuilder builder) {
        log.error("xxxxxxxx I AM HERE xxxxxxx ");
        var conf = builder.setAddDiscoveredCustomizers(false).withCustomizerIgnore(this).build();
        builder.setAddDiscoveredCustomizers(true);
        boolean isEnabled = conf.getValue("feature.enabled", boolean.class);
        if (!isEnabled) {
            builder.withMappingIgnore("feature.**");
        }
    }

I am using withMappingIgnore as noted in the javadoc, but still getting java.util.NoSuchElementException for each and every props inside the configmapping it is still trying to build via "buildMappings"

logs

2025-01-24 21:23:49,495 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:23:49,577 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (main) Loading feature 1 configurations
2025-01-24 21:23:56,468 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (main) Loading feature 1 configurations
2025-01-24 21:23:56,517 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:00,165 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:03,602 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:09,548 DEBUG [org.hib.val.int.uti.Version] (Quarkus Main Thread) HV000001: Hibernate Validator 8.0.2.Final
2025-01-24 21:24:09,873 ERROR [org.acme.con.CustomConfigBuilder] (Quarkus Main Thread) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:09,873 ERROR [org.acme.con.CustomConfigBuilder] (Quarkus Main Thread) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:09,880 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:09,880 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:13,994 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:13,994 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:14,023 ERROR [io.qua.run.Quarkus] (Quarkus Main Thread) Error running Quarkus: java.lang.ExceptionInInitializerError
Caused by: io.smallrye.config.ConfigValidationException: Configuration validation failed:
    java.util.NoSuchElementException: SRCFG00014: The config property feature.test is required but it could not be found in any config source

[radcortez]

The withMappingIgnore is used to ignore configuration names found in the mapping paths, so the mapping is still present in the builder.

Because the mappings are discovered and added automatically, you need to remove them explicitly by calling builder.getMappingsBuilder().getMappings().remove(Class).

[manofthepeace]

Thanks for taking the time. I have tried the following;

added getCustomizers to SmallRyeConfigBuilder
added setAddDiscoveredCustomizersto SmallryeConfigBuilder

Could not find an other way to not have an infinite loop

    @Override
    public void configBuilder(SmallRyeConfigBuilder builder) {
        log.error("xxxxxxxx I AM HERE xxxxxxx ");
        builder.getCustomizers().remove(this);
        var conf = builder.setAddDiscoveredCustomizers(false).build();
        boolean isEnabled = conf.getValue("feature.enabled", boolean.class);
        if (!isEnabled) {
            builder.getMappingsBuilder().getMappings().remove(MyConfig.class);
        }
    }

This gives the same result as above. NoSuchElementException for all properties.

In smallrye config I still needed to have the customizers list as a CopyOnWriteArrayList otherwise I could get the concurentModificationException

[radcortez]

I wrote a sample of my idea here:
radcortez/quarkus-config-playground@7de1888

Let me know if this works for you.