How to set a Quarkus vault property with a value from a file

[raheal]

This is in relation to updating config source and overriding a Quarkus Vault property from a Config Source

I have a requirement to set a property in my Quarkus app, whereby the value of this property (a quarkus-vault property) needs to read from a file. For example, in the microprofile-config.properties file, this would be an example of such property:

myProperty.val=<value_from_a_file>

One thing I have tried is to create a ConfigSource, which by creating a hashmap and setting the property there, it works fine. I referred to this web page:

https://quarkus.io/guides/config-extending-support

Here is an example of the ConfigSource implementation:

public class SampleConfigSource implements ConfigSource {

private static final Map<String, String> configuration = new HashMap<>();
private static final Logger LOGGER = LoggerFactory.getLogger(SampleConfigSource.class);
private static final String secretFilePath = ConfigProvider.getConfig().getValue("application.vault.secret.file", String.class);

static {
    try {
        // read the value from the file and put it in the configuration map
        final String content = Files.readAllLines(Paths.get(secretFilePath)).get(0);
        configuration.put("quarkus.vault.authentication.app-role.secret-id", content);
    } catch (Exception e) {
        LOGGER.error(e.getMessage(), e);
    }
}

@Override
public Set<String> getPropertyNames() {
    return configuration.keySet();
}

@Override
public String getValue(String s) {
    return configuration.get(s);
}

@Override
public String getName() {
    return SampleConfigSource.class.getName();
   }
  }

src/main/resources/META-INF/Services/org.eclipse.microprofile.config.spi.ConfigSource
com.example.config.SampleConfigSource

src/main/resources/META-INF/microprofile-config.properties
....
application.vault.secret.file=external/secret.txt
quarkus.vault.connect-timeout=30S
quarkus.vault.read-timeout=30S
quarkus.vault.url=https://prod-vault
quarkus.vault.authentication.app-role.role-id=abcde
....

Now the problem is that the microprofile-config.properties has the main properties for setting up vault. It also requires the quarkus.vault.authentication.app-role.secret-id property as well, but because I'm reading this value from a file, I've added it to the SampleConfigSource class. I would have thought that the property would have initialised fine on bootup, but instead I get the following stack trace error:

15:37:05 ERROR [io.quarkus.runtime.Application] Failed to start application (with profile dev): java.lang.UnsupportedOperationException: unknown authType null
at io.quarkus.vault.runtime.VaultAuthManager.login(VaultAuthManager.java:165)
at io.quarkus.vault.runtime.VaultAuthManager.vaultLogin(VaultAuthManager.java:145)
at io.quarkus.vault.runtime.VaultAuthManager.login(VaultAuthManager.java:116)
at io.quarkus.vault.runtime.VaultAuthManager_Subclass.login$$superforward1(Unknown Source)
at io.quarkus.vault.runtime.VaultAuthManager_Subclass$$function$$4.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.proceed(InvocationInterceptor.java:62)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.monitor(InvocationInterceptor.java:51)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
at io.quarkus.vault.runtime.VaultAuthManager_Subclass.login(Unknown Source)
at io.quarkus.vault.runtime.VaultAuthManager.login(VaultAuthManager.java:95)
at io.quarkus.vault.runtime.VaultAuthManager.getClientToken(VaultAuthManager.java:79)
at io.quarkus.vault.runtime.VaultAuthManager_Subclass.getClientToken$$superforward1(Unknown Source)
at io.quarkus.vault.runtime.VaultAuthManager_Subclass$$function$$10.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.proceed(InvocationInterceptor.java:62)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.monitor(InvocationInterceptor.java:51)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
at io.quarkus.vault.runtime.VaultAuthManager_Subclass.getClientToken(Unknown Source)
at io.quarkus.vault.runtime.VaultKvManager.readSecret(VaultKvManager.java:36)
at io.quarkus.vault.runtime.VaultKvManager_Subclass.readSecret$$superforward1(Unknown Source)
at io.quarkus.vault.runtime.VaultKvManager_Subclass$$function$$4.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.proceed(InvocationInterceptor.java:62)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.monitor(InvocationInterceptor.java:51)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
at io.quarkus.vault.runtime.VaultKvManager_Subclass.readSecret(Unknown Source)
at io.quarkus.vault.runtime.VaultKvManager_ClientProxy.readSecret(Unknown Source)
at io.quarkus.vault.runtime.config.VaultConfigSource.fetchSecrets(VaultConfigSource.java:131)
at io.quarkus.vault.runtime.config.VaultConfigSource.lambda$fetchSecrets$2(VaultConfigSource.java:127)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
at io.quarkus.vault.runtime.config.VaultConfigSource.fetchSecrets(VaultConfigSource.java:127)
at io.quarkus.vault.runtime.config.VaultConfigSource.lambda$fetchSecrets$0(VaultConfigSource.java:120)
at java.base/java.util.Optional.ifPresent(Optional.java:183)
at io.quarkus.vault.runtime.config.VaultConfigSource.fetchSecrets(VaultConfigSource.java:120)
at io.quarkus.vault.runtime.config.VaultConfigSource.fetchSecretsFirstTime(VaultConfigSource.java:103)
at io.quarkus.vault.runtime.config.VaultConfigSource.getSecretConfig(VaultConfigSource.java:81)
at io.quarkus.vault.runtime.config.VaultConfigSource.getValue(VaultConfigSource.java:62)
at io.smallrye.config.ConfigValueConfigSourceWrapper.getConfigValue(ConfigValueConfigSourceWrapper.java:20)
at io.smallrye.config.SmallRyeConfigSources.getValue(SmallRyeConfigSources.java:29)
at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
at io.smallrye.config.SecretKeysConfigSourceInterceptor.getValue(SecretKeysConfigSourceInterceptor.java:22)
at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
at io.smallrye.config.FallbackConfigSourceInterceptor.getValue(FallbackConfigSourceInterceptor.java:24)
at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
at io.smallrye.config.RelocateConfigSourceInterceptor.getValue(RelocateConfigSourceInterceptor.java:29)
at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
at io.smallrye.config.RelocateConfigSourceInterceptor.getValue(RelocateConfigSourceInterceptor.java:29)
at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
at io.smallrye.config.ProfileConfigSourceInterceptor.convertProfile(ProfileConfigSourceInterceptor.java:120)
at io.smallrye.config.ProfileConfigSourceInterceptor.(ProfileConfigSourceInterceptor.java:44)
at io.smallrye.config.SmallRyeConfigBuilder$1.getInterceptor(SmallRyeConfigBuilder.java:180)
at io.smallrye.config.SmallRyeConfigBuilder$InterceptorWithPriority.getInterceptor(SmallRyeConfigBuilder.java:473)
at io.smallrye.config.SmallRyeConfig$ConfigSources.(SmallRyeConfig.java:522)
at io.smallrye.config.SmallRyeConfig.(SmallRyeConfig.java:66)
at io.smallrye.config.SmallRyeConfigBuilder.build(SmallRyeConfigBuilder.java:419)
at io.quarkus.runtime.generated.Config.readConfig(Unknown Source)
at io.quarkus.deployment.steps.RuntimeConfigSetup.deploy(Unknown Source)
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:104)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:41)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:120)
at io.quarkus.runner.GeneratedMain.main(Unknown Source)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
at java.base/java.lang.Thread.run(Thread.java:834)

If I put the quarkus.vault.authentication.app-role.secret-id property inside the microprofile-config.properties file (rather than SampleConfigSource class and instead hard code the secret in the property rather than reading from the file, then it works perfectly fine.

If I just use the SampleConfigSource to read the value from the file and put it in a different property name that does not start with quarkus.vault.* (e.g. application.property.key), then the property is set correctly. It is only when I read from the file and set the value (from the file) into the quarkus.vault.authentication.app-role.secret-id config map within my custom ConfigSource (SampleConfigSource) then it throws this error on startup.

It's almost like as if that quarkus.vault.authentication.app-role.secret-id property must live in microprofile-config.properties, but it can't read the value from the ConfigSource.

Another way I did it was to update the microprofile-config.properties to include the quarkus.vault.authentication.app-role.secret-id and set it to the property that contains the value in the file (e.g.):

quarkus.vault.authentication.app-role.secret-id=${app.value.read.from.file}

where app.value.read.from file is in the configurationMap in the SampleConfigSource that contains the value read from the file. But that does not work either and I get the same stack trace error.

Is there any other way where I can programatically override the property (to read the value from a file)? Starting to feel like it might not be possible to do this for quarkus.vault.* properties but would really appreciate any insight or suggestions. Many thanks,

Just to add, essentially I'm trying to override the quarkus.vault.authentication.app-role.secret-id field, but it doesn't seem to work as it always relies on that property being read from the microprofile-config.properties or application.properties

[quarkus-bot[bot]]

/cc @vsevel

[edufolly]

Hi @raheal

try to use SmallRye FileSystem Config Source to load properties from files.

I did this to work with docker swarm secrets, but you can read any text file and load to @ConfigProperty.

I created a PoC repo to check how it works. I hope that help you. https://github.com/edufolly/work-with-secrets/tree/prod

Best.