Containerise Olog-ES

[willrogers]

We want three containers and docker compose configuration to coordinate them.

• Can someone describe the three required containers?
• Where should the code live?
• What does ES stand for? ElasticSearch?

See Olog/phoebus-olog#39

[shroffk]

A simple example for docker compose using kafka is available here:
https://github.com/ControlSystemStudio/phoebus/tree/master/services/alarm-server/src/test/resources/docker

[shroffk]

The 3 required containers are:

1. elastic 6.4

2. mongoDB
   both images for 1 and 2 are already published and we simply need to thread them together in a docker compose

3. olog-es
   we will have to create a dockerfile to build an image which hosts the olog-es REST service.

[willrogers]

How do you build the olog-es REST service?

[georgweiss]

It's a different repo: https://github.com/Olog/olog-es

[georgweiss]

Sorry, should have clarified that earlier...

[willrogers]

I think I am making pretty good progress on this but I am not sure how to debug this problem:

olog_1     | 2021-03-09 10:46:18.538  INFO 9 --- [l'}-mongo:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:1, serverValue:2}] to mongo:27017
olog_1     | 2021-03-09 10:46:18.815  INFO 9 --- [l'}-mongo:27017] org.mongodb.driver.cluster               : Monitor thread successfully connected to server with description ServerDescription{address=mongo:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 4, 4]}, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=190528900}
olog_1     | 2021-03-09 10:46:24.609  WARN 9 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'authenticationResource': Unsatisfied dependency expressed through field 'authenticationManager'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.security.authentication.AuthenticationManager' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

@shroffk @georgweiss any ideas?

[willrogers]

I think I have got past that one. Now I cannot connect to the elastic container, but this is more of a Docker problem than an olog one:

olog_1     | 2021-03-09 11:09:27.516  WARN 9 --- [           main] gov.bnl.olog.ElasticConfig               : Failed to create template for index olog_log
olog_1     |
olog_1     | java.net.ConnectException: Connection refused
olog_1     |    at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:959) ~[elasticsearch-rest-client-6.8.6.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:233) ~[elasticsearch-rest-client-6.8.6.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1749) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1708) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at org.elasticsearch.client.IndicesClient.getIndexTemplate(IndicesClient.java:1618) ~[elasticsearch-rest-high-level-client-6.8.4.jar!/:6.8.6]
olog_1     |    at gov.bnl.olog.ElasticConfig.elasticIndexValidation(ElasticConfig.java:185) ~[classes!/:na]
olog_1     |    at gov.bnl.olog.ElasticConfig.getIndexClient(ElasticConfig.java:92) ~[classes!/:na]
olog_1     |    at gov.bnl.olog.ElasticConfig$$EnhancerBySpringCGLIB$$4f83853.CGLIB$getIndexClient$0(<generated>) ~[classes!/:na]
olog_1     |    at gov.bnl.olog.ElasticConfig$$EnhancerBySpringCGLIB$$4f83853$$FastClassBySpringCGLIB$$8703823c.invoke(<generated>) ~[classes!/:na]
olog_1     |    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at gov.bnl.olog.ElasticConfig$$EnhancerBySpringCGLIB$$4f83853.getIndexClient(<generated>) ~[classes!/:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
olog_1     |    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
olog_1     |    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:484) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1287) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1207) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:116) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1287) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1207) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:116) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879) ~[spring-beans-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878) ~[spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) ~[spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
olog_1     |    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) ~[spring-boot-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
olog_1     |    at gov.bnl.olog.Application.main(Application.java:38) ~[classes!/:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
olog_1     |    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
olog_1     |    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
olog_1     |    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[olog-es-2.0.jar:na]
olog_1     |    at org.springframework.boot.loader.Launcher.launch(Launcher.java:107) ~[olog-es-2.0.jar:na]
olog_1     |    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[olog-es-2.0.jar:na]
olog_1     |    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) ~[olog-es-2.0.jar:na]
olog_1     | Caused by: java.net.ConnectException: Connection refused
olog_1     |    at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:na]
olog_1     |    at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[na:na]
olog_1     |    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
olog_1     |    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
olog_1     |    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
olog_1     |    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
olog_1     |    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
olog_1     |    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

[georgweiss]

Indeed seems like a Docker thing...

[georgweiss]

By default Elastic binds to 127.0.0.1:9200. Not sure how Docker treats that....

[willrogers]

Aha I think I have got it working. Here is the response to https://localhost:8181/Olog/logs:

[]

[georgweiss]

Right, empty JSON array...

[willrogers]

A minimal version is here: https://github.com/willrogers/olog-es-docker

[shroffk]

Documentation of how to run Olog-es with a user specified application.properties