Plugin does not get Username Header

[jnerin]

Problem description

Steps to reproduce the problem

1. cd /opt/graylog/plugin/
2. wget https://github.com/Graylog2/graylog-plugin-auth-sso/releases/download/1.0.3/graylog-plugin-auth-sso-1.0.3.jar
3. configure trusted proxy setting on graylog or uncheck the security checkbox that validates it.
4. Setup a proxy that sends some header with username
5. Setup sso to use that header as username header and create new users.

Environment

• Graylog Version: 2.1.1-1
• Plugin Version: 1.0.3 (although it reports itself to graylog as being version 1.0.0 according to graylog debug output)
• Elasticsearch Version: 2.3.1 (should be 2.3.5 according to http://docs.graylog.org/en/2.1/pages/configuration/elasticsearch.html)
• MongoDB Version: v3.2.5
• Operating System: Ubuntu 14.04.4 LTS
• Browser version: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36 and Mozilla/5.0 (Windows NT 6.1; WOW64; rv:48.0) Gecko/20100101 Firefox/48.0

Pluging config is:

• Username Header: X-Forwarded-User
• trusted proxy set up as 127.0.0.1/32, 0:0:0:0:0:0:0:1/128, but the checkbox unchecked
• Automatically create users checked
• Default user role: Reader

Graylog package from https://packages.graylog2.org/releases/graylog-omnibus/ubuntu/graylog_2.1.1-1_amd64.deb, this AWS instance is an update of a pre 2.x version following instructions on http://docs.graylog.org/en/2.1/pages/configuration/graylog_ctl.html#migrate-manually-from-1-x-to-2-1-x , that might be relevant.

For us I think the main error is those lines we get on the output when running graylog with debug:

2016-09-21 11:36:14,912 DEBUG: org.graylog.plugins.auth.sso.SsoAuthRealm - Trusted header X-Forwarded-User is not set.
2016-09-21 11:36:14,913 DEBUG: org.apache.shiro.realm.AuthenticatingRealm - Looked up AuthenticationInfo [null] from doGetAuthenticationInfo
2016-09-21 11:36:14,913 DEBUG: org.apache.shiro.realm.AuthenticatingRealm - No AuthenticationInfo found for submitted AuthenticationToken [org.graylog2.shared.security.HttpHeadersToken@e8e18e3].  Returning null.

But I can see in tcpdump that the traffic going to graylog has this header:

tcpdump "host 127.0.0.1 and port 9000" -A -i lo

GET /config.js HTTP/1.0
X-Remote-User: dummy
Remote-User: dummy
X-Forwarded-User: dummy
X-Remote-User-Email: dummy@example.com
X-Forwarded-For: 10.x.x.x
Host: graylog
Connection: close
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:48.0) Gecko/20100101 Firefox/48.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://graylog/
Cache-Control: max-age=0

I know that we sent multiple headers, we have been beating this for a while.

Graylog is not automatically creating the users and users receive a login box, even when creating the user manually the user is not granted access.

Relevant parts of graylog debug messages:

2016-09-21 11:35:39,054 INFO : org.graylog2.bootstrap.CmdLineTool - Running in Debug mode
2016-09-21 11:35:39,083 DEBUG: org.graylog2.shared.plugins.PluginLoader - Scanning directory </opt/graylog/plugin> for plugins...
2016-09-21 11:35:39,083 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading [7] plugins
2016-09-21 11:35:39,149 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading </opt/graylog/plugin/graylog-plugin-auth-sso-1.0.3.jar>
2016-09-21 11:35:39,150 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading </opt/graylog/plugin/graylog-plugin-map-widget-1.1.1.jar>
2016-09-21 11:35:39,150 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading </opt/graylog/plugin/usage-statistics-2.1.1.jar>
2016-09-21 11:35:39,150 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading </opt/graylog/plugin/graylog-plugin-enterprise-integration-1.1.1.jar>
2016-09-21 11:35:39,150 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading </opt/graylog/plugin/graylog-plugin-pipeline-processor-1.1.1.jar>
2016-09-21 11:35:39,151 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading </opt/graylog/plugin/graylog-plugin-collector-1.1.1.jar>
2016-09-21 11:35:39,151 DEBUG: org.graylog2.shared.plugins.PluginLoader - Loading </opt/graylog/plugin/graylog-plugin-beats-1.1.1.jar>
2016-09-21 11:35:39,153 DEBUG: org.graylog2.shared.plugins.PluginProperties - No value found for attribute <Graylog-Plugin-Properties-Path> in JAR manifest of file </opt/graylog/plugin/graylog-plugin-auth-sso-1.0.3.jar>
2016-09-21 11:35:39,154 DEBUG: org.graylog2.shared.plugins.PluginLoader - Creating isolated class loader for <file:/opt/graylog/plugin/graylog-plugin-auth-sso-1.0.3.jar>
2016-09-21 11:35:39,154 DEBUG: org.graylog2.shared.plugins.PluginProperties - No value found for attribute <Graylog-Plugin-Properties-Path> in JAR manifest of file </opt/graylog/plugin/graylog-plugin-map-widget-1.1.1.jar>
2016-09-21 11:35:39,155 DEBUG: org.graylog2.shared.plugins.PluginLoader - Creating isolated class loader for <file:/opt/graylog/plugin/graylog-plugin-map-widget-1.1.1.jar>
2016-09-21 11:35:39,155 DEBUG: org.graylog2.shared.plugins.PluginProperties - No value found for attribute <Graylog-Plugin-Properties-Path> in JAR manifest of file </opt/graylog/plugin/usage-statistics-2.1.1.jar>
2016-09-21 11:35:39,155 DEBUG: org.graylog2.shared.plugins.PluginLoader - Creating isolated class loader for <file:/opt/graylog/plugin/usage-statistics-2.1.1.jar>
2016-09-21 11:35:39,155 DEBUG: org.graylog2.shared.plugins.PluginProperties - No value found for attribute <Graylog-Plugin-Properties-Path> in JAR manifest of file </opt/graylog/plugin/graylog-plugin-enterprise-integration-1.1.1.jar>
2016-09-21 11:35:39,156 DEBUG: org.graylog2.shared.plugins.PluginLoader - Creating isolated class loader for <file:/opt/graylog/plugin/graylog-plugin-enterprise-integration-1.1.1.jar>
2016-09-21 11:35:39,156 DEBUG: org.graylog2.shared.plugins.PluginProperties - Loading <org.graylog.plugins.graylog-plugin-pipeline-processor/graylog-plugin.properties> from </opt/graylog/plugin/graylog-plugin-pipeline-processor-1.1.1.jar>
2016-09-21 11:35:39,156 DEBUG: org.graylog2.shared.plugins.PluginLoader - Using shared class loader for <file:/opt/graylog/plugin/graylog-plugin-pipeline-processor-1.1.1.jar>
2016-09-21 11:35:39,157 DEBUG: org.graylog2.shared.plugins.PluginProperties - No value found for attribute <Graylog-Plugin-Properties-Path> in JAR manifest of file </opt/graylog/plugin/graylog-plugin-collector-1.1.1.jar>
2016-09-21 11:35:39,157 DEBUG: org.graylog2.shared.plugins.PluginLoader - Creating isolated class loader for <file:/opt/graylog/plugin/graylog-plugin-collector-1.1.1.jar>
2016-09-21 11:35:39,157 DEBUG: org.graylog2.shared.plugins.PluginProperties - No value found for attribute <Graylog-Plugin-Properties-Path> in JAR manifest of file </opt/graylog/plugin/graylog-plugin-beats-1.1.1.jar>
2016-09-21 11:35:39,157 DEBUG: org.graylog2.shared.plugins.PluginLoader - Creating isolated class loader for <file:/opt/graylog/plugin/graylog-plugin-beats-1.1.1.jar>
2016-09-21 11:35:39,158 DEBUG: org.graylog2.shared.plugins.PluginLoader - Creating shared class loader for 1 plugins: [file:/opt/graylog/plugin/graylog-plugin-pipeline-processor-1.1.1.jar]
2016-09-21 11:35:39,175 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Single Sign-On (SSO) Authentication Provider 1.0.0 [org.graylog.plugins.auth.sso.SsoAuthPlugin]
2016-09-21 11:35:39,175 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Elastic Beats Input 1.1.1 [org.graylog.plugins.beats.BeatsInputPlugin]
2016-09-21 11:35:39,175 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Collector 1.1.1 [org.graylog.plugins.collector.CollectorPlugin]
2016-09-21 11:35:39,176 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Enterprise Integration Plugin 1.1.1 [org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2016-09-21 11:35:39,176 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: MapWidgetPlugin 1.1.1 [org.graylog.plugins.map.MapWidgetPlugin]
2016-09-21 11:35:39,176 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Pipeline Processor Plugin 1.1.1 [org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2016-09-21 11:35:39,177 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Anonymous Usage Statistics 2.1.1 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2016-09-21 11:35:39,185 DEBUG: org.graylog2.bootstrap.CmdLineTool - Loading configuration from config file: /opt/graylog/conf/graylog.conf
2016-09-21 11:35:39,302 INFO : org.graylog2.bootstrap.CmdLineTool - Running with JVM arguments: -Xms1g -Xmx1500m -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow 
-Dlog4j.configurationFile=file:///opt/graylog/conf/log4j2.xml -Djava.library.path=/opt/graylog/server/lib/sigar/ -Dgraylog2.installation_source=unknown

[...]

2016-09-21 11:35:41,981 DEBUG: com.google.inject.internal.BytecodeGen - Loading class org.graylog.plugins.auth.sso.SsoAuthRealm FastClass with com.google.inject.internal.BytecodeGen$BridgeClassLoader@6f667ad1
2016-09-21 11:35:41,985 DEBUG: com.google.inject.internal.BytecodeGen - Loading class org.graylog.plugins.auth.sso.SsoAuthPermissions FastClass with com.google.inject.internal.BytecodeGen$BridgeClassLoader@6f667ad1
2016-09-21 11:35:41,987 DEBUG: com.google.inject.internal.BytecodeGen - Loading class org.graylog.plugins.auth.sso.audit.SsoAuthAuditEventTypes FastClass with com.google.inject.internal.BytecodeGen$BridgeClassLoader@6f667ad1

[...]

2016-09-21 11:35:59,676 DEBUG: REST API -     GET     /api/plugins/org.graylog.plugins.auth.sso/config (class org.graylog.plugins.auth.sso.SsoConfigResource)
2016-09-21 11:35:59,676 DEBUG: REST API -     PUT     /api/plugins/org.graylog.plugins.auth.sso/config (class org.graylog.plugins.auth.sso.SsoConfigResource)

[...]

2016-09-21 11:36:00,614 DEBUG: org.graylog2.shared.security.ShiroSecurityBinding - Resource method org.graylog.plugins.auth.sso.SsoConfigResource#get requires an authenticated user.
2016-09-21 11:36:00,614 DEBUG: org.graylog2.shared.security.ShiroSecurityBinding - Resource method org.graylog.plugins.auth.sso.SsoConfigResource#get requires an authorization checks.
2016-09-21 11:36:00,617 DEBUG: org.graylog2.shared.security.ShiroSecurityBinding - Resource method org.graylog.plugins.auth.sso.SsoConfigResource#update requires an authenticated user.
2016-09-21 11:36:00,617 DEBUG: org.graylog2.shared.security.ShiroSecurityBinding - Resource method org.graylog.plugins.auth.sso.SsoConfigResource#update requires an authorization checks.

[...]

2016-09-21 11:36:02,966 CONFIG: org.glassfish.jersey.server.ApplicationHandler - Jersey application initialized.
Root Resource Classes:
[...]
  org.graylog.plugins.auth.sso.SsoConfigResource

[...]

2016-09-21 11:36:14,881 DEBUG: org.apache.shiro.authc.pam.ModularRealmAuthenticator - Realm [org.graylog2.security.realm.AccessTokenAuthenticator@22cd75bf] does not support token org.graylog2.shared.security.HttpHeadersToken@e8e18e3.  Skipping realm.
2016-09-21 11:36:14,881 DEBUG: org.apache.shiro.authc.pam.ModularRealmAuthenticator - Realm [org.graylog2.security.realm.SessionAuthenticator@a32b7ee] does not support token org.graylog2.shared.security.HttpHeadersToken@e8e18e3.  Skipping realm.
2016-09-21 11:36:14,881 DEBUG: org.apache.shiro.authc.pam.ModularRealmAuthenticator - Realm [org.graylog2.security.realm.PasswordAuthenticator@7123109a] does not support token org.graylog2.shared.security.HttpHeadersToken@e8e18e3.  Skipping realm.
2016-09-21 11:36:14,881 DEBUG: org.apache.shiro.authc.pam.ModularRealmAuthenticator - Realm [org.graylog2.security.realm.RootAccountRealm@380af38c] does not support token org.graylog2.shared.security.HttpHeadersToken@e8e18e3.  Skipping realm.
2016-09-21 11:36:14,893 DEBUG: org.mongodb.driver.protocol.command - Sending command {find : BsonString{value='cluster_config'}} to database graylog on connection [connectionId{localValue:7, serverValue:133}] to server 127.0.0.1:27017
2016-09-21 11:36:14,905 DEBUG: org.mongodb.driver.protocol.command - Command execution completed
2016-09-21 11:36:14,912 DEBUG: org.graylog.plugins.auth.sso.SsoAuthRealm - Trusted header X-Forwarded-User is not set.
2016-09-21 11:36:14,913 DEBUG: org.apache.shiro.realm.AuthenticatingRealm - Looked up AuthenticationInfo [null] from doGetAuthenticationInfo
2016-09-21 11:36:14,913 DEBUG: org.apache.shiro.realm.AuthenticatingRealm - No AuthenticationInfo found for submitted AuthenticationToken [org.graylog2.shared.security.HttpHeadersToken@e8e18e3].  Returning null.
2016-09-21 11:36:14,913 DEBUG: org.graylog2.shared.security.ShiroAuthenticationFilter - Unable to authenticate user.
org.apache.shiro.authc.AuthenticationException: No account information found for authentication token [org.graylog2.shared.security.HttpHeadersToken@e8e18e3] by this Authenticator instance.  Please check that it is configured correctly.
        at org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:202) ~[graylog.jar:?]
        at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106) ~[graylog.jar:?]
        at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:270) ~[graylog.jar:?]
        at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:256) ~[graylog.jar:?]
        at org.graylog2.shared.security.ShiroSecurityContext.loginSubject(ShiroSecurityContext.java:101) ~[graylog.jar:?]
        at org.graylog2.shared.security.ShiroAuthenticationFilter.filter(ShiroAuthenticationFilter.java:48) [graylog.jar:?]
        at org.graylog2.rest.resources.system.SessionsResource.validateSession(SessionsResource.java:183) [graylog.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
2016-09-21 11:36:14,922 DEBUG: org.graylog2.rest.accesslog - 10.17.3.150 - [-] "GET api/system/sessions" Mozilla/5.0 (Windows NT 6.1; WOW64; rv:48.0) Gecko/20100101 Firefox/48.0 200 -1
2016-09-21 11:36:14,952 DEBUG: org.glassfish.grizzly.nio.transport.TCPNIOTransport - TCPNIOConnection (TCPNIOConnection{localSocketAddress={/10.17.0.142:9000}, peerSocketAddress={/10.17.3.150:4856}}) (composite) write 607 bytes

[hc4]

X-Forwarded-User must be set for all REST requests.
I've spent several hours, before notced this :)

The easiest way is to host graylog WEB and REST on same address (see docs)

[jnerin]

Ok, I changed set-external-ip to be http://myserver/api, and I even tried setting up graylog on localhost 80 and using squid with the equivalent config (using basic_ncsa_auth instead of kerberos) provided on another bug comment (#16 (comment)), but I still can't make it work.

Only way has been using a browser addon to always send a X-Remote-Username, change the SSO config to use that header (disabling the trusted proxies), and then it creates the user and it "works".

But I can't make it work with standard auth header, and X-Remote-User header with the squid config in that comment, when javascript overrides the auth header with the bogus session auth header (the one with "password" session) everything breaks and I got the login dialog.

[martinrm77]

I use Apache 2.4 with Shibboleth 2.6 and SAML2 and it works ok. Just have to change from default using environment variables to using http headers and configure your attribute-map.xml and the user header in Graylog SSO.

This is graylog 2.1.2 though.

[kitsushadow]

@martinrm77 can you provide some feedback on your configuration of your shibboleth.xml and your attribute-map.xml

We are really struggling to get NameID to map to the Graylog user we would like to login as

[kitsushadow]

@jnerin did you ever get this working?

[martinrm77]

I actually didnt change that much, let me summarize:

shibboleth2.xml

<ApplicationDefaults entityID="https://graylog.example.com/shibboleth"
  homeURL="https://graylog.example.com/"
  REMOTE_USER="User">

attribute-map.xml

<!-- Custom attributes from ADFS -->
<Attribute name="fullName" id="fullName"/>
<Attribute name="email" id="email"/>
<Attribute name="http://schemas.xmlsoap.org/claims/Group" id="Group"/>
<Attribute name="urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified" id="User"/>

We get our SAML2 from Microsoft ADFS 3.0 - and these are some values from drop-downs, that should be pretty standard.
I had to create our own ID's, because the built-in filters for EPPN and the like dropped all our usual data, because it had to be in a specific format. Instead of fixing all the filters, its easier to just use another ID, like "User".

We use Apache for SSL and SSO auth, so here is the relevant apache config stuff inside the virtual server:

<Location />
 RequestHeader set X-Graylog-Server-URL "https://graylog.example.com/api"
 RequestHeader unset Cookie
 ProxyPass        http://graylog-server1:9000/
 ProxyPassReverse http://graylog-server1:9000/

 AuthType shibboleth
 ShibRequestSetting requireSession 1
 ShibUseEnvironment Off
 ShibUseHeaders On
 # To enable when graylog sso module supports url-encoded header values
 #ShibRequestSetting encoding URL
 # Only allow persons in security group
 require shib-attr Group ~ OUR-SECURITY-GROUP-NAME

</Location>

<Location /api>
 AuthType shibboleth
 ShibRequestSetting requireSession 1
 require shib-attr Group ~ OUR-SECURITY-GROUP-NAME
</Location>

[kitsushadow]

Awesome! I will review our configs and see what we are missing.