Non-primary user group is not considered by ACL?

[mark-bb]

Hello.

Ozone 2.0.0.

I use the following config:

<property><name>hadoop.security.group.mapping</name><value>org.apache.hadoop.security.LdapGroupsMapping</value></property>
<property><name>hadoop.security.group.mapping.ldap.url</name><value>ldap://xxx:389</value></property>
<property><name>hadoop.security.group.mapping.ldap.base</name><value>dc=domain,dc=com</value></property>
<property><name>hadoop.security.group.mapping.ldap.userbase</name><value>ou=People,dc=domain,dc=com</value></property>
<property><name>hadoop.security.group.mapping.ldap.groupbase</name><value>ou=Groups,dc=domain,dc=com</value></property>
<property><name>hadoop.security.group.mapping.ldap.bind.user</name><value>cn=bind,ou=ServiceAccounts,dc=domain,dc=com</value></property>
<property><name>hadoop.security.group.mapping.ldap.bind.password.file</name><value>/opt/hadoop/etc/hadoop/ldap_bind_password.txt</value></property>
<property><name>hadoop.security.group.mapping.ldap.search.filter.user</name><value>(&amp;(objectClass=posixAccount)(uid={0}))</value></property>
<property><name>hadoop.security.group.mapping.ldap.search.filter.group</name><value>(objectClass=posixGroup)</value></property>
<property><name>hadoop.security.group.mapping.ldap.search.attr.member</name><value>memberUid</value></property>
<property><name>hadoop.security.group.mapping.ldap.search.attr.group.name</name><value>cn</value></property>

ldapsearch shows that myuser is member of the hadoop group:

# extended LDIF
#
# LDAPv3
# base <dc=domain,dc=com> with scope subtree
# filter: (&(objectclass=posixGroup)(cn=hadoop))
# requesting: dn gidNumber memberuid
#

# hadoop, Groups, domain.com
dn: cn=hadoop,ou=Groups,dc=domain,dc=com
gidNumber: 5000
memberUid: hadoop
memberUid: myuser

but myuser has another primary group (not equal to hadoop):

# extended LDIF
#
# LDAPv3
# base <dc=domain,dc=com> with scope subtree
# filter: (&(objectclass=PosixAccount)(uid=myuser))
# requesting: dn gidNumber
#

# myuser, People, domain.com
dn: uid=myuser,ou=People,dc=domain,dc=com
gidNumber: 5001

I have the following ACL

ozone sh bucket getacl /user/hadoop

[ {
  "type" : "USER",
  "name" : "hadoop",
  "aclScope" : "ACCESS",
  "aclList" : [ "ALL" ]
}, {
  "type" : "GROUP",
  "name" : "hadoop",
  "aclScope" : "ACCESS",
  "aclList" : [ "READ", "LIST" ]
} ]

But myuser is unable to list the directory:

hadoop fs -ls /user/hadoop/

ls: User myuser doesn't have READ permission to access bucket Volume:user Bucket:hadoop

So, I suspect, that Ozone considers the primary group of myuser only, whereas it should consider all the myusers groups including hadoop.

Questions:

1. Is there the "hdfs groups" command analogue for ozone, where I could check how ozone lists the user groups?
2. Did I miss something in my configuration to allow myuser have an access to the bucket?

[mark-bb]

Seems, that the problem is not related to Ozone, but rather to my LDAP config.
I've started HDFS in the same environment, and I really see, that not all groups of myuser are enumerated.

hdfs groups -D "fs.defaultFS=hdfs://nameservice1/" myuser
myuser : myuser

But it looks really strange, since my LDAP search returns expected result (2 groups are listed for myuser)

# extended LDIF
#
# LDAPv3
# base <dc=domain,dc=com> with scope subtree
# filter: (&(objectclass=posixGroup)(memberUid=myuser))
# requesting: cn
#

# hadoop, Groups, domain.com
dn: cn=hadoop,ou=Groups,dc=domain,dc=com
cn: hadoop

# myuser, Groups, domain.com
dn: cn=myuser,ou=Groups,dc=domain,dc=com
cn: myuser

and I believe, that I configured the system correctly to make some similar LDAP call as above with the following properties:

<property><name>hadoop.security.group.mapping.ldap.search.attr.member</name><value>memberUid</value></property>
<property><name>hadoop.security.group.mapping.ldap.search.attr.group.name</name><value>cn</value></property>

So, why don't I get the following expected output with such a config?

hdfs groups -D "fs.defaultFS=hdfs://nameservice1/" myuser
myuser : myuser hadoop

[mark-bb]

For those who are interesting: this was a LDAP configuration problem.
The memberUid attribute of the posixGroup object must be equal to the uidNumber attribute value of the corresponding posixAccount object, not to the uid one.

So, if I want myuser

# filter: (&(objectclass=posixAccount)(uid=myuser))
# requesting: uid uidNumber gidNumber
#

# myuser, People, domain.com
dn: uid=myuser,ou=People,dc=domain,dc=com
uid: myuser
uidNumber: 5001  <--
gidNumber: 5001

be a member of groups hadoop and myuser, I have to define them as follows:

# filter: (&(objectclass=posixGroup)(memberUid=5001))  <--
# requesting: cn memberUid
#

# hadoop, Groups, domain.com
dn: cn=hadoop,ou=Groups,dc=domain,dc=com
cn: hadoop
memberUid: 5000
memberUid: 5001  <--

# myuser, Groups, domain.com
dn: cn=myuser,ou=Groups,dc=domain,dc=com
cn: myuser
memberUid: 5001  <--

Here are the details of posixGroup support implemented quite a long time ago:
https://issues.apache.org/jira/browse/HADOOP-9477

[adoroszlai]

Thanks @mark-bb for sharing your solution.