Kyuubi, Kerberos, and Ranger yarn-cluster mode policy.download.auth.users must be *

[sohurdc]

With Kyuubi, Kerberos, and Ranger, when spark.submit.deployMode=cluster is used, does the policy.download.auth.users setting in the Ranger Admin UI have to be configured as *? Using * allows all users to download Ranger policies, which is very insecure.

When policy.download.auth.users set to hive, I have log:
25/07/07 11:46:30 ERROR RangerAdminRESTClient: Error getting Roles; service not found. secureMode=true, user=[email protected] (auth:KERBEROS), response=404, serviceName=hadoop3_hive, lastKnownRoleVersion=-1, lastActivationTimeInMillis=0
25/07/07 11:46:30 WARN RangerAdminRESTClient: Received 404 error code with body:[null], Ignoring
25/07/07 11:46:30 WARN RangerAdminRESTClient: Error getting policies. secureMode=true, user=[email protected] (auth:KERBEROS), response={"httpStatusCode":403,"statusCode":0}, serviceName=hadoop3_hive

When policy.download.auth.users set to *, it seems ok:
25/07/07 15:10:05 INFO RangerBasePlugin: Created PolicyRefresher Thread(PolicyRefresher(serviceName=hadoop3_hive)-85)
25/07/07 15:10:06 INFO RangerRolesProvider: RangerRolesProvider(serviceName=hadoop3_hive): found updated version. lastKnownRoleVersion=-1; newVersion=2
25/07/07 15:10:06 INFO PolicyRefresher: PolicyRefresher(serviceName=hadoop3_hive): found updated version. lastKnownVersion=-1; newVersion=33

But * means everyone can download the ranger policy,which is very insecure.

[hw-zhangyl]

Could you please share your ranger-spark-security.xml configuration? Our cluster also integrates Kyuubi and Ranger, and we're seeing 'simple user authentication failed' errors when the Spark Driver tries to fetch policies, indicating a lack of Kerberos authentication. Have you encountered this issue before?

[sohurdc]

Kyuubi, Kerberos, and Ranger works well with spark yarn-client mode . But work with spark yarn-cluster mode when Spark Driver tries to fetch policies, I has problem above.
My ranger-spark-security.xml

<configuration>
 <property>
 <name>ranger.plugin.spark.policy.rest.url</name>
 <value>http://xxx.com:6081</value>
 </property>

 <property>
 <name>ranger.plugin.spark.service.name</name>
 <value>hadoop3_spark</value>
 </property>

 <property>
 <name>ranger.plugin.spark.policy.cache.dir</name>
 <value>/tmp/ranger/spark</value>
 </property>

 <property>
 <name>ranger.plugin.spark.policy.pollIntervalMs</name>
 <value>30000</value>
 </property>

 <property>
 <name>ranger.plugin.spark.policy.source.impl</name>
 <value>org.apache.ranger.admin.client.RangerAdminRESTClient</value>
 </property>

 <property>
	<name>ranger.plugin.spark.enable.implicit.userstore.enricher</name>
	<value>true</value>
 </property>

 <property>
	<name>ranger.plugin.hive.policy.cache.dir</name>
	<value>/tmp/ranger/hive</value>
 </property>

</configuration>