Skip to content

Commit ffce075

Browse files
committed
[DOP-25451] Fix HDFS connector ignoring user & password
1 parent 59ad3c6 commit ffce075

File tree

4 files changed

+19
-1
lines changed

4 files changed

+19
-1
lines changed

docker/Dockerfile.worker

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,5 +66,13 @@ RUN --mount=type=cache,target=/root/.cache/pypoetry \
6666
--without docs,dev \
6767
&& python -m compileall -j 4 .venv
6868

69-
RUN sed -i 's/python -m/coverage run -m/g' /app/entrypoint.sh
7069
ENV SYNCMASTER__WORKER__CREATE_SPARK_SESSION_FUNCTION=tests.spark.get_worker_spark_session
70+
71+
# Collect coverage from worker
72+
RUN sed -i 's/python -m/coverage run -m/g' /app/entrypoint.sh
73+
74+
# Replace kinit binary with dummy, to skip Kerberos interaction in tests
75+
RUN mkdir -p /app/.local/bin && \
76+
echo "#!/bin/bash" > /app/.local/bin/kinit \
77+
&& chmod +x /app/.local/bin/kinit
78+
ENV PATH="/app/.local/bin:$PATH"

docs/changelog/0.2.2.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
0.2.2 (2025-04-11)
2+
==================
3+
4+
Bug fixes
5+
---------
6+
7+
- Fix ``HDFS`` connection was trying to use anonymous auth instead of user/password.

docs/changelog/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
:caption: Changelog
44

55
DRAFT
6+
0.2.2
67
0.2.1
78
0.2.0
89
0.1.5

syncmaster/worker/handlers/file/hdfs.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,6 @@ def connect(self, spark: SparkSession):
2525

2626
self.file_connection = HDFS(
2727
cluster=self.connection_dto.cluster,
28+
user=self.connection_dto.user,
29+
password=self.connection_dto.password,
2830
).check()

0 commit comments

Comments
 (0)