This repository was archived by the owner on Jan 9, 2020. It is now read-only.
Sync hdfs-kerberos-support to branch-2.2-kubernetes #472
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ranch-2.2-kubernetes-g Branch 2.2 kubernetes
…st for the container UID
PySpark Submission Failing on --py-files
…us-uids An entrypoint.sh script to add a passwd entry if one does not exist
…8s#420) This probably is not the correct fix long-term as we should find the specific module that is pulling in the bad jersey-1 version. But until we can track down what the specific offending module is, this will have to do.
…DFS locality support (apache-spark-on-k8s#412) * Flag-guard expensive DNS lookup of cluster node full names, part of HDFS locality support * Clean up a bit * Improve unit tests
* Initial architecture documentation. Initial full documentation for the submission client. Templates for the external shuffle service and the scheduler backend. * Add title to scheduler backend doc. * edits for PR review feedback
…or (apache-spark-on-k8s#424) * allow configuration to set environment variables on driver and executor as below: --conf spark.executorEnv.[EnvironmentVariableName] --conf spark.driverEnv.[EnvironmentVariableName] * change the driver environment key prefix to spark.kubernetes.driverEnv.
This reverts commit 737abdc.
This reverts commit 410dc9c.
Direct copy of revised design doc
…oxish-patch-2 Update external shuffle service docs
This fixes local integration testing
* Support HDFS rack locality * Fix unit tests * Address review comments * Address some review comments * Use traits for InetAddress and RackResolver util classes * Disables expensive DNS lookup by default
Required for ./dev/check-license to pass
…ark-on-k8s#437) * Use a secret to mount small files in driver and executors. Allows bypassing the resource staging server in a few scenarios. * Fix scalstyle * Address comments and add tests. * Lightly brush up formatting. * Make the working directory empty so that added files don't clobber existing binaries. * Address comments. * Drop testing file size to N+1 of the limit
…-k8s#444) * Use a list of environment variables for JVM options. * Fix merge conflicts.
…fle-version Bumping version to v2.2.0-kubernetes-0.3.0
…rvice-account Support service account override
* Fix conversion from GB to MiB Previously we applied a value which was in units MiB to Kubernetes _as if_ it were in MB units. Now, place MiB units in variable names and apply them to Kubernetes as the correct MiB units. * Fix test * Update tests
|
rerun unit tests please |
|
This is good to merge because unit-tests issues are solved with #414 |
|
Thanks @ifilonenko. Merging into hdfs-kerberos-support. |
|
Manually pushed to hdfs-kerberos-support. Closing. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note the target branch is
hdfs-kerberos-support. There were a few conflicts in imports and constants.@ifilonenko When you merge this, please avoid squashing so the history can show original commits. Maybe "rebase merging" is the right option to use?