Load configuration directory in run time#284
Conversation
The choices of configuration now: 1. From `HADOOP_CONF_DIR`. 2. From `HADOOP_HOME`. 3. From passed kv pair. In the case that users want to set configuration in run time(say having multiple HDFS clients with different configurations), they have two workarounds: 1. Set the environment variables manually via `std::env::set_var`. 2. Load configuration manually and build client by `client::ClientBuilder::with_config`. The first one introduces unsafe code and the latter is much more troublesome. Since the lib has the ability to read and parse the configuration by itself, I think adding an option for `client::ClientBuilder` to override the path read from environment variables is reasonable. Now `client::ClientBuilder` accepts config path via `with_config_dir`. And the path would be passed to `common::Configuration` to generate the final configs together with those passed by `with_config`. Small refactoring is made to `client::ClientBuilder`, decoupling the configuration reading and parsing code.
|
Ping @Kimahriman @PeterlitsZo 👀 |
Kimahriman
left a comment
Kimahriman
left a comment
There was a problem hiding this comment.
a few comments but looks good
Lostlitus
force-pushed
the
load-config-dir-in-run-time
branch
from
7f2591c to
467876c
Compare
|
I add some comments here. Very good work 😃 thank you. |
Lostlitus
force-pushed
the
load-config-dir-in-run-time
branch
2 times, most recently
from
6934854 to
15bf215
Compare
|
Can you add this to the Python side as well? Should be fairly straightforward, don't worry about adding any tests |
|
Looks like a build for |
|
OK, I try the python one |
Besides, fix some small problem: 1. Typo and format cases in README.md. 2. compile error in `rust/build.rs`.
Lostlitus
force-pushed
the
load-config-dir-in-run-time
branch
from
15bf215 to
40ab816
Compare
|
It seems that python one only have test for |
3 participants
Followup for #283