Skip to content

Commit 7129321

Browse files
dsevillapitrou
andauthored
GH-47560: [C++] Fix host handling for default HDFS URI (#47458)
### Rationale for this change In #25324 a fix is introduced for the python HadoopFileSystem, but it does not work if you use `from_uri()`, as it is passed to the underlying C++ implementation of the options parsing. The "default" case is not handled as in the python case, as the whole "hdfs://default" is passed to the underlying hdfs library, that expect "default" to search in `$HADOOP_CONF_DIR/core-site.xml`. ### What changes are included in this PR? Handle the `HadoopFileSystem.from_uri()` (or `FileSystem.from_uri()` when using `hdfs://default:xxx`) special HDFS URIs. ### Are these changes tested? There are no specific tests for this feature, but existing HDFS CI jobs pass. ### Are there any user-facing changes? Not exactly, but the documentation is honored for the `from_uri()` case. * GitHub Issue: #47560 Lead-authored-by: Diego Sevilla Ruiz <[email protected]> Co-authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Antoine Pitrou <[email protected]>
1 parent 37c87db commit 7129321

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

cpp/src/arrow/filesystem/hdfs.cc

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -363,8 +363,14 @@ Result<HdfsOptions> HdfsOptions::FromUri(const Uri& uri) {
363363
options_map.emplace(kv.first, kv.second);
364364
}
365365

366+
// Special case host = "default" or "hdfs://default" as stated by GH-47560.
367+
// If given the string "default", libhdfs selects the default filesystem
368+
// from `core-site.xml`.
366369
std::string host;
367-
host = uri.scheme() + "://" + uri.host();
370+
if (uri.host() == "default")
371+
host = uri.host();
372+
else
373+
host = uri.scheme() + "://" + uri.host();
368374

369375
// configure endpoint
370376
const auto port = uri.port();

0 commit comments

Comments
 (0)