Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
...
### Added
- upath.implementations.ftp: added FTPPath support

## [0.3.6] - 2025-11-13
### Added
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ For more examples, see the [example notebook here][example-notebook].
- `memory:` Ephemeral filesystem in RAM
- `az:`, `adl:`, `abfs:` and `abfss:` Azure Storage _(requires `adlfs`)_
- `data:` RFC 2397 style data URLs _(requires `fsspec>=2023.12.2`)_
- `ftp:` FTP filesystem
- `github:` GitHub repository filesystem
- `hf:` Hugging Face filesystem _(requires `huggingface_hub`)_
- `http:` and `https:` HTTP(S)-based filesystem
Expand Down
16 changes: 16 additions & 0 deletions docs/api/implementations.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,22 @@ Data URL scheme implementation for embedded data.

---

## upath.implementations.ftp

::: upath.implementations.ftp.FTPPath
options:
heading_level: 3
show_root_heading: true
show_root_full_path: false
members: []
show_bases: true

**Protocol:** `ftp://`

FTP (File Transfer Protocol) implementation.

---

## upath.implementations.cached

::: upath.implementations.cached.SimpleCachePath
Expand Down
1 change: 1 addition & 0 deletions docs/api/types.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ These help ensure correct parameter names and types when configuring different f
- S3StorageOptions
- AzureStorageOptions
- DataStorageOptions
- FTPStorageOptions
- GitHubStorageOptions
- HDFSStorageOptions
- HTTPStorageOptions
Expand Down
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ if http_path.exists():
- :fontawesome-solid-memory: `memory:` Ephemeral filesystem in RAM
- :fontawesome-brands-microsoft: `az:`, `adl:`, `abfs:` and `abfss:` Azure Storage _(requires `adlfs`)_
- :fontawesome-solid-database: `data:` RFC 2397 style data URLs _(requires `fsspec>=2023.12.2`)_
- :fontawesome-solid-network-wired: `ftp:` FTP filesystem
- :fontawesome-brands-github: `github:` GitHub repository filesystem
- :fontawesome-solid-globe: `http:` and `https:` HTTP(S)-based filesystem
- :fontawesome-solid-server: `hdfs:` Hadoop distributed filesystem
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ dev = [
"cheroot",
# "hadoop-test-cluster",
# "pyarrow",
"pyftpdlib",
"typing_extensions; python_version<'3.11'",
]
dev-third-party = [
Expand Down
8 changes: 8 additions & 0 deletions typesafety/test_upath_signatures.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@
cls: HfPath
- module: upath.implementations.data
cls: DataPath
- module: upath.implementations.ftp
cls: FTPPath
- module: upath.implementations.github
cls: GitHubPath
- module: upath.implementations.hdfs
Expand Down Expand Up @@ -259,6 +261,8 @@
cls: HfPath
- module: upath.implementations.data
cls: DataPath
- module: upath.implementations.ftp
cls: FTPPath
- module: upath.implementations.github
cls: GitHubPath
- module: upath.implementations.hdfs
Expand Down Expand Up @@ -576,6 +580,8 @@
cls: HfPath
- module: upath.implementations.data
cls: DataPath
- module: upath.implementations.ftp
cls: FTPPath
- module: upath.implementations.github
cls: GitHubPath
- module: upath.implementations.hdfs
Expand Down Expand Up @@ -972,6 +978,8 @@
cls: HfPath
- module: upath.implementations.data
cls: DataPath
- module: upath.implementations.ftp
cls: FTPPath
- module: upath.implementations.github
cls: GitHubPath
- module: upath.implementations.hdfs
Expand Down
17 changes: 17 additions & 0 deletions typesafety/test_upath_types.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
protocol: hf
- cls_fqn: upath.implementations.data.DataPath
protocol: data
- cls_fqn: upath.implementations.ftp.FTPPath
protocol: ftp
- cls_fqn: upath.implementations.github.GitHubPath
protocol: github
- cls_fqn: upath.implementations.hdfs.HDFSPath
Expand Down Expand Up @@ -112,6 +114,8 @@
protocol: hf
- cls_fqn: upath.implementations.data.DataPath
protocol: data
- cls_fqn: upath.implementations.ftp.FTPPath
protocol: ftp
- cls_fqn: upath.implementations.github.GitHubPath
protocol: github
- cls_fqn: upath.implementations.hdfs.HDFSPath
Expand Down Expand Up @@ -185,6 +189,10 @@
cls: DataPath
supported_example_name: use_listings_cache
supported_example_value: False
- module: upath.implementations.ftp
cls: FTPPath
supported_example_name: host
supported_example_value: '"ftp.example.com"'
- module: upath.implementations.github
cls: GitHubPath
supported_example_name: org
Expand Down Expand Up @@ -258,6 +266,10 @@
cls: DataPath
supported_example_name: use_listings_cache
unsupported_example_value: '"blub"'
- module: upath.implementations.ftp
cls: FTPPath
supported_example_name: host
unsupported_example_value: '123'
- module: upath.implementations.github
cls: GitHubPath
supported_example_name: repo
Expand Down Expand Up @@ -318,6 +330,8 @@
cls: HfPath
- module: upath.implementations.data
cls: DataPath
- module: upath.implementations.ftp
cls: FTPPath
- module: upath.implementations.github
cls: GitHubPath
- module: upath.implementations.hdfs
Expand Down Expand Up @@ -367,6 +381,9 @@
- module: upath.implementations.data
cls: DataPath
td: DataStorageOptions
- module: upath.implementations.ftp
cls: FTPPath
td: FTPStorageOptions
- module: upath.implementations.github
cls: GitHubPath
td: GitHubStorageOptions
Expand Down
3 changes: 3 additions & 0 deletions upath/_stat.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ def _convert_value_to_timestamp(value: Any) -> int | float:
if isinstance(value, (int, float)):
return value
elif isinstance(value, str):
if len(value) == 14:
return datetime.strptime(value, r"%Y%m%d%H%M%S").timestamp()
if value.endswith("Z"):
value = value[:-1] + "+00:00"
return datetime.fromisoformat(value).timestamp()
Expand Down Expand Up @@ -262,6 +264,7 @@ def st_mtime(self) -> int | float:
"timeModified",
"modificationTime",
"modified_at",
"modify",
]:
try:
raw_value = self._info[key]
Expand Down
7 changes: 7 additions & 0 deletions upath/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -838,6 +838,13 @@ def __new__(
**_: Any,
) -> _uimpl.data.DataPath: ...
@overload # noqa: E301
def __new__(
cls,
*args: JoinablePathLike,
protocol: Literal["ftp"],
**_: Any,
) -> _uimpl.ftp.FTPPath: ...
@overload # noqa: E301
def __new__(
cls,
*args: JoinablePathLike,
Expand Down
57 changes: 57 additions & 0 deletions upath/implementations/ftp.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
from __future__ import annotations

import sys
from collections.abc import Iterator
from ftplib import error_perm as FTPPermanentError # nosec B402
from typing import TYPE_CHECKING

from upath.core import UPath
from upath.types import JoinablePathLike

if TYPE_CHECKING:
from typing import Literal

if sys.version_info >= (3, 11):
from typing import Self
from typing import Unpack
else:
from typing_extensions import Self
from typing_extensions import Unpack

from upath._chain import FSSpecChainParser
from upath.types.storage_options import FTPStorageOptions

__all__ = ["FTPPath"]


class FTPPath(UPath):
__slots__ = ()

if TYPE_CHECKING:

def __init__(
self,
*args: JoinablePathLike,
protocol: Literal["ftp"] | None = ...,
chain_parser: FSSpecChainParser = ...,
**storage_options: Unpack[FTPStorageOptions],
) -> None: ...

def mkdir(
self,
mode: int = 0o777,
parents: bool = False,
exist_ok: bool = False,
) -> None:
try:
return super().mkdir(mode, parents, exist_ok)
except FTPPermanentError as e:
if e.args[0].startswith("550") and exist_ok:
return
raise FileExistsError(str(self)) from e

def iterdir(self) -> Iterator[Self]:
if not self.is_dir():
raise NotADirectoryError(str(self))
else:
return super().iterdir()
4 changes: 4 additions & 0 deletions upath/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@
from upath.implementations.cloud import HfPath as _HfPath
from upath.implementations.cloud import S3Path as _S3Path
from upath.implementations.data import DataPath as _DataPath
from upath.implementations.ftp import FTPPath as _FTPPath
from upath.implementations.github import GitHubPath as _GitHubPath
from upath.implementations.hdfs import HDFSPath as _HDFSPath
from upath.implementations.http import HTTPPath as _HTTPPath
Expand Down Expand Up @@ -92,6 +93,7 @@ class _Registry(MutableMapping[str, "type[upath.UPath]"]):
"az": "upath.implementations.cloud.AzurePath",
"data": "upath.implementations.data.DataPath",
"file": "upath.implementations.local.FilePath",
"ftp": "upath.implementations.ftp.FTPPath",
"local": "upath.implementations.local.FilePath",
"gcs": "upath.implementations.cloud.GCSPath",
"gs": "upath.implementations.cloud.GCSPath",
Expand Down Expand Up @@ -226,6 +228,8 @@ def get_upath_class(
@overload
def get_upath_class(protocol: Literal["data"]) -> type[_DataPath]: ...
@overload
def get_upath_class(protocol: Literal["ftp"]) -> type[_FTPPath]: ...
@overload
def get_upath_class(protocol: Literal["github"]) -> type[_GitHubPath]: ...
@overload
def get_upath_class(protocol: Literal["hdfs"]) -> type[_HDFSPath]: ...
Expand Down
38 changes: 38 additions & 0 deletions upath/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -649,3 +649,41 @@ def hf_fixture_with_readonly_mocked_hf_api(
hf_test_repo, mock_hf_api, mock_hf_filesystem_open
):
return "hf://" + hf_test_repo


@pytest.fixture(scope="module")
def ftp_server(tmp_path_factory):
"""Fixture providing a writable FTP filesystem."""
pytest.importorskip("pyftpdlib")

tmp_path = tmp_path_factory.mktemp("ftp-server")

P = subprocess.Popen(
[
sys.executable,
"-m",
"pyftpdlib",
"-d",
str(tmp_path),
"-u",
"user",
"-P",
"pass",
"-w",
]
)
try:
time.sleep(1)
yield {
"host": "localhost",
"port": 2121,
"username": "user",
"password": "pass",
}
finally:
P.terminate()
P.wait()
try:
shutil.rmtree(tmp_path)
except Exception:
pass
21 changes: 21 additions & 0 deletions upath/tests/implementations/test_ftp.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import pytest

from upath import UPath
from upath.tests.cases import BaseTests
from upath.tests.utils import skip_on_windows


@skip_on_windows
class TestUPathFTP(BaseTests):

@pytest.fixture(autouse=True)
def path(self, ftp_server):
self.path = UPath("", protocol="ftp", **ftp_server)
self.prepare_file_system()


def test_ftp_path_mtime(ftp_server):
path = UPath("file1.txt", protocol="ftp", **ftp_server)
path.touch()
mtime = path.stat().st_mtime
assert isinstance(mtime, float)
1 change: 1 addition & 0 deletions upath/tests/test_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"az",
"data",
"file",
"ftp",
"gcs",
"gs",
"hdfs",
Expand Down
29 changes: 29 additions & 0 deletions upath/types/storage_options.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
"AzureStorageOptions",
"HfStorageOptions",
"DataStorageOptions",
"FTPStorageOptions",
"GitHubStorageOptions",
"HDFSStorageOptions",
"HTTPStorageOptions",
Expand Down Expand Up @@ -204,6 +205,34 @@ class DataStorageOptions(_AbstractStorageOptions, total=False):
# No specific options for Data URIs at the moment


class FTPStorageOptions(_AbstractStorageOptions, total=False):
"""Storage options for FTP filesystem"""

# Connection settings
host: str # The remote server name/ip to connect to (required)
port: int # Port to connect with (default: 21)

# Authentication
username: (
str | None
) # User's identifier for authentication (anonymous if not given)
password: str | None # User's password on the server
acct: str | None # Account string for authentication (some servers require this)

# Performance settings
block_size: int | None # Read-ahead or write buffer size

# FTP-specific settings
tempdir: (
str | None
) # Directory on remote to put temporary files when in a transaction
timeout: int # Timeout of the FTP connection in seconds (default: 30)
encoding: str # Encoding for dir and filenames in FTP connection (default: "utf-8")

# Security settings
tls: bool # Use FTP-TLS (default: False)


class GitHubStorageOptions(_AbstractStorageOptions, total=False):
"""Storage options for GitHub repository filesystem"""

Expand Down
Loading