Skip to content

Commit 2bdf3c6

Browse files
committed
Update DefaultStacIO to fix parsing ascii in urls
1 parent 7621455 commit 2bdf3c6

File tree

2 files changed

+18
-6
lines changed

2 files changed

+18
-6
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,8 @@ python -m pip install 'pystac[orjson]'
3535
```
3636

3737
If you would like to use a custom `RetryStacIO` class for automatically retrying
38-
network requests when reading with PySTAC, you'll need
38+
network requests when reading with PySTAC, or if you have non-ASCII characters in
39+
your urls you'll need
3940
[`urllib3`](https://urllib3.readthedocs.io/en/stable/):
4041

4142
```shell

pystac/stac_io.py

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -286,8 +286,9 @@ def read_text_from_href(self, href: str) -> str:
286286
"""Reads file as a UTF-8 string.
287287
288288
If ``href`` has a "scheme" (e.g. if it starts with "https://") then this will
289-
use :func:`urllib.request.urlopen` to open the file and read the contents;
290-
otherwise, :func:`open` will be used to open a local file.
289+
use :func:`urllib.request.urlopen` (or func:`urllib3.request` if available)
290+
to open the file and read the contents; otherwise, :func:`open` will be used
291+
to open a local file.
291292
292293
Args:
293294
@@ -297,9 +298,19 @@ def read_text_from_href(self, href: str) -> str:
297298
if _is_url(href):
298299
try:
299300
logger.debug(f"GET {href} Headers: {self.headers}")
300-
req = Request(href, headers=self.headers)
301-
with urlopen(req) as f:
302-
href_contents = f.read().decode("utf-8")
301+
if HAS_URLLIB3:
302+
with urllib3.request(
303+
"GET",
304+
href,
305+
headers=self.headers,
306+
preload_content=False, # type: ignore
307+
) as f:
308+
href_contents = f.read().decode("utf-8")
309+
else:
310+
req = Request(href, headers=self.headers)
311+
with urlopen(req) as f:
312+
href_contents = f.read().decode("utf-8")
313+
303314
except HTTPError as e:
304315
raise Exception(f"Could not read uri {href}") from e
305316
else:

0 commit comments

Comments
 (0)