- 
                Notifications
    You must be signed in to change notification settings 
- Fork 126
          Allow links to have file:// prefix - but don't write them that way by default
          #1489
        
          New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| Codecov ReportAttention: Patch coverage is  
 
 Additional details and impacted files@@            Coverage Diff             @@
##             main    #1489      +/-   ##
==========================================
- Coverage   91.31%   91.22%   -0.09%     
==========================================
  Files          53       53              
  Lines        7286     7296      +10     
  Branches      883      887       +4     
==========================================
+ Hits         6653     6656       +3     
- Misses        453      459       +6     
- Partials      180      181       +1     ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
 | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems fine to me, but looks like windoze is cranky 😢
| Ok jeez. Down to the one failing Python 3.10 windows test: ================================== FAILURES ===================================
__________ StacIOTest.test_read_write_collection_with_file_protocol ___________
self = <tests.test_stac_io.StacIOTest testMethod=test_read_write_collection_with_file_protocol>
    def test_read_write_collection_with_file_protocol(self) -> None:
        collection = pystac.read_file(
            "file://" + TestCases.get_path("data-files/collections/multi-extent.json")
        )
        with tempfile.TemporaryDirectory() as tmp_dir:
            dest_href = os.path.join(tmp_dir, "collection.json")
>           pystac.write_file(collection, dest_href="file://" + dest_href)
tests\test_stac_io.py:33: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pystac\__init__.py:201: in write_file
    obj.save_object(
pystac\stac_object.py:476: in save_object
    stac_io.save_json(dest_href, self.to_dict(include_self_link=include_self_link))
pystac\collection.py:581: in to_dict
    d = super().to_dict(
pystac\catalog.py:645: in to_dict
    "links": [link.to_dict(transform_href=transform_hrefs) for link in links],
pystac\catalog.py:645: in <listcomp>
    "links": [link.to_dict(transform_href=transform_hrefs) for link in links],
pystac\link.py:387: in to_dict
    "href": self.get_href(transform_href=transform_href),
pystac\link.py:181: in get_href
    and self.owner.get_root()
pystac\stac_object.py:325: in get_root
    root_link.resolve_stac_object()
pystac\link.py:330: in resolve_stac_object
    obj = stac_io.read_stac_object(target_href, root=root)
pystac\stac_io.py:235: in read_stac_object
    d = self.read_json(source, *args, **kwargs)
pystac\stac_io.py:206: in read_json
    txt = self.read_text(source, *args, **kwargs)
pystac\stac_io.py:283: in read_text
    return self.read_text_from_href(href)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pystac.stac_io.DefaultStacIO object at 0x0000023DC7CA3E80>
href = '/D:/a/pystac/pystac/tests/data-files/collections/multi-extent.json'
    def read_text_from_href(self, href: str) -> str:
        """Reads file as a UTF-8 string.
    
        If ``href`` has a "scheme" (e.g. if it starts with "https://") then this will
        use :func:`urllib.request.urlopen` to open the file and read the contents;
        otherwise, :func:`open` will be used to open a local file.
    
        Args:
    
            href : The URI of the file to open.
        """
        href_contents: str
        if _is_url(href):
            try:
                logger.debug(f"GET {href} Headers: {self.headers}")
                req = Request(href, headers=self.headers)
                with urlopen(req) as f:
                    href_contents = f.read().decode("utf-8")
            except HTTPError as e:
                raise Exception(f"Could not read uri {href}") from e
        else:
            href = safe_urlparse(href).path
>           with open(href, encoding="utf-8") as f:
E           OSError: [Errno 22] Invalid argument: '/D:/a/pystac/pystac/tests/data-files/collections/multi-extent.json'
I guess it is probably the leading slash that is the problem. | 
| I checked when we can deprecate Python 3.10 and it's not until April (https://numpy.org/neps/nep-0029-deprecation_policy.html) so I'll keep chugging | 
file:/// prefix - but don't write them that way by defaultfile:// prefix - but don't write them that way by default
      …by default (stac-utils#1489) * Allow reading and writing when href startswith file:/// * Add test of reading and writing to hrefs starting with file:/// * Ensure that file:/// are interpretted as absolute urls * Try to fix windows * Try to fix windows * Try to fix windows * Add some print debugging * Add more print debugging * Add more print debugging * Moved is_absolute to os dependent test * Fix os-dependent test * Strip initial slash and see if it passes * Add more print debugging * Just strip off the leading slash * Only for windows * Fix windows handling --------- Co-authored-by: Pete Gadomski <[email protected]>
Related Issue(s):
related to #1347 milder version of #1472 (that one should wait until v2.0)
Description:
Allow links to start with
file:///but don't write them that way. Also handles reading/writing to path that is prefixed withfile:///PR Checklist:
scripts/test)