Upgrade packaging process for V2 beta release (#215)

Packaging with pyproject.toml and Hatch
- Add pyproject.toml. Set Python ≥3.10.
- Introduce Hatch for packaging and dependency management.
- Update cloudbuild.yaml and run_test.sh to align with Hatch.

Author: jm-rivera

CHANGELOG.md

@@ -1,5 +1,8 @@
 # Changelog
 
+## 2.0.0 [forthcoming]
+
+
 ## 1.4.3
 
 **Date** - 11/10/2020

cloudbuild.yaml

@@ -1,7 +1,7 @@
 steps:
   - id: api_python
     name: python:3.10-slim
-    entrypoint: /bin/sh
+    entrypoint: /bin/bash
     args:
       - -c
-      - "./run_test.sh -s -a"
+      - "./run_test.sh -s && hatch run test:all"

datacommons_client/README.md

@@ -0,0 +1,46 @@
+# Data Commons Python API
+
+This is a Python library for accessing data in the Data Commons Graph.
+
+To get started, install this package from pip.
+
+```bash
+pip install datacommons-client
+```
+
+To get additional functionality to work with Pandas DataFrames, install the package
+with the optional Pandas dependency.
+
+```bash
+pip install "datacommons-client[Pandas]"
+```
+
+Once the package is installed, import `datacommons_client`.
+
+```python
+import datacommons_client as dc
+```
+
+For more detail on getting started with the API, please visit our
+[API Overview](https://docs.datacommons.org/api/).
+
+When you are ready to use the API, you can refer to `examples` for
+examples on how to use this package to perform various tasks. More tutorials and
+documentation can be found on our [tutorials page](https://docs.datacommons.org/tutorials/)!
+
+## About Data Commons
+
+[Data Commons](https://datacommons.org/) is an open knowledge repository that
+provides a unified view across multiple public data sets and statistics. You can
+view what [datasets](https://datacommons.org/datasets) are currently ingested
+and browse the graph using our [browser](https://datacommons.org/browser).
+
+## License
+
+Apache 2.0
+
+## Support
+
+For general questions or issues about the API, please open an issue on our
+[issues](https://github.com/google/datacommons/issues) page. For all other
+questions, please send an email to `[email protected]`.

datacommons_client/__init__.py

@@ -0,0 +1,20 @@
+__version__ = "2.0.0b1"
+"""
+Data Commons Client Package
+
+This package provides a Python client for interacting with the Data Commons API.
+"""
+
+from datacommons_client.client import DataCommonsClient
+from datacommons_client.endpoints.base import API
+from datacommons_client.endpoints.node import NodeEndpoint
+from datacommons_client.endpoints.observation import ObservationEndpoint
+from datacommons_client.endpoints.resolve import ResolveEndpoint
+
+__all__ = [
+    "DataCommonsClient",
+    "API",
+    "NodeEndpoint",
+    "ObservationEndpoint",
+    "ResolveEndpoint",
+]

datacommons_client/endpoints/node.py

@@ -3,8 +3,7 @@
 from datacommons_client.endpoints.base import API
 from datacommons_client.endpoints.base import Endpoint
 from datacommons_client.endpoints.payloads import NodeRequestPayload
-from datacommons_client.endpoints.payloads import \
-    normalize_properties_to_string
+from datacommons_client.endpoints.payloads import normalize_properties_to_string
 from datacommons_client.endpoints.response import NodeResponse
 
 

datacommons_client/endpoints/payloads.py

@@ -5,8 +5,7 @@
 from enum import Enum
 from typing import Optional
 
-from datacommons_client.utils.error_handling import \
-    InvalidObservationSelectError
+from datacommons_client.utils.error_handling import InvalidObservationSelectError
 
 
 def normalize_properties_to_string(properties: str | list[str]) -> str:

datacommons_client/tests/endpoints/test_payloads.py

@@ -5,8 +5,7 @@
 from datacommons_client.endpoints.payloads import ObservationRequestPayload
 from datacommons_client.endpoints.payloads import ObservationSelect
 from datacommons_client.endpoints.payloads import ResolveRequestPayload
-from datacommons_client.utils.error_handling import \
-    InvalidObservationSelectError
+from datacommons_client.utils.error_handling import InvalidObservationSelectError
 
 
 def test_node_payload_normalize():

datacommons_client/tests/endpoints/test_resolve_endpoint.py

@@ -1,8 +1,7 @@
 from unittest.mock import MagicMock
 
 from datacommons_client.endpoints.base import API
-from datacommons_client.endpoints.resolve import \
-    _resolve_correspondence_expression
+from datacommons_client.endpoints.resolve import _resolve_correspondence_expression
 from datacommons_client.endpoints.resolve import flatten_resolve_response
 from datacommons_client.endpoints.resolve import ResolveEndpoint
 from datacommons_client.endpoints.response import ResolveResponse

docs/development.md

@@ -1,37 +1,55 @@
 # Python API Development
 
-The Python API currently supports `python>=3.7`.
+This client library supports `python>=3.10`.
 
-To set up the Python environment for tests, run:
+## Set up
+If you haven't already, clone this repository.
+
+```bash
+git clone https://github.com/datacommonsorg/api-python.git
+cd api-python
+```
+
+To set up the Python environment for development, run:
 
 ```bash
 ./run_test.sh -s
 ```
 
-To test, run:
+This will install `hatch`, which is the main tool used to manage the
+environment, dependencies, and development tools. You can also manually install
+`hatch` and create a virtual environment.
 
 ```bash
-./run_test.sh -a
+pip install hatch
+hatch env create
 ```
 
-To format, run:
+## Code style and linting
+We use `isort` and `yapf` for code formatting. Check formatting with:
 
 ```bash
-./run_test.sh -f
+hatch run lint:check
 ```
 
-To debug the continuous integration tests, run:
+To automatically fix formatting run:
 
 ```bash
-gcloud builds submit . --project=datcom-ci --config=cloudbuild.yaml
+hatch run lint:format
 ```
 
-Both commands will run the same set of tests.
+## Running tests
 
-To run the examples:
+To test, run:
 
 ```bash
-python -m datacommons.examples.XXX
+hatch run test:all
+```
+
+To debug the continuous integration tests, run:
+
+```bash
+gcloud builds submit . --project=datcom-ci --config=cloudbuild.yaml
 ```
 
-where XXX is the module you want to run.
+Both commands will run the same set of tests.

docs/release.md

@@ -1,5 +1,41 @@
 # Python API Release
 
+## Releasing the `datacommons_client` package
+Support for V2 of the Data Commons API is being released as a new client library
+called `datacommons_client`.
+
+To release:
+1. Update [CHANGELOG.md](../CHANGELOG.md) with relevant changes.
+2. Bump the version by running `hatch version` followed by `patch`, `minor`, `major`, a 
+specific version number, or `--pre beta` for a beta version, for example.
+3. Build the package
+```bash
+hatch build
+```
+4. (optionally) Test the deployment process locally
+```bash
+hatch run release:localtest
+```
+5. Test the deployment process on Test PyPi
+```bash
+hatch run release:testpypi
+```
+
+6. Once verified, upload to PyPI:
+```bash
+hatch run release:pypi
+```
+
+7. Create a version tag on Git:
+```bash
+hatch run release:tag
+```
+
+---
+
+## Releasing the legacy packages
+
+
 Note: Always release `datacommons_pandas` when `datacommons` is released.
 
 **If this is your first time releasing to PyPI**, please review the PyPI guide