Merge branch 'main' into wikipedia

Author: TimidRobot

Pipfile

@@ -8,18 +8,19 @@ flickrapi = "*"
 GitPython = "*"
 google-api-python-client = "*"
 h11 = ">=0.16.0"  # Ensure dependency is secure
-internetarchive = "*"
+internetarchive = ">=5.5.1"
 jupyterlab = ">=3.6.7"
 matplotlib = "*"
 numpy = "*"
 pandas = "*"
 plotly = "*"
+pillow = ">=11.3.0"  # Ensure dependency is secure
 Pyarrow = "*"
 Pygments = "*"
 python-dotenv = "*"
 requests = ">=2.31.0"
 seaborn = "*"
-urllib3 = ">=1.26.18"
+urllib3 = ">=2.5.0"
 wordcloud = "*"
 
 [dev-packages]

Pipfile.lock

@@ -9,6 +9,23 @@ This project seeks to quantify the size and diversity of the commons--the
 collection of works that are openly licensed or in the public domain.
 
 
+### Meaningful
+
+The reports generated by this project (and the data fetched and processed to
+support it) seeks to be meaningful. We hope this project will provide data and
+analysis that helps inform discussions about the commons--the collection of
+works that are openly licensed or in the public domain.
+
+The goal of this project is to help answer questions like:
+- How has the world's use of the commons changed over time?
+- How is the knowledge and culture of the commons distributed?
+  - Who has access (and how much) to the commons?
+- What significant trends can be observed in the commons?
+  - Which public domain dedication or licenses are the most popular?
+  - What are the correlations between public domain dedication or licenses and
+    region, language, domain/endeavor, etc.?
+
+
 ## Code of conduct
 
 [`CODE_OF_CONDUCT.md`][org-coc]:
@@ -57,6 +74,7 @@ Quantifying/
 │   ├── 1-fetch/
 │   ├── 2-process/
 │   ├── 3-report/
+│   ├── plot.py # Data visualizations with matplotlib
 │   └── shared.py
 ├── .cc-metadata.yml
 ├── .flake8  # Python tool configuration
@@ -105,10 +123,9 @@ modules:
 [homebrew]: https://brew.sh/
 
 
-### Running scripts that require client credentials
+### Managing client credentials
 
-To successfully run scripts that require client credentials, you will need to
-follow these steps:
+Client credentials should be stored in an environment file:
 1. Copy the contents of the `env.example` file in the script's directory to
    `.env`:
     ```shell
@@ -121,8 +138,22 @@ follow these steps:
     GCS_CX = your_pse_id
     ```
 3. Save the changes to the `.env` file.
-4. You should now be able to run scripts that require client credentials
-   without any issues.
+
+You should now be able to run scripts that require client credentials without
+any issues. The `.env` file is ignored by git to help ensure sensitive data is
+not distributed.
+
+
+### Running the scripts
+
+All of the scripts should be run from the root of the repository using pipenv. For example:
+```bash
+pipenv run ./scripts/1-fetch/github_fetch.py -h
+```
+
+When run this way, the shared library (`scripts/shared.py`) provides easy access
+to all of the necessary paths and all of the modules managed by pipenv are
+available.
 
 
 ### Static analysis