Merge branch 'main' into license-classify-modeling

Author: TimidRobot

README.md

@@ -9,7 +9,7 @@ This project seeks to quantify the size and diversity of the commons--the
 collection of works that are openly licensed or in the public domain.
 
 
-## Code of Conduct
+## Code of conduct
 
 [`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md):
 > The Creative Commons team is committed to fostering a welcoming community.
@@ -27,59 +27,98 @@ collection of works that are openly licensed or in the public domain.
 See [`CONTRIBUTING.md`](CONTRIBUTING.md).
 
 
-
 ## Development
 
 
 ### Prerequisites
 
 This repository uses [pipenv][pipenvdocs] to manage the required Python
 modules:
-- Linux: [Installing Pipenv][pipenvinstall]
-- macOS:
-  1. Install [Homebrew][homebrew]
-  2. Install pipenv:
-        ```
+1. Install `pipenv`:
+   - Linux: [Installing Pipenv][pipenvinstall]
+   - macOS:
+     1. Install [Homebrew][homebrew]
+     2. Install pipenv:
+        ```shell
         brew install pipenv
         ```
+   - Windows: [Installing Pipenv][pipenvinstall]
+2. Create the Python virtual environment and install prerequisites using
+   `pipenv`:
+    ```shell
+    pipenv sync --dev
+    ```
 
 [pipenvdocs]: https://pipenv.pypa.io/en/latest/
+[pipenvinstall]: https://pipenv.pypa.io/en/latest/installation/
 [homebrew]: https://brew.sh/
-[pipenvinstall]: https://pipenv.pypa.io/en/latest/install/#installing-pipenv
 
 
-### Running Scripts that Require Client Credentials
+### Running scripts that require client cedentials
 
-To successfully run scripts that require client credentials, you will need to follow these steps:
-  1. Copy the contents of the `env.example` file in the script's directory to `.env`:
-        ```
-        cp env.example .env
-        ```
-  2. Uncomment the variables in the `.env` file and assign values as needed. See [`sources.md`](sources.md) on how to get credentials:
-        ```
-        GOOGLE_API_KEYS=your_api_key
-        PSE_KEY=your_pse_key
-       ```
-  3. Save the changes to the `.env` file.
+To successfully run scripts that require client credentials, you will need to
+follow these steps:
+1. Copy the contents of the `env.example` file in the script's directory to
+   `.env`:
+    ```shell
+    cp env.example .env
+    ```
+2. Uncomment the variables in the `.env` file and assign values as needed. See
+   [`sources.md`](sources.md) on how to get credentials:
+    ```
+    GOOGLE_API_KEYS=your_api_key
+    PSE_KEY=your_pse_key
+    ```
+3. Save the changes to the `.env` file.
+4. You should now be able to run scripts that require client credentials
+   without any issues.
+
+
+### Static analysis
 
-  4. You should now be able to run scripts that require client credentials without any issues.
+The [`dev/tools.sh`][tools-sh] helper script runs the static analysis tools
+(`black`, `flake8`, and `isort`):
+```shell
+./dev/tools.sh
+```
 
+It can also accept command-line arguments to specify specific files or
+directories to check:
+```shell
+./dev/tools.sh PATH/TO/MY/FILE.PY
+```
 
-### Tooling
+[tools-sh]: /dev/tools.sh
+
+
+### Resources
 
 - **[Python Guidelines — Creative Commons Open Source][ccospyguide]**
-- [Black][black]: the uncompromising Python code formatter
-- [flake8][flake8]: a python tool that glues together pep8, pyflakes, mccabe,
-  and third-party plugins to check the style and quality of some python code.
-- [isort][isort]: A Python utility / library to sort imports.
+- [Black][black]: _the uncompromising Python code formatter_
+- [flake8][flake8]: _a python tool that glues together pep8, pyflakes, mccabe,
+  and third-party plugins to check the style and quality of some python code._
+- [isort][isort]: _A Python utility / library to sort imports_
+  - (It doesn't import any libraries, it only sorts and formats them.)
+- [ppypa/pipenv][pipenv]: _Python Development Workflow for Humans._
 
 [ccospyguide]: https://opensource.creativecommons.org/contributing-code/python-guidelines/
 [black]: https://github.com/psf/black
 [flake8]: https://gitlab.com/pycqa/flake8
 [isort]: https://pycqa.github.io/isort/
+[pipenv]: https://github.com/pypa/pipenv
+
+
+### GitHub Actions
+
+The [`.github/workflows/python_static_analysis.yml`][workflow-static-analysis]
+GitHub Actions workflow performs static analysis (`black`, `flake8`, and
+`isort`) on committed changes. The workflow is triggered automatically when you
+push changes to the main branch or open a pull request.
+
+[workflow-static-analysis]: .github/workflows/python_static_analysis.yml
 
 
-## Data Sources
+## Data sources
 
 Kindly visit the [`sources.md`](sources.md) file for it.
 
@@ -89,12 +128,13 @@ Kindly visit the [`sources.md`](sources.md) file for it.
 For information on past efforts, see [`history.md`](history.md).
 
 
-## Copying & License
+## Copying & license
 
 
 ### Code
 
-[`LICENSE`](LICENSE): the code within this repository is licensed under the Expat/[MIT][mit] license.
+[`LICENSE`](LICENSE): the code within this repository is licensed under the
+Expat/[MIT][mit] license.
 
 [mit]: http://www.opensource.org/licenses/MIT "The MIT License | Open Source Initiative"
 
@@ -119,4 +159,4 @@ The documentation within the project is licensed under a [Creative Commons
 Attribution 4.0 International License][cc-by].
 
 [cc-by-png]: https://licensebuttons.net/l/by/4.0/88x31.png#floatleft "CC BY 4.0 license button"
-[cc-by]: https://creativecommons.org/licenses/by/4.0/ "Creative Commons Attribution 4.0 International License"
+[cc-by]: https://creativecommons.org/licenses/by/4.0/ "Creative Commons Attribution 4.0 International License"

flickr/data_cleaning.py

@@ -57,7 +57,7 @@ def save_new_data(
 
 
 def main():
-    drop_empty_column("final.csv", "dataset/cleaned_license10.csv")
+    drop_empty_column("hs.csv", "dataset/cleaned_license10.csv")
     drop_duplicate_id(
         "dataset/cleaned_license10.csv", "dataset/cleaned_license10.csv"
     )