diff --git a/.github/workflows/deploy_mkdocs.yml b/.github/workflows/deploy_mkdocs.yml new file mode 100644 index 0000000..a331257 --- /dev/null +++ b/.github/workflows/deploy_mkdocs.yml @@ -0,0 +1,30 @@ +name: Publish docs via GitHub Pages +on: + push: + branches: + - main + +jobs: + build: + name: Deploy docs + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - name: Checkout main + uses: actions/checkout@v4 + + - name: Setup Python + uses: actions/setup-python@v5 + with: + python-version: '3.10' + + - name: Install dependencies + run: | + pip install mkdocs mkdocs-material + + - name: Deploy docs + run: | + mkdocs gh-deploy --force --clean + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d0bb2bc..abaea5a 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -12,6 +12,10 @@ env: jobs: auto-release: runs-on: ubuntu-latest + permissions: + contents: write + pull-requests: write + issues: write if: "!contains(github.event.head_commit.message, 'ci skip') && !contains(github.event.head_commit.message, 'skip ci')" steps: - uses: actions/checkout@v3 @@ -40,4 +44,4 @@ jobs: run: | ~/auto shipit -vv env: - GH_TOKEN: ${{ secrets.AUTO_TOKEN }} + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.gitignore b/.gitignore index af4ba71..72ed386 100644 --- a/.gitignore +++ b/.gitignore @@ -135,5 +135,7 @@ dmypy.json # Pycharm .idea/ +.serena/ + # Internal documentation internal/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..2d78a35 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,86 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +ReproSchema Python library and CLI for working with ReproSchema format - a standardized way to represent assessments, questionnaires, and protocols used in research. The library provides validation, conversion utilities, and integrations with REDCap and FHIR. + +## Key Commands + +### Development Setup +```bash +# Install in development mode +pip install -e . + +# Install with all development dependencies +pip install -e ".[dev]" + +# Run pre-commit hooks +pre-commit install +pre-commit run --all-files +``` + +### Testing +```bash +# Run tests with coverage +pytest --cov=reproschema + +# Run specific test file +pytest reproschema/tests/test_validate.py + +# Run tests in parallel +pytest -n auto +``` + +### Linting and Formatting +```bash +# Format code with black +black reproschema/ + +# Run flake8 linter +flake8 reproschema/ + +# Sort imports +isort reproschema/ + +# Check spelling +codespell +``` + +## Architecture + +### Core Modules + +- **reproschema/cli.py**: Main CLI entry point using Click framework. Provides commands for validation, conversion, migration. +- **reproschema/models/model.py**: Pydantic models auto-generated from LinkML schema. DO NOT modify directly - changes should be made in ReproNim/reproschema repository. +- **reproschema/validate.py**: Schema validation using PyShacl +- **reproschema/redcap2reproschema.py**: Convert REDCap CSV to ReproSchema format +- **reproschema/reproschema2redcap.py**: Convert ReproSchema to REDCap CSV format +- **reproschema/reproschema2fhir.py**: Convert ReproSchema to FHIR Questionnaire resources +- **reproschema/output2redcap.py**: Process reproschema-ui output into REDCap CSV + +### Conversion Workflows + +1. **REDCap → ReproSchema**: Requires CSV data dictionary + YAML config file +2. **ReproSchema → REDCap**: Takes protocol directory, outputs CSV +3. **ReproSchema → FHIR**: Converts activities/items to FHIR Questionnaire +4. **UI Output → REDCap**: Processes survey responses to REDCap format + +## CLI Commands + +- `reproschema validate ` - Validate ReproSchema format +- `reproschema redcap2reproschema ` - Convert REDCap to ReproSchema +- `reproschema reproschema2redcap ` - Convert ReproSchema to REDCap +- `reproschema reproschema2fhir ` - Convert to FHIR +- `reproschema output2redcap ` - Process UI output +- `reproschema migrate ` - Migrate to new schema version +- `reproschema convert` - Convert between formats (jsonld, n-triples, turtle) + +## Important Notes + +- Python 3.10+ required +- The Pydantic models in `reproschema/models/model.py` are auto-generated - DO NOT edit directly +- All schema changes must be made in the LinkML source at ReproNim/reproschema repository +- Uses pre-commit for code quality - commits may require running twice +- Line length limit: 79 characters (enforced by black) diff --git a/docs/cli_usage.md b/docs/cli_usage.md new file mode 100644 index 0000000..fdd423c --- /dev/null +++ b/docs/cli_usage.md @@ -0,0 +1,30 @@ +## CLI usage + +This package installs `reproschema` Command Line Interface (CLI). + +``` +$ reproschema --help + +$ A client to support interactions with ReproSchema + + To see help for a specific command, run + + reproschema COMMAND --help e.g. reproschema validate --help + +Options: + --version + -l, --log-level [DEBUG|INFO|WARNING|ERROR|CRITICAL] + Log level name [default: INFO] + --help Show this message and exit. + +Commands: + convert Converts a path to a different format, jsonld,... + create + migrate Updates to a new reproschema version + redcap2reproschema Converts REDCap CSV files to Reproschema format. + reproschema2redcap Converts reproschema protocol to REDCap CSV format. + serve + validate Validates if the path has a valid reproschema format + reproschema2fhir Generates FHIR questionnaire resources from reproschema activities + output2redcap Generates redcap csv given the audio and survey data from reproschema ui +``` diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..ed426c0 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,15 @@ +# Welcome to Reproschema-py + +For full documentation visit [Reproschema-py](https://github.com/ReproNim/reproschema-py). +reproschema requires Python 3.10+ + +## Project layout + mkdocs.yml # The configuration file. + docs/ + index.md # The documentation homepage. + installation.md #A tutorial for installation. + cli_usage.md #Notes on CLI usage + reproschema2redcap.md #How to convert from reproschema to redcap + redcap2reproschema.md #How to convert from redcap to reproschema + output2redcap.md #How to convert the output into redcap + reproschema2fhir.md #How to convert the reproschema into an FHIR Questionnaire Resource diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 0000000..eed9aca --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,25 @@ +## Installation + +Use the following command to install reproschema: + +``` +pip install reproschema +``` + +### Developer installation + +Fork this repo to your own GitHub account, then clone and install your forked repo in the developer mode: + +``` +git clone https://github.com//reproschema-py.git +cd reproschema-py +pip install -e . +``` +#### Style +This repo uses pre-commit to check styling. +- Install pre-commit with pip: `pip install pre-commit` +- In order to use it with the repository, you have to run `pre-commit install` in the root directory the first time you use it. + +When pre-commit is used, you may have to run git commit twice, +since pre-commit may make additional changes to your code for styling and will +not commit these changes by default. diff --git a/docs/output2redcap.md b/docs/output2redcap.md new file mode 100644 index 0000000..b0fa496 --- /dev/null +++ b/docs/output2redcap.md @@ -0,0 +1,10 @@ +## `output2redcap` +The `output2redcap` function is designed to process the output from reproschema-ui into a REDCap CSV file as seen [here](reproschema/example/redcap). + + +### CLI Usage + +The `output2redcap` function has been integrated into a CLI tool, use the following command: +```bash +reproschema output2redcap ./path/to/your_reproschema-ui_files ./path/to/directory_you_want_to_save_output +``` diff --git a/docs/redcap2reproschema.md b/docs/redcap2reproschema.md new file mode 100644 index 0000000..67a0691 --- /dev/null +++ b/docs/redcap2reproschema.md @@ -0,0 +1,45 @@ +## `redcap2reproschema` +The `redcap2reproschema` function is designed to process a given REDCap CSV file and YAML configuration to generate the output in the reproschema format. + +### Prerequisites +Before the conversion, ensure you have the following: + +**YAML Configuration File**: + - Download [templates/redcap2rs.yaml](templates/redcap2rs.yaml) and fill it out with your protocol details. + +### YAML File Configuration +In the `templates/redcap2rs.yaml` file, provide the following information: + +- **protocol_name**: A unique identifier for your protocol. Use underscores for spaces and avoid special characters. +- **protocol_display_name**: Name that will appear in the application. +- **protocol_description**: A brief description of your protocol. +- **redcap_version**: Version of your redcap file (you can customize it). + +Example: +```yaml +protocol_name: "My_Protocol" +protocol_display_name: "Assessment Protocol" +protocol_description: "This protocol is for assessing cognitive skills." +redcap_version: "X.XX.X" +``` +### CLI Usage + +The `redcap2reproschema` function has been integrated into a CLI tool, use the following command: +```bash +reproschema redcap2reproschema path/to/your_redcap_data_dic.csv path/to/your_redcap2rs.yaml +``` + +Optionally you can provide a path to the output directory (default is the current directory) by adding the option: `--output-path PATH` +### Python Function Usage + +You can also use the `redcap2reproschema` function from the `reproschema-py` package in your Python code. + +```python +from reproschema import redcap2reproschema + +csv_path = "path-to/your_redcap_data_dic.csv" +yaml_path = "path-to/your_redcap2rs.yaml" +output_path = "path-to/directory_you_want_to_save_output" + +redcap2reproschema(csv_path, yaml_path, output_path) +``` diff --git a/docs/reproschema2fhir.md b/docs/reproschema2fhir.md new file mode 100644 index 0000000..7a7529c --- /dev/null +++ b/docs/reproschema2fhir.md @@ -0,0 +1,11 @@ +## `reproschema2fhir` +The `reproschema2fhir` function is designed to convert reproschema activities and items into a FHIR Questionnaire resource as seen [here](reproschema/example/fhir). + +### CLI Usage + +The `reproschema2fhir` function has been integrated into a CLI tool, use the following command: +```bash +reproschema reproschema2fhir ./path/to/your_reproschema_activities ./path/to/directory_you_want_to_save_output +``` +### Notes +1. The script requires an active internet connection to access the GitHub repository. diff --git a/docs/reproschema2redcap.md b/docs/reproschema2redcap.md new file mode 100644 index 0000000..3b35ee3 --- /dev/null +++ b/docs/reproschema2redcap.md @@ -0,0 +1,31 @@ +## `reproschema2redcap` + +### CLI Usage + +You can use this feature directly from the command line. To convert ReproSchema protocol to REDCap CSV format, use the following command + +``` +reproschema reproschema2redcap +``` + +- ``: The path to the root folder of a protocol. For example, to convert the reproschema-demo-protocol provided by ReproNim, you can use the following commands: + ```bash + git clone https://github.com/ReproNim/reproschema-demo-protocol.git + cd reproschema-demo-protocol + pwd + ``` + In this case, the output from `pwd` (which shows your current directory path) should be your ``. +- ``: The name of the output CSV file where the converted data will be saved. + +### Python Function Usage + +You can also use the `reproschema2redcap` function from the `reproschema-py` package in your Python code. + +```python +from reproschema import reproschema2redcap + +input_dir_path = "path-to/reproschema-demo-protocol" +output_csv_filename = "output.csv" + +reproschema2redcap(input_dir_path, output_csv_filename) +``` diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..969336d --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,11 @@ +site_name: reproschema-py +nav: + - Home: index.md + - Installation: installation.md + - CLI Usage: cli_usage.md + - Converting from reproschema to redcap: reproschema2redcap.md + - Converting from redcap to reproschema: redcap2reproschema.md + - Converting the output into redcap: output2redcap.md + - Converting the reproschema into an FHIR Questionnaire Resource: reproschema2fhir.md + +theme: readthedocs