Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 79 additions & 5 deletions .github/workflows/test-docs-scripts.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Test Documentation Scripts
name: Documentation Scripts

# This workflow tests the documentation setup scripts to ensure they work correctly.
# It runs nightly and on-demand to avoid slowing down regular development workflow,
Expand All @@ -14,7 +14,7 @@ on:
# Allow manual triggering for testing

jobs:
test-system-setup:
ubuntu:
name: Test System Setup Scripts (${{ matrix.os }})
runs-on: ${{ matrix.os }}
# Only run if the event is scheduled, manual, or PR has test-doc-scripts label
Expand Down Expand Up @@ -58,11 +58,61 @@ jobs:
source ~/.cargo/env
./website/docs/developers/scripts/setup/format-and-lint.sh

- name: Test run tests
- name: Verify installations
run: |
echo "Verifying installed tools..."
source ~/.cargo/env
rustc --version
cargo --version
node --version
npm --version
protoc --version
sqlite3 --version
wasm-pack --version

macos:
name: Test System Setup Scripts (${{ matrix.os }})
runs-on: ${{ matrix.os }}
# Only run if the event is scheduled, manual, or PR has test-doc-scripts label
# This prevents long-running tests from blocking regular development workflow
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || contains(github.event.pull_request.labels.*.name, 'test-doc-scripts')
strategy:
matrix:
os: [macos-latest]

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Test system dependencies installation
run: |
./website/docs/developers/scripts/setup/install-system-deps-macos.sh

- name: Test Node.js installation
run: |
./website/docs/developers/scripts/setup/install-nodejs-macos.sh

- name: Test Rust installation
run: |
./website/docs/developers/scripts/setup/install-rust.sh

- name: Test WASM tools installation
run: |
# Source cargo environment first
source ~/.cargo/env
./website/docs/developers/scripts/setup/run-tests.sh
./website/docs/developers/scripts/setup/install-wasm-tools.sh

- name: Test specialised builds
run: |
# Source cargo environment first
source ~/.cargo/env
./website/docs/developers/scripts/setup/build-specialized.sh

- name: Test format and lint
run: |
# Source cargo environment first
source ~/.cargo/env
./website/docs/developers/scripts/setup/format-and-lint.sh

- name: Verify installations
run: |
Expand All @@ -76,7 +126,7 @@ jobs:
sqlite3 --version
wasm-pack --version

test-docker-setup:
docker-ubuntu:
name: Test Docker Setup Script (${{ matrix.os }})
runs-on: ${{ matrix.os }}
# Only run if the event is scheduled, manual, or PR has test-doc-scripts label
Expand All @@ -98,3 +148,27 @@ jobs:
run: |
sudo docker --version
sudo docker run hello-world

docker-macos:
name: Test Docker Setup Script (${{ matrix.os }})
runs-on: ${{ matrix.os }}
# Only run if the event is scheduled, manual, or PR has test-doc-scripts label
# This prevents long-running tests from blocking regular development workflow
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || contains(github.event.pull_request.labels.*.name, 'test-doc-scripts')
strategy:
matrix:
os: [macos-latest]

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Test Docker installation
run: |
./website/docs/developers/scripts/setup/install-docker-macos.sh

- name: Verify Docker installation
run: |
# Docker Desktop needs to be started manually on macOS
# We'll just check if it was installed via Homebrew
docker --version || echo "Docker installed but not running (expected on macOS CI)"
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,9 @@ cargo-build-test.json
tests.tsv

# Generated API docs should not be committed
website/static/api-docs/
website/static/api-docs/

ledger/3.0.0devnet
ledger/3.0.0mainnet
ledger/3.0.1devnet
ledger/berkeley_rc1
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Development Tools**: Enhanced formatting infrastructure with MDX file
support and trailing whitespace management
([#1234](https://github.com/o1-labs/openmina/pull/1234))
- **Documentation**: add instructions for developers using MacOS based systems,
and test the installation commands in CI
([#1247](https://github.com/o1-labs/openmina/pull/1234)).

### Changed

Expand Down
49 changes: 49 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,55 @@ referencing Claude.
6. **Verify commit message contains no emojis and follows 80-character wrap**
7. Proceed with testing or committing changes

### Documentation Script Testing

When modifying developer setup scripts in `website/docs/developers/scripts/`,
always test them using the documentation testing workflow:

#### Testing Documentation Scripts

The project includes automated testing of developer setup scripts to ensure they
work correctly across different platforms. This prevents developers from
encountering broken installation instructions.

**When to test:**

- After modifying any script in `website/docs/developers/scripts/setup/`
- When adding new dependencies or tools to the setup process
- When changing installation procedures
- When adding support for a new distribution or platform

**How to trigger tests:**

1. **For PRs**: Add the `test-doc-scripts` label to your pull request
2. **Manual testing**: Use GitHub CLI:
`gh pr edit <PR_NUMBER> --add-label test-doc-scripts`
3. **Remove and re-add**: If tests need to be re-run, remove the label first:
```bash
gh pr edit <PR_NUMBER> --remove-label test-doc-scripts
gh pr edit <PR_NUMBER> --add-label test-doc-scripts
```

**What gets tested:**

- System dependencies installation (Ubuntu/macOS)
- Rust toolchain setup (including taplo, wasm-pack, etc.)
- Node.js installation
- Docker installation
- Build processes and formatting tools
- Tool version verification

**Why this matters:**

- Ensures documentation stays current with actual requirements
- Prevents "command not found" errors for new developers
- Tests across multiple platforms (Ubuntu 22.04, 24.04, macOS)
- Catches environment drift and dependency changes
- Runs nightly to detect breaking changes early

The tests are designed to run on-demand via labels to avoid slowing down regular
development workflow, as they can take significant time to complete.

### Critical Pre-Commit Requirements

- **MANDATORY**: Run `make fix-trailing-whitespace` before every commit
Expand Down
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ fix-trailing-whitespace: ## Remove trailing whitespaces from all files
-not -path "./website/static/api-docs/*" \
-not -path "./website/.docusaurus/*" \
-not -path "./.git/*" \
-exec sed -i 's/[[:space:]]*$$//' {} + && \
-exec sed -i'' -e "s/[[:space:]]*$$//" {} + && \
echo "Trailing whitespaces removed."

.PHONY: check-trailing-whitespace
Expand Down Expand Up @@ -139,7 +139,10 @@ clean: ## Clean build artifacts
download-circuits: ## Download the circuits used by Mina from GitHub
@if [ ! -d "circuit-blobs" ]; then \
git clone --depth 1 https://github.com/openmina/circuit-blobs.git; \
ln -s -b "$$PWD"/circuit-blobs/* ledger/; \
ln -s "$$PWD"/circuit-blobs/3.0.0devnet ledger/; \
ln -s "$$PWD"/circuit-blobs/3.0.0mainnet ledger/; \
ln -s "$$PWD"/circuit-blobs/3.0.1devnet ledger/; \
ln -s "$$PWD"/circuit-blobs/berkeley_rc1 ledger/; \
else \
echo "circuit-blobs already exists, skipping download."; \
fi
Expand Down
76 changes: 69 additions & 7 deletions website/docs/developers/getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,18 @@ sidebar_position: 1
# Getting Started for Developers

import CodeBlock from "@theme/CodeBlock";
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
import InstallRustSh from "!!raw-loader!./scripts/setup/install-rust.sh";
import InstallSystemDepsSh from "!!raw-loader!./scripts/setup/install-system-deps.sh";
import InstallSystemDepsMacOSSh from "!!raw-loader!./scripts/setup/install-system-deps-macos.sh";
import InstallNodejsSh from "!!raw-loader!./scripts/setup/install-nodejs.sh";
import InstallNodejsMacOSSh from "!!raw-loader!./scripts/setup/install-nodejs-macos.sh";
import InstallDockerSh from "!!raw-loader!./scripts/setup/install-docker.sh";
import InstallDockerMacOSSh from "!!raw-loader!./scripts/setup/install-docker-macos.sh";
import InstallWasmToolsSh from "!!raw-loader!./scripts/setup/install-wasm-tools.sh";
import CloneAndBuildSh from "!!raw-loader!./scripts/setup/clone-and-build.sh";
import BuildSpecializedSh from "!!raw-loader!./scripts/setup/build-specialized.sh";
import RunTestsSh from "!!raw-loader!./scripts/setup/run-tests.sh";
import FormatAndLintSh from "!!raw-loader!./scripts/setup/format-and-lint.sh";

Welcome to OpenMina development! This guide will help you set up your
Expand All @@ -22,8 +26,10 @@ development environment and build OpenMina from source.

### System Requirements

- **Operating System**: Ubuntu 20.04+ or Debian 11+ (other Linux distributions
may work but are not officially supported)
- **Operating System**:
- **Linux**: Ubuntu 20.04+ or Debian 11+ (other Linux distributions may work
but are not officially supported)
- **macOS**: macOS 12.0+ (Monterey or later)
- **Memory**: At least 8GB RAM (16GB recommended)
- **Storage**: At least 20GB free space
- **Network**: Stable internet connection for downloading dependencies
Expand All @@ -32,24 +38,64 @@ development environment and build OpenMina from source.

#### 1. Rust Toolchain

OpenMina requires both stable and nightly Rust toolchains:
OpenMina requires both stable and nightly Rust toolchains, plus additional tools
for development:

<CodeBlock language="bash">{InstallRustSh}</CodeBlock>

This installs:

- Rust 1.84 (stable) and nightly toolchains
- Required components: `rustfmt`, `clippy`, `rust-src`
- `taplo-cli`: TOML formatter required for `make format`

#### 2. System Dependencies

<Tabs groupId="operating-systems">
<TabItem value="linux" label="Linux (Ubuntu/Debian)">

<CodeBlock language="bash">{InstallSystemDepsSh}</CodeBlock>

</TabItem>
<TabItem value="macos" label="macOS">

<CodeBlock language="bash">{InstallSystemDepsMacOSSh}</CodeBlock>

</TabItem>
</Tabs>

#### 3. Additional Development Tools

**Node.js (for documentation and frontend):**

<Tabs groupId="operating-systems">
<TabItem value="linux" label="Linux (Ubuntu/Debian)">

<CodeBlock language="bash">{InstallNodejsSh}</CodeBlock>

</TabItem>
<TabItem value="macos" label="macOS">

<CodeBlock language="bash">{InstallNodejsMacOSSh}</CodeBlock>

</TabItem>
</Tabs>

**Docker (optional, for containerized builds):**

<Tabs groupId="operating-systems">
<TabItem value="linux" label="Linux (Ubuntu/Debian)">

<CodeBlock language="bash">{InstallDockerSh}</CodeBlock>

</TabItem>
<TabItem value="macos" label="macOS">

<CodeBlock language="bash">{InstallDockerMacOSSh}</CodeBlock>

</TabItem>
</Tabs>

#### 4. WASM Tools (for web node development)

<CodeBlock language="bash">{InstallWasmToolsSh}</CodeBlock>
Expand All @@ -69,7 +115,11 @@ components:

### 3. Run Tests

<CodeBlock language="bash">{RunTestsSh}</CodeBlock>
You can run a different set of tests. The command `make help` will give you the
whole set of targets. You can also visit the file
[ci.yaml](https://github.com/o1-labs/openmina/blob/develop/.github/workflows/ci.yaml)
to explore the targets used for testing in our continuous integration
environment.

## Development Workflow

Expand Down Expand Up @@ -108,12 +158,24 @@ make clean # Clean build artifacts

### Environment Configuration

Some components require environment variables:
Some components require environment variables and database setup:

#### SQLite Database (Required for `make check`)

The heartbeats processor requires an SQLite database to be initialized before
running `make check`:

```bash
# For SQLx database operations (heartbeats processor)
# Create the SQLite database with required schema
sqlite3 /tmp/heartbeats.db < tools/heartbeats-processor/schema.sql

# Set the database URL environment variable
export DATABASE_URL="sqlite:///tmp/heartbeats.db"
```

#### Additional Environment Variables

```bash
# For archive node development
export OPENMINA_ARCHIVE_ADDRESS="http://localhost:3007"
export PG_USER="openmina"
Expand Down
5 changes: 5 additions & 0 deletions website/docs/developers/scripts/setup/format-and-lint.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# Set up SQLite database for heartbeats processor (required for make check/lint)
echo "Setting up SQLite database for heartbeats processor..."
sqlite3 /tmp/heartbeats.db < tools/heartbeats-processor/schema.sql
export DATABASE_URL="sqlite:///tmp/heartbeats.db"

# Format code (required before commits)
echo "Formatting code..."
make format
Expand Down
9 changes: 9 additions & 0 deletions website/docs/developers/scripts/setup/install-docker-macos.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Install Docker Desktop for Mac
# Note: This requires manual download and installation
echo "Docker Desktop for Mac needs to be installed manually."
echo "Please download it from: https://www.docker.com/products/docker-desktop/"
echo "Alternatively, you can use Homebrew Cask:"
echo "brew install --cask docker"

# Install Docker via Homebrew Cask (user will need to start Docker Desktop manually)
brew install --cask docker
5 changes: 5 additions & 0 deletions website/docs/developers/scripts/setup/install-nodejs-macos.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Install Node.js (version 18+ recommended) via Homebrew
brew install node@18

# Link Node.js if needed
brew link node@18
5 changes: 4 additions & 1 deletion website/docs/developers/scripts/setup/install-rust.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,7 @@ rustup install nightly
rustup component add rustfmt clippy --toolchain 1.84

# Add required components for nightly
rustup component add rustfmt clippy --toolchain nightly
rustup component add rustfmt clippy rust-src --toolchain nightly

# Install taplo (TOML formatter, required for `make format`)
cargo install taplo-cli --locked
Loading
Loading