From 2051ee1fcade29f91a259d663dee55b5020bedc1 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 11:36:43 +0100 Subject: [PATCH 01/17] Consolidate PDP documentation into single installation guide - Merged 6 separate PDP sub-pages into one comprehensive guide - Updated SUMMARY.md to reflect simplified structure - Maintains all content while improving navigation and readability --- SUMMARY.md | 7 +- storage-providers/pdp/enable-pdp.md | 211 ----- .../pdp/install-and-run-curio.md | 191 ---- .../pdp/install-and-run-lotus.md | 115 --- storage-providers/pdp/install-and-run-pdp.md | 871 ++++++++++++++++++ .../pdp/install-and-run-yugabytedb.md | 72 -- storage-providers/pdp/prerequisites.md | 69 -- storage-providers/pdp/use-pdp.md | 249 ----- 8 files changed, 872 insertions(+), 913 deletions(-) delete mode 100644 storage-providers/pdp/enable-pdp.md delete mode 100644 storage-providers/pdp/install-and-run-curio.md delete mode 100644 storage-providers/pdp/install-and-run-lotus.md create mode 100644 storage-providers/pdp/install-and-run-pdp.md delete mode 100644 storage-providers/pdp/install-and-run-yugabytedb.md delete mode 100644 storage-providers/pdp/prerequisites.md delete mode 100644 storage-providers/pdp/use-pdp.md diff --git a/SUMMARY.md b/SUMMARY.md index 801e56bc9..7a856dbbb 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -81,12 +81,7 @@ - [Sales](storage-providers/skills/sales.md) - [Industry](storage-providers/skills/industry.md) - [PDP](storage-providers/pdp/README.md) - - [Prerequisites](storage-providers/pdp/prerequisites.md) - - [Install & Run Lotus](storage-providers/pdp/install-and-run-lotus.md) - - [Install & Run YugabyteDB](storage-providers/pdp/install-and-run-yugabytedb.md) - - [Install & Run Curio](storage-providers/pdp/install-and-run-curio.md) - - [Enable PDP](storage-providers/pdp/enable-pdp.md) - - [Use PDP](storage-providers/pdp/use-pdp.md) + - [Install & Run PDP](storage-providers/pdp/install-and-run-pdp.md) ## Nodes diff --git a/storage-providers/pdp/enable-pdp.md b/storage-providers/pdp/enable-pdp.md deleted file mode 100644 index 5f3196586..000000000 --- a/storage-providers/pdp/enable-pdp.md +++ /dev/null @@ -1,211 +0,0 @@ ---- -description: >- - This section enables Proof of Data Possession (PDP) for a Storage Provider - node using Curio. These steps guide you through running a standalone PDP - service using Curio and pdptool. ---- - -# Enable PDP - -{% hint style="danger" %} -**DEPRECATED DEVELOPER TOOL**\ -\ -This documentation refers to the legacy `pdptool`, which is intended only for low-level developer testing.\ -It is not the recommended method for onboarding or interacting with PDP Storage Providers. - -For current usage, including working with live PDP SPs and submitting real deals, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk) and [Synapse dApp Tutorial](https://github.com/FIL-Builders/fs-upload-dapp). -{% endhint %} - -
PDP DocumentationCurio_placeholder.webp
Filecoin Slack - #fil-pdpFilecoin.svg.png
Filecoin Wallet - MetaMask SetupFilecoin.svg.png
- -### Attach Storage Locations - -With Curio running with the GUI layer: - -```sh -curio run --layers=gui -``` - -Run the following commands in your Curio CLI to attach storage paths: - -```sh -curio cli storage attach --init --seal /fast-storage/path -curio cli storage attach --init --store /long-term-storage/path -``` - -{% hint style="info" %} -Your fast-storage path should point to high-performance storage media such as NVMe or SSD -{% endhint %} - -*** - -### Add a PDP Configuration Layer - -Browse to the Configurations page of the Curio GUI. - -Create a new layer named pdp. Enable and set to `true` the following under Subsystems: - -{% hint style="info" %} -You may find it helpful to search for the setting names in your browser. -{% endhint %} - -* ✅ `EnableParkPiece` -* ✅ `EnablePDP` -* ✅ `EnableCommP` -* ✅ `EnableMoveStorage` - -In the HTTP section: - -* ✅ Enable: `true` -* 🌐 DomainName: `your domain (e.g., pdp.mydomain.com)` -* 📡 ListenAddress: `0.0.0.0:443` - -{% hint style="info" %} -You must point your domain’s A record to your server’s public IP address for Let’s Encrypt to issue a certificate. -{% endhint %} - -*** - -### Set Up PDP Service Keys - -Build the pdptool: - -```sh -cd curio/cmd/pdptool -go build . -``` - -Generate a service secret: - -```sh -./pdptool create-service-secret -``` - -```sh -# Example output: - ------BEGIN PUBLIC KEY----- -LxP9MzVmHdC7KwYBvNAo1jXuIRfGXqQyo2JzE4Uctn0a5eFZbs6Wlvq3dKYgphTD -XAqRsm38LPt2iVcGb9MruZJxEkBhO71wDdNyaFMoXpCJnUqRAezvKlfbIg== ------END PUBLIC KEY----- -``` - -Browse to the **PDP** page of the Curio GUI and in the **Services** section: - -* Select **Add PDP Service** -* Input a **Service Name** of your choice (e.g. `pdp-service`) -* Copy the previously generated public key into the **Public Key** field. -* Select **Add Service** - -*** - -### Import your Filecoin Wallet Private Key: - -{% hint style="info" %} -There are several ways to obtain private keys for Ethereum addresses. For this guide, we will use a new delegated Filecoin wallet address. -{% endhint %} - -Create a new delegated wallet: - -```sh -lotus wallet new delegated -``` - -```sh -# Example output: - -t410fuo4dghaeiqzokiqnxruzdr6e3cjktnxprrc56bi -``` - -{% hint style="info" %} -You can display your Lotus wallets at any time by running: -{% endhint %} - -```sh -lotus wallet list -``` - -Export & convert your new delegated wallet address private key: - -```sh -lotus wallet export | xxd -r -p | jq -r '.PrivateKey' | base64 -d | xxd -p -c 32 -``` - -```sh -# Example output: - -d4c2e3f9a716bb0e47fa91b2cf4a29870be3c5982fd6eafed71e8ac3f9c0b12 -``` - -Browse to the **PDP** page of the Curio GUI and in the **Owner Address** section: - -* Select **Import Key** -* Copy the previously generated private wallet key into the **Private Key (Hex)** field. -* Select **Import Key** - -{% hint style="success" %} -Your 0x wallet address - the delegated Ethereum address derived from your Filecoin Metamask private key - will be added to the **Owner Address** section of the Curio PDP page. -{% endhint %} - -Make sure to send a small amount of FIL to your 0x wallet - we recommend 5 FIL to ensure uninterrupted PDP operation during initial setup and testing. - -{% hint style="danger" %} -Important: Secure your private key material. Don’t expose or store it in plain text without protection. -{% endhint %} - -*** - -### Restart and Verify - -Restart Curio with both layers: - -```sh -curio run --layers=gui,pdp -``` - -{% hint style="warning" %} -If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run: -{% endhint %} - -```sh -sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/curio -``` - -Test the PDP service: - -```sh -./pdptool ping --service-url https://your-domain.com --service-name -``` - -{% hint style="success" %} -Use the service name specified in the **Service Name** field when you added your public **PDP Service** key - e.g. `pdp-service` -{% endhint %} - -Expected output: - -```sh -Ping successful: Service is reachable and JWT token is valid. -``` - -{% hint style="info" %} -Note: The first ping often fails. Try again after a short delay. -{% endhint %} - -*** - -### 🎉 You’re Ready! - -You’ve successfully launched a PDP-enabled Filecoin Storage Provider stack. Your system is now: - -* ✅ Syncing with the Filecoin network via Lotus -* ✅ Recording deal and sector metadata in YugabyteDB -* ✅ Operating Curio to manage sealing and coordination -* ✅ Submitting Proof of Data Possession to verify storage integrity - -*** - -### 🔜 Next Steps - -* 🚙 Take PDP for a test drive with the [Use PDP](use-pdp.md) guide -* 🧭 Monitor logs and metrics -* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) diff --git a/storage-providers/pdp/install-and-run-curio.md b/storage-providers/pdp/install-and-run-curio.md deleted file mode 100644 index 195984507..000000000 --- a/storage-providers/pdp/install-and-run-curio.md +++ /dev/null @@ -1,191 +0,0 @@ ---- -description: >- - Curio is the core PDP client that coordinates sealing, interacts with Lotus - and submits PDP proofs. ---- - -# Install & Run Curio - -
Curio DocumentationCurio_placeholder.webp
Filecoin Slack - #fil-curio-helpFilecoin.svg.png
- -### System Configuration - -Before you proceed with the installation, you should increase the UDP buffer size: - -```sh -sudo sysctl -w net.core.rmem_max=2097152 -sudo sysctl -w net.core.rmem_default=2097152 -``` - -To make this change persistent across reboots: - -```sh -echo 'net.core.rmem_max=2097152' | sudo tee -a /etc/sysctl.conf -echo 'net.core.rmem_default=2097152' | sudo tee -a /etc/sysctl.conf -``` - -### Build Curio - -Clone the repository and switch to the PDP branch: - -```sh -git clone https://github.com/filecoin-project/curio.git -cd curio -git checkout synapse -``` - -{% hint style="info" %} -Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below. -{% endhint %} - -```sh -# For Filecoin Mainnet: -make clean build - -# For Calibration Testnet: -make clean calibnet -``` - -{% hint style="success" %} -This step will take a few minutes to complete. -{% endhint %} - -### Install and Verify Curio - -Run the following to install the compiled binary: - -```sh -sudo make install -``` - -This will place curio in `/usr/local/bin` - -Verify the installation: - -```shell -curio --version -``` - -Expected output: - -```sh -# Example output for Mainnet: -curio version 1.24.4+mainnet+git_f954c0a_2025-04-06T15:46:32-04:00 - -# Example output for Calibration: -curio version 1.24.4+calibnet+git_f954c0a_2025-04-06T15:46:32-04:00 -``` - -*** - -### Guided Setup - -Curio provides a utility to help you set up a new miner interactively. Run the following command: - -```sh -curio guided-setup -``` - -#### 1️⃣ Select "Create a new miner" - -Use the arrow keys to navigate the guided setup menu and select "**Create a new miner**". - -#### 2️⃣ Enter Your YugabyteDB Connection Details - -If you used the default installation steps from this guide, the following values should work: - -* Host: `127.0.0.1` -* Port: `5433` -* Username: `yugabyte` -* Password: `yugabyte` -* Database: `yugabyte` - -You can verify these settings by running the following command from the Yugabyte directory: - -```sh -./bin/yugabyted status -``` - -After selecting "**Continue to connect and update schema**", Curio will automatically create the required tables and schema in the database. - -#### 3️⃣ Set Wallet Addresses - -For this step, use the two BLS wallets you created earlier with Lotus: - -* Use **wallet 1** for the **Owner Address** -* Use **wallet 2** for the **Worker Address** -* Use **wallet 1** again for the **Sender Address** - -These addresses must match the Lotus wallets created earlier. - -{% hint style="info" %} -You can display your Lotus wallets at any time by running: -{% endhint %} - -```sh -lotus wallet list -``` - -#### 4️⃣ Choose Sector Size - -Choose sector size: - -* `64 GiB` - -{% hint style="info" %} -💡 Selecting a sector size is required during the Curio guided setup, but **PDP itself doesn’t use sectors**. Proof set sizes in PDP are **arbitrary and fully flexible.** -{% endhint %} - -#### 5️⃣ Create Miner Actor - -Review the information to ensure all inputs are correct. Then select "**Continue to verify the addresses and create a new miner actor**" to proceed. - -{% hint style="info" %} -This step may take a few minutes to complete as Curio pushes the message and waits for it to land on-chain. -{% endhint %} - -Once the actor is created, Curio will: - -* `Register your miner ID` - -{% hint style="info" %} -If the guided setup fails after creating the miner actor, run the following command to complete the installation: -{% endhint %} - -```sh -curio config new-cluster -``` - -#### 6️⃣ Telemetry (Optional) - -You’ll be asked whether to share anonymised or signed telemetry with the Curio team to help improve the software. - -Select your preference and continue. - -#### 7️⃣ Save Database Configuration - -At the final step of the guided setup, you’ll be prompted to choose where to save your database configuration file. - -Use the arrow keys to select a location. A common default is: - -```sh -/home/your-username/curio.env -``` - -Once selected, setup will complete, and the miner configuration will be stored. - -#### 8️⃣ Launch the Curio Web GUI - -To explore the Curio interface visually, start the GUI layer: - -```sh -curio run --layers=gui -``` - -Then, open your browser and go to: - -```sh -http://127.0.0.1:4701 -``` - -This will launch the Curio web GUI locally. diff --git a/storage-providers/pdp/install-and-run-lotus.md b/storage-providers/pdp/install-and-run-lotus.md deleted file mode 100644 index 17dda25b4..000000000 --- a/storage-providers/pdp/install-and-run-lotus.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -description: >- - Lotus is your gateway to the Filecoin network. It syncs the chain, manages - wallets, and is required for Curio to interact with your node. ---- - -# Install & Run Lotus - -
Lotus Documentationlotus-logo-big.png
Filecoin Slack - #fil-lotus-helpFilecoin.svg.png
- -### Build Lotus Daemon - -Clone and check out Lotus: - -```sh -git clone https://github.com/filecoin-project/lotus.git -cd lotus -git checkout $(curl -s https://api.github.com/repos/filecoin-project/lotus/releases/latest | jq -r .tag_name) -``` - -#### **Build and Install for Mainnet** - -```sh -make clean lotus -sudo make install-daemon -lotus --version -``` - -#### **Build and Install for Calibration** - -```sh -make clean && make GOFLAGS="-tags=calibnet" lotus -sudo make install-daemon -lotus --version -``` - -{% hint style="success" %} -You should see something like: `lotus version 1.33.1+mainnet+git.ff88d8269` -{% endhint %} - -*** - -### Import a Snapshot and Start the Daemon - -Download the Snapshot - -#### Mainnet: - -```sh -aria2c -x5 -o snapshot.car.zst https://forest-archive.chainsafe.dev/latest/mainnet/ -``` - -#### Calibration: - -```sh -aria2c -x5 -o snapshot.car.zst https://forest-archive.chainsafe.dev/latest/calibnet/ -``` - -#### Import and Start the Daemon - -```sh -lotus daemon --import-snapshot snapshot.car.zst --remove-existing-chain --halt-after-import -nohup lotus daemon > ~/lotus.log 2>&1 & -``` - -{% hint style="warning" %} -If you encounter errors related to `EnableEthRPC` or `EnableIndexer`, run the command below and restart Lotus -{% endhint %} - -```sh -sed -i 's/EnableEthRPC = .*/EnableEthRPC = true/; s/EnableIndexer = .*/EnableIndexer = true/' ~/.lotus/config.toml -``` - -#### **Monitor Sync Progress** - -```sh -lotus sync wait -``` - -To monitor continuously: - -```sh -lotus sync wait --watch -``` - -#### **Monitor Logs** - -```sh -tail -f ~/lotus.log -``` - -*** - -### Create Wallets - -You’ll need to create **two BLS wallets**: - -* One for **owner**: used to fund sector pledges and submit proofs -* One for **worker**: used to publish and manage storage deals - -```sh -lotus wallet new bls # Create owner wallet -lotus wallet new bls # Create worker wallet -lotus wallet list # List all created wallets -``` - -Make sure to send a small amount of FIL (Mainnet) or tFIL (Calibration) to each wallet - we recommend 1 FIL/tFIL per wallet to ensure the creation of your Storage Provider in Curio. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). - -{% hint style="info" %} -Both wallets will be used during Curio initialisation. -{% endhint %} - -{% hint style="danger" %} -[Back up](https://lotus.filecoin.io/lotus/manage/manage-fil/#exporting-and-importing-addresses) your wallet keys securely before continuing. Losing them will result in permanent loss of access to funds. -{% endhint %} diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md new file mode 100644 index 000000000..cc625e1b9 --- /dev/null +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -0,0 +1,871 @@ +--- +description: >- + This guide walks you through setting up a PDP-enabled Filecoin Storage + Provider using Lotus, YugabyteDB, and Curio +--- + +# Install & Run PDP + +{% hint style="danger" %} +**DEPRECATED DEVELOPER TOOL** + +This documentation refers to the legacy `pdptool`, which is intended only for low-level developer testing. +It is not the recommended method for onboarding or interacting with PDP Storage Providers. + +For current usage, including working with live PDP SPs and submitting real deals, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk) and [Synapse dApp Tutorial](https://github.com/FIL-Builders/fs-upload-dapp). +{% endhint %} + +
Lotus Documentationlotus-logo-big.png
Yugabyte Documentationyugabyte.svg
Curio DocumentationCurio_placeholder.webp
PDP DocumentationCurio_placeholder.webp
Filecoin Slack - #fil-pdpFilecoin.svg.png
Filecoin Slack - #fil-curio-helpFilecoin.svg.png
Filecoin Slack - #fil-lotus-helpFilecoin.svg.png
Filecoin Wallet - MetaMask SetupFilecoin.svg.png
Yugabyte Slackyugabyte.svg
+ +## Prerequisites + +{% hint style="info" %} +This guide is written specifically for Ubuntu 22.04. If you are using a different Linux distribution, refer to the relevant documentation for package installation and compatibility. +{% endhint %} + +Before starting, make sure you have a user with **sudo privileges**. This section prepares your system for the PDP stack. + +### System Package Installation + +```sh +sudo apt update && sudo apt upgrade -y && sudo apt install -y \ +mesa-opencl-icd ocl-icd-opencl-dev gcc git jq pkg-config curl clang \ +build-essential hwloc libhwloc-dev libarchive-dev wget ntp python-is-python3 aria2 +``` + +*** + +### Install Go + +```sh +sudo rm -rf /usr/local/go +wget https://go.dev/dl/go1.23.7.linux-amd64.tar.gz +sudo tar -C /usr/local -xzf go1.23.7.linux-amd64.tar.gz +echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc +source ~/.bashrc +go version +``` + +{% hint style="success" %} +You should see something like: `go version go1.23.7 linux/amd64` +{% endhint %} + +*** + +### Install Rust + +```sh +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh +``` + +{% hint style="info" %} +When prompted, choose the option 1) Proceed with standard installation (default — just press Enter). +{% endhint %} + +```sh +source $HOME/.cargo/env +rustc --version +``` + +{% hint style="success" %} +You should see something like: `rustc 1.86.0 (05f9846f8 2025-03-31)` +{% endhint %} + +*** + +### Add Go and Rust to Secure Sudo Path + +```sh +sudo tee /etc/sudoers.d/dev-paths < ~/lotus.log 2>&1 & +``` + +{% hint style="warning" %} +If you encounter errors related to `EnableEthRPC` or `EnableIndexer`, run the command below and restart Lotus +{% endhint %} + +```sh +sed -i 's/EnableEthRPC = .*/EnableEthRPC = true/; s/EnableIndexer = .*/EnableIndexer = true/' ~/.lotus/config.toml +``` + +#### **Monitor Sync Progress** + +```sh +lotus sync wait +``` + +To monitor continuously: + +```sh +lotus sync wait --watch +``` + +#### **Monitor Logs** + +```sh +tail -f ~/lotus.log +``` + +*** + +### Create Wallets + +You'll need to create **two BLS wallets**: + +* One for **owner**: used to fund sector pledges and submit proofs +* One for **worker**: used to publish and manage storage deals + +```sh +lotus wallet new bls # Create owner wallet +lotus wallet new bls # Create worker wallet +lotus wallet list # List all created wallets +``` + +Make sure to send a small amount of FIL (Mainnet) or tFIL (Calibration) to each wallet - we recommend 1 FIL/tFIL per wallet to ensure the creation of your Storage Provider in Curio. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). + +{% hint style="info" %} +Both wallets will be used during Curio initialisation. +{% endhint %} + +{% hint style="danger" %} +[Back up](https://lotus.filecoin.io/lotus/manage/manage-fil/#exporting-and-importing-addresses) your wallet keys securely before continuing. Losing them will result in permanent loss of access to funds. +{% endhint %} + +## Install & Run YugabyteDB + +### Set ulimit configuration + +{% hint style="info" %} + Before starting Yugabyte, you must increase the default `ulimit` values to ensure system limits do not interfere with the database. +{% endhint %} + +To do this: + +#### **Persist new limits across reboots** + +Add these lines to `/etc/security/limits.conf`: + +```sh +echo "$(whoami) soft nofile 1048576" | sudo tee -a /etc/security/limits.conf +echo "$(whoami) hard nofile 1048576" | sudo tee -a /etc/security/limits.conf +``` + +This ensures the increased limits are automatically applied to future sessions. + +#### **Apply limit immediately (for current shell only)** + +```sh +ulimit -n 1048576 +# Verify limit change: +ulimit -n +``` + +{% hint style="success" %} +This should output `1048576`. +{% endhint %} + +### Install Yugabyte + +```sh +wget https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz +tar xvfz yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz +cd yugabyte-2.25.1.0 +./bin/post_install.sh +``` + +### Start the DB + +```sh +./bin/yugabyted start \ + --advertise_address 127.0.0.1 \ + --master_flags rpc_bind_addresses=127.0.0.1 \ + --tserver_flags rpc_bind_addresses=127.0.0.1 +``` + +{% hint style="danger" %} + If you encounter locale-related errors when starting Yugabyte for the first time, run: +{% endhint %} + +```sh +sudo locale-gen en_US.UTF-8 +``` + +{% hint style="success" %} +Visit `127.0.0.1:15433` to confirm successful installation. This is the YugabyteDB web UI — it should display the dashboard if the service is running correctly and all nodes are healthy. +{% endhint %} + +{% hint style="info" %} +You can also check your Yugabyte cluster details directly in the CLI with: +{% endhint %} + +```sh +./bin/yugabyted status +``` + +## Install & Run Curio + +Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. + +### System Configuration + +Before you proceed with the installation, you should increase the UDP buffer size: + +```sh +sudo sysctl -w net.core.rmem_max=2097152 +sudo sysctl -w net.core.rmem_default=2097152 +``` + +To make this change persistent across reboots: + +```sh +echo 'net.core.rmem_max=2097152' | sudo tee -a /etc/sysctl.conf +echo 'net.core.rmem_default=2097152' | sudo tee -a /etc/sysctl.conf +``` + +### Build Curio + +Clone the repository and switch to the PDP branch: + +```sh +git clone https://github.com/filecoin-project/curio.git +cd curio +git checkout synapse +``` + +{% hint style="info" %} +Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below. +{% endhint %} + +```sh +# For Filecoin Mainnet: +make clean build + +# For Calibration Testnet: +make clean calibnet +``` + +{% hint style="success" %} +This step will take a few minutes to complete. +{% endhint %} + +### Install and Verify Curio + +Run the following to install the compiled binary: + +```sh +sudo make install +``` + +This will place curio in `/usr/local/bin` + +Verify the installation: + +```shell +curio --version +``` + +Expected output: + +```sh +# Example output for Mainnet: +curio version 1.24.4+mainnet+git_f954c0a_2025-04-06T15:46:32-04:00 + +# Example output for Calibration: +curio version 1.24.4+calibnet+git_f954c0a_2025-04-06T15:46:32-04:00 +``` + +*** + +### Guided Setup + +Curio provides a utility to help you set up a new miner interactively. Run the following command: + +```sh +curio guided-setup +``` + +#### 1️⃣ Select "Create a new miner" + +Use the arrow keys to navigate the guided setup menu and select "**Create a new miner**". + +#### 2️⃣ Enter Your YugabyteDB Connection Details + +If you used the default installation steps from this guide, the following values should work: + +* Host: `127.0.0.1` +* Port: `5433` +* Username: `yugabyte` +* Password: `yugabyte` +* Database: `yugabyte` + +You can verify these settings by running the following command from the Yugabyte directory: + +```sh +./bin/yugabyted status +``` + +After selecting "**Continue to connect and update schema**", Curio will automatically create the required tables and schema in the database. + +#### 3️⃣ Set Wallet Addresses + +For this step, use the two BLS wallets you created earlier with Lotus: + +* Use **wallet 1** for the **Owner Address** +* Use **wallet 2** for the **Worker Address** +* Use **wallet 1** again for the **Sender Address** + +These addresses must match the Lotus wallets created earlier. + +{% hint style="info" %} +You can display your Lotus wallets at any time by running: +{% endhint %} + +```sh +lotus wallet list +``` + +#### 4️⃣ Choose Sector Size + +Choose sector size: + +* `64 GiB` + +{% hint style="info" %} +💡 Selecting a sector size is required during the Curio guided setup, but **PDP itself doesn't use sectors**. Proof set sizes in PDP are **arbitrary and fully flexible.** +{% endhint %} + +#### 5️⃣ Create Miner Actor + +Review the information to ensure all inputs are correct. Then select "**Continue to verify the addresses and create a new miner actor**" to proceed. + +{% hint style="info" %} +This step may take a few minutes to complete as Curio pushes the message and waits for it to land on-chain. +{% endhint %} + +Once the actor is created, Curio will: + +* `Register your miner ID` + +{% hint style="info" %} +If the guided setup fails after creating the miner actor, run the following command to complete the installation: +{% endhint %} + +```sh +curio config new-cluster +``` + +#### 6️⃣ Telemetry (Optional) + +You'll be asked whether to share anonymised or signed telemetry with the Curio team to help improve the software. + +Select your preference and continue. + +#### 7️⃣ Save Database Configuration + +At the final step of the guided setup, you'll be prompted to choose where to save your database configuration file. + +Use the arrow keys to select a location. A common default is: + +```sh +/home/your-username/curio.env +``` + +Once selected, setup will complete, and the miner configuration will be stored. + +#### 8️⃣ Launch the Curio Web GUI + +To explore the Curio interface visually, start the GUI layer: + +```sh +curio run --layers=gui +``` + +Then, open your browser and go to: + +```sh +http://127.0.0.1:4701 +``` + +This will launch the Curio web GUI locally. + +## Enable PDP + +This section enables Proof of Data Possession (PDP) for a Storage Provider node using Curio. These steps guide you through running a standalone PDP service using Curio and pdptool. + +### Attach Storage Locations + +With Curio running with the GUI layer: + +```sh +curio run --layers=gui +``` + +Run the following commands in your Curio CLI to attach storage paths: + +```sh +curio cli storage attach --init --seal /fast-storage/path +curio cli storage attach --init --store /long-term-storage/path +``` + +{% hint style="info" %} +Your fast-storage path should point to high-performance storage media such as NVMe or SSD +{% endhint %} + +*** + +### Add a PDP Configuration Layer + +Browse to the Configurations page of the Curio GUI. + +Create a new layer named pdp. Enable and set to `true` the following under Subsystems: + +{% hint style="info" %} +You may find it helpful to search for the setting names in your browser. +{% endhint %} + +* ✅ `EnableParkPiece` +* ✅ `EnablePDP` +* ✅ `EnableCommP` +* ✅ `EnableMoveStorage` + +In the HTTP section: + +* ✅ Enable: `true` +* 🌐 DomainName: `your domain (e.g., pdp.mydomain.com)` +* 📡 ListenAddress: `0.0.0.0:443` + +{% hint style="info" %} +You must point your domain's A record to your server's public IP address for Let's Encrypt to issue a certificate. +{% endhint %} + +*** + +### Set Up PDP Service Keys + +Build the pdptool: + +```sh +cd curio/cmd/pdptool +go build . +``` + +Generate a service secret: + +```sh +./pdptool create-service-secret +``` + +```sh +# Example output: + +-----BEGIN PUBLIC KEY----- +LxP9MzVmHdC7KwYBvNAo1jXuIRfGXqQyo2JzE4Uctn0a5eFZbs6Wlvq3dKYgphTD +XAqRsm38LPt2iVcGb9MruZJxEkBhO71wDdNyaFMoXpCJnUqRAezvKlfbIg== +-----END PUBLIC KEY----- +``` + +Browse to the **PDP** page of the Curio GUI and in the **Services** section: + +* Select **Add PDP Service** +* Input a **Service Name** of your choice (e.g. `pdp-service`) +* Copy the previously generated public key into the **Public Key** field. +* Select **Add Service** + +*** + +### Import your Filecoin Wallet Private Key: + +{% hint style="info" %} +There are several ways to obtain private keys for Ethereum addresses. For this guide, we will use a new delegated Filecoin wallet address. +{% endhint %} + +Create a new delegated wallet: + +```sh +lotus wallet new delegated +``` + +```sh +# Example output: + +t410fuo4dghaeiqzokiqnxruzdr6e3cjktnxprrc56bi +``` + +{% hint style="info" %} +You can display your Lotus wallets at any time by running: +{% endhint %} + +```sh +lotus wallet list +``` + +Export & convert your new delegated wallet address private key: + +```sh +lotus wallet export | xxd -r -p | jq -r '.PrivateKey' | base64 -d | xxd -p -c 32 +``` + +```sh +# Example output: + +d4c2e3f9a716bb0e47fa91b2cf4a29870be3c5982fd6eafed71e8ac3f9c0b12 +``` + +Browse to the **PDP** page of the Curio GUI and in the **Owner Address** section: + +* Select **Import Key** +* Copy the previously generated private wallet key into the **Private Key (Hex)** field. +* Select **Import Key** + +{% hint style="success" %} +Your 0x wallet address - the delegated Ethereum address derived from your Filecoin Metamask private key - will be added to the **Owner Address** section of the Curio PDP page. +{% endhint %} + +Make sure to send a small amount of FIL to your 0x wallet - we recommend 5 FIL to ensure uninterrupted PDP operation during initial setup and testing. + +{% hint style="danger" %} +Important: Secure your private key material. Don't expose or store it in plain text without protection. +{% endhint %} + +*** + +### Restart and Verify + +Restart Curio with both layers: + +```sh +curio run --layers=gui,pdp +``` + +{% hint style="warning" %} +If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run: +{% endhint %} + +```sh +sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/curio +``` + +Test the PDP service: + +```sh +./pdptool ping --service-url https://your-domain.com --service-name +``` + +{% hint style="success" %} +Use the service name specified in the **Service Name** field when you added your public **PDP Service** key - e.g. `pdp-service` +{% endhint %} + +Expected output: + +```sh +Ping successful: Service is reachable and JWT token is valid. +``` + +{% hint style="info" %} +Note: The first ping often fails. Try again after a short delay. +{% endhint %} + +## Use PDP + +PDP ensures that your data is verifiably stored by a Filecoin Storage Provider using cryptographic proofs without needing to retrieve the file itself. + +### Prerequisites + +Before beginning, ensure: + +* You have access to a terminal with internet connectivity +* Your system has pdptool installed (bundled with Curio) + +### If pdptool is not installed: + +* **Option 1**: Clone Curio and build pdptool: + +```sh +git clone +https://github.com/filecoin-project/curio.git + +cd curio +cd cmd/pdptool +go build . +``` + +* **Option 2**: Install the [Docker version ](https://github.com/LesnyRumcajs/pdp)of pdptool - Provided by our friends at [ChainSafe](https://chainsafe.io/) + +*** + +### Authenticate Your Client (JWT Token) + +You first need to authenticate your pdptool with a PDP-enabled Storage Provider + +Generate a service secret: + +```sh +./pdptool create-service-secret +``` + +```sh +# Example output: + +-----BEGIN PUBLIC KEY----- +LxP9MzVmHdC7KwYBvNAo1jXuIRfGXqQyo2JzE4Uctn0a5eFZbs6Wlvq3dKYgphTD +XAqRsm38LPt2iVcGb9MruZJxEkBhO71wDdNyaFMoXpCJnUqRAezvKlfbIg== +-----END PUBLIC KEY----- +``` + +{% hint style="success" %} +Reach out in the [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) channel in Filecoin Slack to register your public key with a PDP-enabled Storage Provider +{% endhint %} + +*** + +### Connect to a PDP Service + +Start by pinging the PDP service to confirm availability: + +```sh +./pdptool ping --service-url https://yablu.net --service-name pdp-service +``` + +{% hint style="success" %} +You should see something like: +{% endhint %} + +```sh +Ping successful: Service is reachable and JWT token is valid. +``` + +*** + +### Create a Proof Set + +Start by creating an empty proof set. This step must happen **before** uploading files: + +```sh +./pdptool create-proof-set \ + --service-url https://yablu.net \ + --service-name pdp-service \ + --recordkeeper 0x6170dE2b09b404776197485F3dc6c968Ef948505 +``` + +```sh +# Example output: + +Proof set creation initiated successfully. +Location: /pdp/proof-sets/created/0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 +Response: +``` + +Use the `0x` transaction hash from the previous output to monitor proof set creation status: + +```sh +./pdptool get-proof-set-create-status \ + --service-url https://yablu.net \ + --service-name pdp-service \ + --tx-hash 0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 +``` + +{% hint style="success" %} +You should see something like: +{% endhint %} + +```sh +Proof Set Creation Status: +Transaction Hash: 0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 +Transaction Status: confirmed +Transaction Successful: true +Proofset Created: true +ProofSet ID: 43 +``` + +{% hint style="info" %} +The proof set creation process can take a few seconds to complete +{% endhint %} + +*** + +### Upload Files to the Storage Provider + +Once your proof set is ready, you can begin uploading files: + +```sh +./pdptool upload-file --service-url https://yablu.net --service-name pdp-service /path/to/file.ext +``` + +{% hint style="success" %} +Example output: +{% endhint %} + +```sh +0: pieceSize: 65536 +baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli:baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli +``` + +*** + +### 🌳 Add File Roots to Proof Set + +After uploading each file, extract its CID and add it to your proof set: + +```sh +./pdptool add-roots \ + --service-url https://yablu.net \ + --service-name pdp-service \ + --proof-set-id \ + --root ++... +``` + +Example using the information returned in the previous steps: + +```sh +./pdptool add-roots \ + --service-url https://yablu.net \ + --service-name pdp-service \ + --proof-set-id 43 \ + --root baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli:baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli +``` + +{% hint style="info" %} +In the above example, `--proof-set-id` came from the Create Proof Set step, and `--root` from the Upload Files to the Storage Provider step. +{% endhint %} + +{% hint style="success" %} +Example output: +{% endhint %} + +```sh +Roots added to proof set ID 43 successfully. +Response: +``` + +*** + +### View a Piece or Proof Set + +You can retrieve a proof set or inspect a file root directly: + +```sh +./pdptool get-proof-set \ + --service-url https://yablu.net \ + --service-name pdp-service 43 +``` + +{% hint style="success" %} +Example output: +{% endhint %} + +```sh +Proof Set ID: 43 +Next Challenge Epoch: 2577608 +Roots: + - Root ID: 0 + Root CID: baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli + Subroot CID: baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli + Subroot Offset: 0 +``` + +*** + +### Retrieve From a Proof Set + +Download a file using an ordered chunks list: + +```sh +./pdptool download-file \ + --service-url https://yablu.net \ + --chunk-file chunks.list \ + --output-file file.ext +``` + +{% hint style="info" %} +💡In the above example, `–chunk-file` and `–output-file` flags were defined in the Upload Files to the Storage Provider step +{% endhint %} + +*** + +## 🎉 You're Ready! + +You've successfully launched a PDP-enabled Filecoin Storage Provider stack. Your system is now: + +* ✅ Syncing with the Filecoin network via Lotus +* ✅ Recording deal and sector metadata in YugabyteDB +* ✅ Operating Curio to manage sealing and coordination +* ✅ Submitting Proof of Data Possession to verify storage integrity + +You've also: + +✅ Connected to a PDP-enabled storage provider +✅ Created a proof set +✅ Uploaded files and added file roots +✅ Verified availability and proof status + +*** + +## 🔜 Next Steps + +* 🧭 Monitor logs and metrics +* 🧭 Track your proof sets in the PDP Explorer: + * [Calibration PDP Explorer](https://calibration.pdp-explorer.eng.filoz.org) + * [Mainnet PDP Explorer](https://pdp-explorer.eng.filoz.org) +* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) + +💬 Questions? Join the conversation on Filecoin Slack: [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) \ No newline at end of file diff --git a/storage-providers/pdp/install-and-run-yugabytedb.md b/storage-providers/pdp/install-and-run-yugabytedb.md deleted file mode 100644 index 857537eb5..000000000 --- a/storage-providers/pdp/install-and-run-yugabytedb.md +++ /dev/null @@ -1,72 +0,0 @@ -# Install & Run YugabyteDB - -
Yugabyte Documentationyugabyte.svg
Filecoin Slaxk - #fil-curio-helpFilecoin.svg.png
Yugabyte Slackyugabyte.svg
- -### Set ulimit configuration - -{% hint style="info" %} - Before starting Yugabyte, you must increase the default `ulimit` values to ensure system limits do not interfere with the database. -{% endhint %} - -To do this: - -#### **Persist new limits across reboots** - -Add these lines to `/etc/security/limits.conf`: - -```sh -echo "$(whoami) soft nofile 1048576" | sudo tee -a /etc/security/limits.conf -echo "$(whoami) hard nofile 1048576" | sudo tee -a /etc/security/limits.conf -``` - -This ensures the increased limits are automatically applied to future sessions. - -#### **Apply limit immediately (for current shell only)** - -```sh -ulimit -n 1048576 -# Verify limit change: -ulimit -n -``` - -{% hint style="success" %} -This should output `1048576`. -{% endhint %} - -### Install Yugabyte - -```sh -wget https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz -tar xvfz yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz -cd yugabyte-2.25.1.0 -./bin/post_install.sh -``` - -### Start the DB - -```sh -./bin/yugabyted start \ - --advertise_address 127.0.0.1 \ - --master_flags rpc_bind_addresses=127.0.0.1 \ - --tserver_flags rpc_bind_addresses=127.0.0.1 -``` - -{% hint style="danger" %} - If you encounter locale-related errors when starting Yugabyte for the first time, run: -{% endhint %} - -```sh -sudo locale-gen en_US.UTF-8 -``` - -{% hint style="success" %} -Visit `127.0.0.1:15433` to confirm successful installation. This is the YugabyteDB web UI — it should display the dashboard if the service is running correctly and all nodes are healthy. -{% endhint %} - -{% hint style="info" %} -You can also check your Yugabyte cluster details directly in the CLI with: -{% endhint %} - -```sh -./bin/yugabyted status -``` diff --git a/storage-providers/pdp/prerequisites.md b/storage-providers/pdp/prerequisites.md deleted file mode 100644 index acd63a0be..000000000 --- a/storage-providers/pdp/prerequisites.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -description: >- - This guide walks you through setting up a PDP-enabled Filecoin Storage - Provider using Lotus, YugabyteDB, and Curio ---- - -# Prerequisites - -{% hint style="info" %} -This guide is written specifically for Ubuntu 22.04. If you are using a different Linux distribution, refer to the relevant documentation for package installation and compatibility. -{% endhint %} - -Before starting, make sure you have a user with **sudo privileges**. This section prepares your system for the PDP stack. - -### System Package Installation - -```sh -sudo apt update && sudo apt upgrade -y && sudo apt install -y \ -mesa-opencl-icd ocl-icd-opencl-dev gcc git jq pkg-config curl clang \ -build-essential hwloc libhwloc-dev libarchive-dev wget ntp python-is-python3 aria2 -``` - -*** - -### Install Go - -```sh -sudo rm -rf /usr/local/go -wget https://go.dev/dl/go1.23.7.linux-amd64.tar.gz -sudo tar -C /usr/local -xzf go1.23.7.linux-amd64.tar.gz -echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc -source ~/.bashrc -go version -``` - -{% hint style="success" %} -You should see something like: `go version go1.23.7 linux/amd64` -{% endhint %} - -*** - -### Install Rust - -```sh -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -``` - -{% hint style="info" %} -When prompted, choose the option 1) Proceed with standard installation (default — just press Enter). -{% endhint %} - -```sh -source $HOME/.cargo/env -rustc --version -``` - -{% hint style="success" %} -You should see something like: `rustc 1.86.0 (05f9846f8 2025-03-31)` -{% endhint %} - -*** - -### Add Go and Rust to Secure Sudo Path - -```sh -sudo tee /etc/sudoers.d/dev-paths <- - This guide walks you through using the PDP client tool (pdptool) to interact - with a Filecoin Storage Provider running the Proof of Data Possession (PDP) - service. ---- - -# Use PDP - -{% hint style="danger" %} -DEPRECATED DEVELOPER TOOL - -This documentation refers to the legacy `pdptool`, which is intended only for low-level developer testing. - -It is not the recommended method for onboarding or interacting with PDP Storage Providers. - -For current usage, including working with live PDP SPs and submitting real deals, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk) and [Synapse dApp Tutorial](https://github.com/FIL-Builders/fs-upload-dapp). -{% endhint %} - -PDP ensures that your data is verifiably stored by a Filecoin Storage Provider using cryptographic proofs without needing to retrieve the file itself. - -### Prerequisites - -Before beginning, ensure: - -* You have access to a terminal with internet connectivity -* Your system has pdptool installed (bundled with Curio) - -### If pdptool is not installed: - -* **Option 1**: Clone Curio and build pdptool: - -```sh -git clone -https://github.com/filecoin-project/curio.git - -cd curio -cd cmd/pdptool -go build . -``` - -* **Option 2**: Install the [Docker version ](https://github.com/LesnyRumcajs/pdp)of pdptool - Provided by our friends at [ChainSafe](https://chainsafe.io/) - -*** - -### Authenticate Your Client (JWT Token) - -You first need to authenticate your pdptool with a PDP-enabled Storage Provider - -Generate a service secret: - -```sh -./pdptool create-service-secret -``` - -```sh -# Example output: - ------BEGIN PUBLIC KEY----- -LxP9MzVmHdC7KwYBvNAo1jXuIRfGXqQyo2JzE4Uctn0a5eFZbs6Wlvq3dKYgphTD -XAqRsm38LPt2iVcGb9MruZJxEkBhO71wDdNyaFMoXpCJnUqRAezvKlfbIg== ------END PUBLIC KEY----- -``` - -{% hint style="success" %} -Reach out in the [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) channel in Filecoin Slack to register your public key with a PDP-enabled Storage Provider -{% endhint %} - -*** - -### Connect to a PDP Service - -Start by pinging the PDP service to confirm availability: - -```sh -./pdptool ping --service-url https://yablu.net --service-name pdp-service -``` - -{% hint style="success" %} -You should see something like: -{% endhint %} - -```sh -Ping successful: Service is reachable and JWT token is valid. -``` - -*** - -### Create a Proof Set - -Start by creating an empty proof set. This step must happen **before** uploading files: - -```sh -./pdptool create-proof-set \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --recordkeeper 0x6170dE2b09b404776197485F3dc6c968Ef948505 -``` - -```sh -# Example output: - -Proof set creation initiated successfully. -Location: /pdp/proof-sets/created/0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 -Response: -``` - -Use the `0x` transaction hash from the previous output to monitor proof set creation status: - -```sh -./pdptool get-proof-set-create-status \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --tx-hash 0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 -``` - -{% hint style="success" %} -You should see something like: -{% endhint %} - -```sh -Proof Set Creation Status: -Transaction Hash: 0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 -Transaction Status: confirmed -Transaction Successful: true -Proofset Created: true -ProofSet ID: 43 -``` - -{% hint style="info" %} -The proof set creation process can take a few seconds to complete -{% endhint %} - -*** - -### Upload Files to the Storage Provider - -Once your proof set is ready, you can begin uploading files: - -```sh -./pdptool upload-file --service-url https://yablu.net --service-name pdp-service /path/to/file.ext -``` - -{% hint style="success" %} -Example output: -{% endhint %} - -```sh -0: pieceSize: 65536 -baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli:baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli -``` - -*** - -### 🌳 Add File Roots to Proof Set - -After uploading each file, extract its CID and add it to your proof set: - -```sh -./pdptool add-roots \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --proof-set-id \ - --root ++... -``` - -Example using the information returned in the previous steps: - -```sh -./pdptool add-roots \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --proof-set-id 43 \ - --root baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli:baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli -``` - -{% hint style="info" %} -In the above example, `--proof-set-id` came from the [Create Proof Set](use-pdp.md#create-a-proof-set) step, and `--root` from the [Upload Files to the Storage Provider](use-pdp.md#upload-files-to-the-storage-provider) step. -{% endhint %} - -{% hint style="success" %} -Example output: -{% endhint %} - -```sh -Roots added to proof set ID 43 successfully. -Response: -``` - -*** - -### View a Piece or Proof Set - -You can retrieve a proof set or inspect a file root directly: - -```sh -./pdptool get-proof-set \ - --service-url https://yablu.net \ - --service-name pdp-service 43 -``` - -{% hint style="success" %} -Example output: -{% endhint %} - -```sh -Proof Set ID: 43 -Next Challenge Epoch: 2577608 -Roots: - - Root ID: 0 - Root CID: baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli - Subroot CID: baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli - Subroot Offset: 0 -``` - -*** - -### Retrieve From a Proof Set - -Download a file using an ordered chunks list: - -```sh -./pdptool download-file \ - --service-url https://yablu.net \ - --chunk-file chunks.list \ - --output-file file.ext -``` - -{% hint style="info" %} -💡In the above example, `–chunk-file` and `–output-file` flags were defined in the [Upload Files to the Storage Provider step](use-pdp.md#upload-files-to-the-storage-provider) -{% endhint %} - -*** - -### You’re Done! - -You’ve now: - -✅ Connected to a PDP-enabled storage provider\ -✅ Created a proof set\ -✅ Uploaded files and added file roots\ -✅ Verified availability and proof status - -🧭 Next: Track your proof sets in the PDP Explorer - -* [Calibration PDP Explorer](https://calibration.pdp-explorer.eng.filoz.org) -* [Mainnet PDP Explorer](https://pdp-explorer.eng.filoz.org) - -💬 Questions? Join the conversation on Filecoin Slack: [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) From c085abfac742a92ad46b5bdf774b84187a99ac09 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 10:44:04 +0000 Subject: [PATCH 02/17] GITBOOK-2: No subject From 6f0795cb301c176e4eaacec4e573b5e97f37ec30 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:07:53 +0100 Subject: [PATCH 03/17] Update PDP documentation with FWSS (Filecoin Warm Storage Service) content - Updated to focus on FWSS instead of legacy PDP implementation - Changed from mainnet to Calibration testnet configuration - Updated Curio to use 'rename' branch instead of 'synapse' - Added emoji headers for better visual organization - Simplified wallet setup with delegated address instructions - Removed detailed pdptool usage section - Updated Slack channels to include #spx-pdp - Improved formatting and readability throughout --- storage-providers/pdp/install-and-run-pdp.md | 469 +++++-------------- 1 file changed, 110 insertions(+), 359 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index cc625e1b9..1e3f0583d 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -17,25 +17,27 @@ For current usage, including working with live PDP SPs and submitting real deals
Lotus Documentationlotus-logo-big.png
Yugabyte Documentationyugabyte.svg
Curio DocumentationCurio_placeholder.webp
PDP DocumentationCurio_placeholder.webp
Filecoin Slack - #fil-pdpFilecoin.svg.png
Filecoin Slack - #fil-curio-helpFilecoin.svg.png
Filecoin Slack - #fil-lotus-helpFilecoin.svg.png
Filecoin Wallet - MetaMask SetupFilecoin.svg.png
Yugabyte Slackyugabyte.svg
-## Prerequisites +## 🚀 Prerequisites -{% hint style="info" %} -This guide is written specifically for Ubuntu 22.04. If you are using a different Linux distribution, refer to the relevant documentation for package installation and compatibility. +{% hint style="warning" %} +**Note:** This guide is written specifically for **Ubuntu 22.04**. If you are using a different Linux distribution, refer to the relevant documentation for package installation and compatibility. {% endhint %} Before starting, make sure you have a user with **sudo privileges**. This section prepares your system for the PDP stack. -### System Package Installation +*** + +### 🧰 System Package Installation ```sh sudo apt update && sudo apt upgrade -y && sudo apt install -y \ -mesa-opencl-icd ocl-icd-opencl-dev gcc git jq pkg-config curl clang \ -build-essential hwloc libhwloc-dev libarchive-dev wget ntp python-is-python3 aria2 + mesa-opencl-icd ocl-icd-opencl-dev gcc git jq pkg-config curl clang \ + build-essential hwloc libhwloc-dev libarchive-dev wget ntp python-is-python3 aria2 ``` *** -### Install Go +### 🐹 Install Go (v1.23.7) ```sh sudo rm -rf /usr/local/go @@ -52,14 +54,14 @@ You should see something like: `go version go1.23.7 linux/amd64` *** -### Install Rust +### 🦀 Install Rust ```sh curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` {% hint style="info" %} -When prompted, choose the option 1) Proceed with standard installation (default — just press Enter). +💡 When prompted, choose the option 1) Proceed with standard installation (default — just press Enter). {% endhint %} ```sh @@ -73,7 +75,7 @@ You should see something like: `rustc 1.86.0 (05f9846f8 2025-03-31)` *** -### Add Go and Rust to Secure Sudo Path +### 🔐 Add Go and Rust to Secure Sudo Path ```sh sudo tee /etc/sudoers.d/dev-paths < ~/lotus.log 2>&1 & ``` -{% hint style="warning" %} -If you encounter errors related to `EnableEthRPC` or `EnableIndexer`, run the command below and restart Lotus +{% hint style="info" %} +💡 If you encounter errors related to `EnableEthRPC` or `EnableIndexer`, run the following command and restart Lotus {% endhint %} ```sh sed -i 's/EnableEthRPC = .*/EnableEthRPC = true/; s/EnableIndexer = .*/EnableIndexer = true/' ~/.lotus/config.toml ``` -#### **Monitor Sync Progress** +**Monitor Sync Progress** ```sh lotus sync wait @@ -160,7 +150,7 @@ To monitor continuously: lotus sync wait --watch ``` -#### **Monitor Logs** +**Monitor Logs** ```sh tail -f ~/lotus.log @@ -168,7 +158,7 @@ tail -f ~/lotus.log *** -### Create Wallets +### 🔐 Create Wallets You'll need to create **two BLS wallets**: @@ -181,27 +171,31 @@ lotus wallet new bls # Create worker wallet lotus wallet list # List all created wallets ``` -Make sure to send a small amount of FIL (Mainnet) or tFIL (Calibration) to each wallet - we recommend 1 FIL/tFIL per wallet to ensure the creation of your Storage Provider in Curio. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). +Make sure to send a small amount of FIL to each wallet - we recommend 1 FIL per wallet to ensure the creation of your Storage Provider in Curio. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). -{% hint style="info" %} -Both wallets will be used during Curio initialisation. +{% hint style="success" %} +✅ Both wallets will be used during Curio initialisation. {% endhint %} -{% hint style="danger" %} -[Back up](https://lotus.filecoin.io/lotus/manage/manage-fil/#exporting-and-importing-addresses) your wallet keys securely before continuing. Losing them will result in permanent loss of access to funds. +{% hint style="warning" %} +⚠️ **Tip:** [Back up](https://lotus.filecoin.io/lotus/manage/manage-fil/#exporting-and-importing-addresses) your wallet keys securely before continuing. Losing them will result in permanent loss of access to funds. {% endhint %} -## Install & Run YugabyteDB +*** -### Set ulimit configuration +## 🐘 Running YugabyteDB -{% hint style="info" %} - Before starting Yugabyte, you must increase the default `ulimit` values to ensure system limits do not interfere with the database. +🧠 Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions. + +### 🛠 Set ulimit configuration + +{% hint style="warning" %} +⚠️ Before starting Yugabyte, you must increase the default `ulimit` values to ensure system limits do not interfere with the database. {% endhint %} To do this: -#### **Persist new limits across reboots** +#### 🔁 **Persist new limits across reboots** Add these lines to `/etc/security/limits.conf`: @@ -212,19 +206,23 @@ echo "$(whoami) hard nofile 1048576" | sudo tee -a /etc/security/limits.conf This ensures the increased limits are automatically applied to future sessions. -#### **Apply limit immediately (for current shell only)** +#### ⚡ **Apply limit immediately (for current shell only)** ```sh ulimit -n 1048576 -# Verify limit change: +``` + +Verify: + +```sh ulimit -n ``` {% hint style="success" %} -This should output `1048576`. +✅ This should output `1048576`. {% endhint %} -### Install Yugabyte +### ⚙️ Install Yugabyte ```sh wget https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz @@ -233,7 +231,7 @@ cd yugabyte-2.25.1.0 ./bin/post_install.sh ``` -### Start the DB +### 🚀 Start the DB ```sh ./bin/yugabyted start \ @@ -242,8 +240,8 @@ cd yugabyte-2.25.1.0 --tserver_flags rpc_bind_addresses=127.0.0.1 ``` -{% hint style="danger" %} - If you encounter locale-related errors when starting Yugabyte for the first time, run: +{% hint style="warning" %} +⚠️ If you encounter locale-related errors when starting Yugabyte for the first time, run: {% endhint %} ```sh @@ -251,22 +249,24 @@ sudo locale-gen en_US.UTF-8 ``` {% hint style="success" %} -Visit `127.0.0.1:15433` to confirm successful installation. This is the YugabyteDB web UI — it should display the dashboard if the service is running correctly and all nodes are healthy. +✅ Visit `http://127.0.0.1:15433` to confirm successful installation. This is the YugabyteDB web UI — it should display the dashboard if the service is running correctly and all nodes are healthy. {% endhint %} {% hint style="info" %} -You can also check your Yugabyte cluster details directly in the CLI with: +💡 You can also check your Yugabyte cluster details directly in the CLI with: {% endhint %} ```sh ./bin/yugabyted status ``` -## Install & Run Curio +*** + +## 🧱 Installing and Configuring Curio -Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. +🧠 Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. -### System Configuration +### ⚙️ System Configuration Before you proceed with the installation, you should increase the UDP buffer size: @@ -282,33 +282,29 @@ echo 'net.core.rmem_max=2097152' | sudo tee -a /etc/sysctl.conf echo 'net.core.rmem_default=2097152' | sudo tee -a /etc/sysctl.conf ``` -### Build Curio +### 🔬 Build Curio Clone the repository and switch to the PDP branch: ```sh git clone https://github.com/filecoin-project/curio.git cd curio -git checkout synapse +git checkout rename ``` {% hint style="info" %} -Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below. +🛠 Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below. {% endhint %} -```sh -# For Filecoin Mainnet: -make clean build +{% hint style="info" %} +⏱️ This step will take a few minutes to complete. +{% endhint %} -# For Calibration Testnet: +```sh make clean calibnet ``` -{% hint style="success" %} -This step will take a few minutes to complete. -{% endhint %} - -### Install and Verify Curio +### ✅ Install and Verify Curio Run the following to install the compiled binary: @@ -320,23 +316,19 @@ This will place curio in `/usr/local/bin` Verify the installation: -```shell +```sh curio --version ``` -Expected output: +Expected example output: ```sh -# Example output for Mainnet: -curio version 1.24.4+mainnet+git_f954c0a_2025-04-06T15:46:32-04:00 - -# Example output for Calibration: curio version 1.24.4+calibnet+git_f954c0a_2025-04-06T15:46:32-04:00 ``` *** -### Guided Setup +### 🔧 Guided Setup Curio provides a utility to help you set up a new miner interactively. Run the following command: @@ -377,7 +369,7 @@ For this step, use the two BLS wallets you created earlier with Lotus: These addresses must match the Lotus wallets created earlier. {% hint style="info" %} -You can display your Lotus wallets at any time by running: +💡 You can display your Lotus wallets at any time by running: {% endhint %} ```sh @@ -399,15 +391,15 @@ Choose sector size: Review the information to ensure all inputs are correct. Then select "**Continue to verify the addresses and create a new miner actor**" to proceed. {% hint style="info" %} -This step may take a few minutes to complete as Curio pushes the message and waits for it to land on-chain. +⏱️ This step may take a few minutes to complete as Curio pushes the message and waits for it to land on-chain. {% endhint %} Once the actor is created, Curio will: -* `Register your miner ID` +* Register your miner ID {% hint style="info" %} -If the guided setup fails after creating the miner actor, run the following command to complete the installation: +💡 If the guided setup fails after creating the miner actor, run the following command to complete the installation: {% endhint %} ```sh @@ -448,11 +440,13 @@ http://127.0.0.1:4701 This will launch the Curio web GUI locally. -## Enable PDP +*** + +## 🧪 Enabling FWSS PDP -This section enables Proof of Data Possession (PDP) for a Storage Provider node using Curio. These steps guide you through running a standalone PDP service using Curio and pdptool. +🧠 This section enables **FWSS Proof of Data Possession (PDP)** on your SP node using Curio. These steps guide you through running a standalone PDP service using Curio and pdptool. -### Attach Storage Locations +### 📦 Attach Storage Locations With Curio running with the GUI layer: @@ -468,19 +462,19 @@ curio cli storage attach --init --store /long-term-storage/path ``` {% hint style="info" %} -Your fast-storage path should point to high-performance storage media such as NVMe or SSD +💡 Your fast-storage path should point to high-performance storage media such as NVMe or SSD {% endhint %} *** -### Add a PDP Configuration Layer +### 🔧 Add a PDP Configuration Layer -Browse to the Configurations page of the Curio GUI. +Browse to the **Configurations** page of the Curio GUI. -Create a new layer named pdp. Enable and set to `true` the following under Subsystems: +Create a new layer named **pdp** and enable the following under Subsystems: {% hint style="info" %} -You may find it helpful to search for the setting names in your browser. +💡 You may find it helpful to search for the setting names in your browser. {% endhint %} * ✅ `EnableParkPiece` @@ -488,55 +482,22 @@ You may find it helpful to search for the setting names in your browser. * ✅ `EnableCommP` * ✅ `EnableMoveStorage` -In the HTTP section: +In the **HTTP** section: * ✅ Enable: `true` * 🌐 DomainName: `your domain (e.g., pdp.mydomain.com)` * 📡 ListenAddress: `0.0.0.0:443` {% hint style="info" %} -You must point your domain's A record to your server's public IP address for Let's Encrypt to issue a certificate. +💡 **Tip:** You must point your domain's A record to your server's public IP address for Let's Encrypt to issue a certificate. {% endhint %} *** -### Set Up PDP Service Keys +### 💰 Import your Filecoin Wallet Private Key: -Build the pdptool: - -```sh -cd curio/cmd/pdptool -go build . -``` - -Generate a service secret: - -```sh -./pdptool create-service-secret -``` - -```sh -# Example output: - ------BEGIN PUBLIC KEY----- -LxP9MzVmHdC7KwYBvNAo1jXuIRfGXqQyo2JzE4Uctn0a5eFZbs6Wlvq3dKYgphTD -XAqRsm38LPt2iVcGb9MruZJxEkBhO71wDdNyaFMoXpCJnUqRAezvKlfbIg== ------END PUBLIC KEY----- -``` - -Browse to the **PDP** page of the Curio GUI and in the **Services** section: - -* Select **Add PDP Service** -* Input a **Service Name** of your choice (e.g. `pdp-service`) -* Copy the previously generated public key into the **Public Key** field. -* Select **Add Service** - -*** - -### Import your Filecoin Wallet Private Key: - -{% hint style="info" %} -There are several ways to obtain private keys for Ethereum addresses. For this guide, we will use a new delegated Filecoin wallet address. +{% hint style="warning" %} +⚠️ There are several ways to obtain private keys for Ethereum addresses. In this guide, we will use a new delegated FIL wallet address. {% endhint %} Create a new delegated wallet: @@ -547,12 +508,11 @@ lotus wallet new delegated ```sh # Example output: - t410fuo4dghaeiqzokiqnxruzdr6e3cjktnxprrc56bi ``` {% hint style="info" %} -You can display your Lotus wallets at any time by running: +💡 You can display your Lotus wallets at any time by running: {% endhint %} ```sh @@ -567,8 +527,7 @@ lotus wallet export | xxd -r -p | jq -r '.Privat ```sh # Example output: - -d4c2e3f9a716bb0e47fa91b2cf4a29870be3c5982fd6eafed71e8ac3f9c0b12 +d4c2e3f9a716bb0e47fa91b2cf4a29870be3c5982fd6eafed71e8ac3f9c0b127 ``` Browse to the **PDP** page of the Curio GUI and in the **Owner Address** section: @@ -578,18 +537,18 @@ Browse to the **PDP** page of the Curio GUI and in the **Owner Address** section * Select **Import Key** {% hint style="success" %} -Your 0x wallet address - the delegated Ethereum address derived from your Filecoin Metamask private key - will be added to the **Owner Address** section of the Curio PDP page. +✅ Your 0x wallet address - the delegated Ethereum address derived from your Filecoin delegated wallet private key - will be added to the **Owner Address** section of the Curio PDP page. {% endhint %} -Make sure to send a small amount of FIL to your 0x wallet - we recommend 5 FIL to ensure uninterrupted PDP operation during initial setup and testing. +Make sure to send a small amount of tFIL to your 0x wallet - we recommend 5 tFIL to ensure uninterrupted PDP operation during initial setup and testing. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). -{% hint style="danger" %} -Important: Secure your private key material. Don't expose or store it in plain text without protection. +{% hint style="warning" %} +⚠️ **Important:** Secure your private key material. Don't expose or store it in plain text without protection. {% endhint %} *** -### Restart and Verify +### 🚀 Restart and Verify Restart Curio with both layers: @@ -598,7 +557,7 @@ curio run --layers=gui,pdp ``` {% hint style="warning" %} -If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run: +⚠️ If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run: {% endhint %} ```sh @@ -607,56 +566,16 @@ sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/curio Test the PDP service: -```sh -./pdptool ping --service-url https://your-domain.com --service-name -``` - -{% hint style="success" %} -Use the service name specified in the **Service Name** field when you added your public **PDP Service** key - e.g. `pdp-service` -{% endhint %} - -Expected output: - -```sh -Ping successful: Service is reachable and JWT token is valid. -``` - {% hint style="info" %} -Note: The first ping often fails. Try again after a short delay. +💡 If `pdptool` is not installed, clone and build Curio: {% endhint %} -## Use PDP - -PDP ensures that your data is verifiably stored by a Filecoin Storage Provider using cryptographic proofs without needing to retrieve the file itself. - -### Prerequisites - -Before beginning, ensure: - -* You have access to a terminal with internet connectivity -* Your system has pdptool installed (bundled with Curio) - -### If pdptool is not installed: - -* **Option 1**: Clone Curio and build pdptool: - ```sh -git clone -https://github.com/filecoin-project/curio.git - -cd curio -cd cmd/pdptool +git clone https://github.com/filecoin-project/curio.git +cd curio/cmd/pdptool go build . ``` -* **Option 2**: Install the [Docker version ](https://github.com/LesnyRumcajs/pdp)of pdptool - Provided by our friends at [ChainSafe](https://chainsafe.io/) - -*** - -### Authenticate Your Client (JWT Token) - -You first need to authenticate your pdptool with a PDP-enabled Storage Provider - Generate a service secret: ```sh @@ -664,208 +583,40 @@ Generate a service secret: ``` ```sh -# Example output: - ------BEGIN PUBLIC KEY----- -LxP9MzVmHdC7KwYBvNAo1jXuIRfGXqQyo2JzE4Uctn0a5eFZbs6Wlvq3dKYgphTD -XAqRsm38LPt2iVcGb9MruZJxEkBhO71wDdNyaFMoXpCJnUqRAezvKlfbIg== ------END PUBLIC KEY----- -``` - -{% hint style="success" %} -Reach out in the [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) channel in Filecoin Slack to register your public key with a PDP-enabled Storage Provider -{% endhint %} - -*** - -### Connect to a PDP Service - -Start by pinging the PDP service to confirm availability: - -```sh -./pdptool ping --service-url https://yablu.net --service-name pdp-service -``` - -{% hint style="success" %} -You should see something like: -{% endhint %} - -```sh -Ping successful: Service is reachable and JWT token is valid. -``` - -*** - -### Create a Proof Set - -Start by creating an empty proof set. This step must happen **before** uploading files: - -```sh -./pdptool create-proof-set \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --recordkeeper 0x6170dE2b09b404776197485F3dc6c968Ef948505 -``` - -```sh -# Example output: - -Proof set creation initiated successfully. -Location: /pdp/proof-sets/created/0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 -Response: -``` - -Use the `0x` transaction hash from the previous output to monitor proof set creation status: - -```sh -./pdptool get-proof-set-create-status \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --tx-hash 0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 -``` - -{% hint style="success" %} -You should see something like: -{% endhint %} - -```sh -Proof Set Creation Status: -Transaction Hash: 0xf91617ef532748efb5a51e64391112e5328fbd9a5b9ac20e5127981cea0012a5 -Transaction Status: confirmed -Transaction Successful: true -Proofset Created: true -ProofSet ID: 43 -``` - -{% hint style="info" %} -The proof set creation process can take a few seconds to complete -{% endhint %} - -*** - -### Upload Files to the Storage Provider - -Once your proof set is ready, you can begin uploading files: - -```sh -./pdptool upload-file --service-url https://yablu.net --service-name pdp-service /path/to/file.ext -``` - -{% hint style="success" %} -Example output: -{% endhint %} - -```sh -0: pieceSize: 65536 -baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli:baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli -``` - -*** - -### 🌳 Add File Roots to Proof Set - -After uploading each file, extract its CID and add it to your proof set: - -```sh -./pdptool add-roots \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --proof-set-id \ - --root ++... -``` - -Example using the information returned in the previous steps: - -```sh -./pdptool add-roots \ - --service-url https://yablu.net \ - --service-name pdp-service \ - --proof-set-id 43 \ - --root baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli:baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli +./pdptool ping --service-url https://your-domain.com --service-name public ``` {% hint style="info" %} -In the above example, `--proof-set-id` came from the Create Proof Set step, and `--root` from the Upload Files to the Storage Provider step. +💡 Always use `public` for the `--service-name` flag {% endhint %} {% hint style="success" %} -Example output: +✅ Expected output: {% endhint %} ```sh -Roots added to proof set ID 43 successfully. -Response: -``` - -*** - -### View a Piece or Proof Set - -You can retrieve a proof set or inspect a file root directly: - -```sh -./pdptool get-proof-set \ - --service-url https://yablu.net \ - --service-name pdp-service 43 -``` - -{% hint style="success" %} -Example output: -{% endhint %} - -```sh -Proof Set ID: 43 -Next Challenge Epoch: 2577608 -Roots: - - Root ID: 0 - Root CID: baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli - Subroot CID: baga6ea4seaqhsevhssmv3j7jjavm4gzdckpjrvbwhhvn73sgibob5bdvtzoqkli - Subroot Offset: 0 -``` - -*** - -### Retrieve From a Proof Set - -Download a file using an ordered chunks list: - -```sh -./pdptool download-file \ - --service-url https://yablu.net \ - --chunk-file chunks.list \ - --output-file file.ext +Ping successful: Service is reachable and JWT token is valid. ``` {% hint style="info" %} -💡In the above example, `–chunk-file` and `–output-file` flags were defined in the Upload Files to the Storage Provider step +💡 Note: The first ping often fails. Try again after a short delay. {% endhint %} *** -## 🎉 You're Ready! +## 🎉 You're Done! -You've successfully launched a PDP-enabled Filecoin Storage Provider stack. Your system is now: +You've successfully launched a **PDP-enabled Filecoin Storage Provider** stack. Your system is now: * ✅ Syncing with the Filecoin network via Lotus * ✅ Recording deal and sector metadata in YugabyteDB * ✅ Operating Curio to manage sealing and coordination -* ✅ Submitting Proof of Data Possession to verify storage integrity - -You've also: - -✅ Connected to a PDP-enabled storage provider -✅ Created a proof set -✅ Uploaded files and added file roots -✅ Verified availability and proof status +* ✅ Enabled Proof of Data Possession (PDP) +* ✅ Connected to your PDP-enabled storage provider *** ## 🔜 Next Steps * 🧭 Monitor logs and metrics -* 🧭 Track your proof sets in the PDP Explorer: - * [Calibration PDP Explorer](https://calibration.pdp-explorer.eng.filoz.org) - * [Mainnet PDP Explorer](https://pdp-explorer.eng.filoz.org) -* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) - -💬 Questions? Join the conversation on Filecoin Slack: [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) \ No newline at end of file +* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) [#spx-pdp](https://filecoinproject.slack.com/archives/C08JQBA7PEX) \ No newline at end of file From b2be55e7b180b79f59dc7eae962dc3e79a67e8d3 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:25:56 +0100 Subject: [PATCH 04/17] Remove emojis from GitBook hint blocks Removed emoji symbols from all hint blocks as they are redundant with GitBook's color-coded alert banners. Kept emojis in section headers for better visual navigation. --- storage-providers/pdp/install-and-run-pdp.md | 56 ++++++++++---------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index 1e3f0583d..e74bebf71 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -61,7 +61,7 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` {% hint style="info" %} -💡 When prompted, choose the option 1) Proceed with standard installation (default — just press Enter). +When prompted, choose the option 1) Proceed with standard installation (default — just press Enter). {% endhint %} ```sh @@ -108,7 +108,7 @@ lotus --version ``` {% hint style="success" %} -✅ You should see something like: `lotus version 1.32.2+calibnet+git.ff88d8269` +You should see something like: `lotus version 1.32.2+calibnet+git.ff88d8269` {% endhint %} *** @@ -131,7 +131,7 @@ nohup lotus daemon > ~/lotus.log 2>&1 & ``` {% hint style="info" %} -💡 If you encounter errors related to `EnableEthRPC` or `EnableIndexer`, run the following command and restart Lotus +If you encounter errors related to `EnableEthRPC` or `EnableIndexer`, run the following command and restart Lotus {% endhint %} ```sh @@ -174,11 +174,11 @@ lotus wallet list # List all created wallets Make sure to send a small amount of FIL to each wallet - we recommend 1 FIL per wallet to ensure the creation of your Storage Provider in Curio. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). {% hint style="success" %} -✅ Both wallets will be used during Curio initialisation. +Both wallets will be used during Curio initialisation. {% endhint %} {% hint style="warning" %} -⚠️ **Tip:** [Back up](https://lotus.filecoin.io/lotus/manage/manage-fil/#exporting-and-importing-addresses) your wallet keys securely before continuing. Losing them will result in permanent loss of access to funds. +**Tip:** [Back up](https://lotus.filecoin.io/lotus/manage/manage-fil/#exporting-and-importing-addresses) your wallet keys securely before continuing. Losing them will result in permanent loss of access to funds. {% endhint %} *** @@ -190,7 +190,7 @@ Make sure to send a small amount of FIL to each wallet - we recommend 1 FIL per ### 🛠 Set ulimit configuration {% hint style="warning" %} -⚠️ Before starting Yugabyte, you must increase the default `ulimit` values to ensure system limits do not interfere with the database. +Before starting Yugabyte, you must increase the default `ulimit` values to ensure system limits do not interfere with the database. {% endhint %} To do this: @@ -219,7 +219,7 @@ ulimit -n ``` {% hint style="success" %} -✅ This should output `1048576`. +This should output `1048576`. {% endhint %} ### ⚙️ Install Yugabyte @@ -241,7 +241,7 @@ cd yugabyte-2.25.1.0 ``` {% hint style="warning" %} -⚠️ If you encounter locale-related errors when starting Yugabyte for the first time, run: +If you encounter locale-related errors when starting Yugabyte for the first time, run: {% endhint %} ```sh @@ -249,11 +249,11 @@ sudo locale-gen en_US.UTF-8 ``` {% hint style="success" %} -✅ Visit `http://127.0.0.1:15433` to confirm successful installation. This is the YugabyteDB web UI — it should display the dashboard if the service is running correctly and all nodes are healthy. +Visit `http://127.0.0.1:15433` to confirm successful installation. This is the YugabyteDB web UI — it should display the dashboard if the service is running correctly and all nodes are healthy. {% endhint %} {% hint style="info" %} -💡 You can also check your Yugabyte cluster details directly in the CLI with: +You can also check your Yugabyte cluster details directly in the CLI with: {% endhint %} ```sh @@ -293,11 +293,11 @@ git checkout rename ``` {% hint style="info" %} -🛠 Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below. +Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below. {% endhint %} {% hint style="info" %} -⏱️ This step will take a few minutes to complete. +This step will take a few minutes to complete. {% endhint %} ```sh @@ -369,7 +369,7 @@ For this step, use the two BLS wallets you created earlier with Lotus: These addresses must match the Lotus wallets created earlier. {% hint style="info" %} -💡 You can display your Lotus wallets at any time by running: +You can display your Lotus wallets at any time by running: {% endhint %} ```sh @@ -383,7 +383,7 @@ Choose sector size: * `64 GiB` {% hint style="info" %} -💡 Selecting a sector size is required during the Curio guided setup, but **PDP itself doesn't use sectors**. Proof set sizes in PDP are **arbitrary and fully flexible.** +Selecting a sector size is required during the Curio guided setup, but **PDP itself doesn't use sectors**. Proof set sizes in PDP are **arbitrary and fully flexible.** {% endhint %} #### 5️⃣ Create Miner Actor @@ -391,7 +391,7 @@ Choose sector size: Review the information to ensure all inputs are correct. Then select "**Continue to verify the addresses and create a new miner actor**" to proceed. {% hint style="info" %} -⏱️ This step may take a few minutes to complete as Curio pushes the message and waits for it to land on-chain. +This step may take a few minutes to complete as Curio pushes the message and waits for it to land on-chain. {% endhint %} Once the actor is created, Curio will: @@ -399,7 +399,7 @@ Once the actor is created, Curio will: * Register your miner ID {% hint style="info" %} -💡 If the guided setup fails after creating the miner actor, run the following command to complete the installation: +If the guided setup fails after creating the miner actor, run the following command to complete the installation: {% endhint %} ```sh @@ -462,7 +462,7 @@ curio cli storage attach --init --store /long-term-storage/path ``` {% hint style="info" %} -💡 Your fast-storage path should point to high-performance storage media such as NVMe or SSD +Your fast-storage path should point to high-performance storage media such as NVMe or SSD {% endhint %} *** @@ -474,7 +474,7 @@ Browse to the **Configurations** page of the Curio GUI. Create a new layer named **pdp** and enable the following under Subsystems: {% hint style="info" %} -💡 You may find it helpful to search for the setting names in your browser. +You may find it helpful to search for the setting names in your browser. {% endhint %} * ✅ `EnableParkPiece` @@ -489,7 +489,7 @@ In the **HTTP** section: * 📡 ListenAddress: `0.0.0.0:443` {% hint style="info" %} -💡 **Tip:** You must point your domain's A record to your server's public IP address for Let's Encrypt to issue a certificate. +**Tip:** You must point your domain's A record to your server's public IP address for Let's Encrypt to issue a certificate. {% endhint %} *** @@ -497,7 +497,7 @@ In the **HTTP** section: ### 💰 Import your Filecoin Wallet Private Key: {% hint style="warning" %} -⚠️ There are several ways to obtain private keys for Ethereum addresses. In this guide, we will use a new delegated FIL wallet address. +There are several ways to obtain private keys for Ethereum addresses. In this guide, we will use a new delegated FIL wallet address. {% endhint %} Create a new delegated wallet: @@ -512,7 +512,7 @@ t410fuo4dghaeiqzokiqnxruzdr6e3cjktnxprrc56bi ``` {% hint style="info" %} -💡 You can display your Lotus wallets at any time by running: +You can display your Lotus wallets at any time by running: {% endhint %} ```sh @@ -537,13 +537,13 @@ Browse to the **PDP** page of the Curio GUI and in the **Owner Address** section * Select **Import Key** {% hint style="success" %} -✅ Your 0x wallet address - the delegated Ethereum address derived from your Filecoin delegated wallet private key - will be added to the **Owner Address** section of the Curio PDP page. +Your 0x wallet address - the delegated Ethereum address derived from your Filecoin delegated wallet private key - will be added to the **Owner Address** section of the Curio PDP page. {% endhint %} Make sure to send a small amount of tFIL to your 0x wallet - we recommend 5 tFIL to ensure uninterrupted PDP operation during initial setup and testing. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). {% hint style="warning" %} -⚠️ **Important:** Secure your private key material. Don't expose or store it in plain text without protection. +**Important:** Secure your private key material. Don't expose or store it in plain text without protection. {% endhint %} *** @@ -557,7 +557,7 @@ curio run --layers=gui,pdp ``` {% hint style="warning" %} -⚠️ If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run: +If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run: {% endhint %} ```sh @@ -567,7 +567,7 @@ sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/curio Test the PDP service: {% hint style="info" %} -💡 If `pdptool` is not installed, clone and build Curio: +If `pdptool` is not installed, clone and build Curio: {% endhint %} ```sh @@ -587,11 +587,11 @@ Generate a service secret: ``` {% hint style="info" %} -💡 Always use `public` for the `--service-name` flag +Always use `public` for the `--service-name` flag {% endhint %} {% hint style="success" %} -✅ Expected output: +Expected output: {% endhint %} ```sh @@ -599,7 +599,7 @@ Ping successful: Service is reachable and JWT token is valid. ``` {% hint style="info" %} -💡 Note: The first ping often fails. Try again after a short delay. +Note: The first ping often fails. Try again after a short delay. {% endhint %} *** From 05d23f9506cbcace0ab448036a19829d6e95b07d Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:30:27 +0100 Subject: [PATCH 05/17] Update PDP documentation banner and remove link matrix - Changed banner from DEPRECATED to ALPHA FEATURE - UNDER DEVELOPMENT - Removed link matrix cards table from top of document - Clarified that PDP is in active development for testing purposes --- storage-providers/pdp/install-and-run-pdp.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index e74bebf71..a769443d8 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -7,16 +7,14 @@ description: >- # Install & Run PDP {% hint style="danger" %} -**DEPRECATED DEVELOPER TOOL** +**ALPHA FEATURE - UNDER DEVELOPMENT** -This documentation refers to the legacy `pdptool`, which is intended only for low-level developer testing. -It is not the recommended method for onboarding or interacting with PDP Storage Providers. +This documentation covers the PDP (Proof of Data Possession) feature, which is currently in alpha and under active development. +This tool is intended for testing and experimental use only. -For current usage, including working with live PDP SPs and submitting real deals, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk) and [Synapse dApp Tutorial](https://github.com/FIL-Builders/fs-upload-dapp). +For production use and submitting real deals with live PDP Storage Providers, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk) and [Synapse dApp Tutorial](https://github.com/FIL-Builders/fs-upload-dapp). {% endhint %} -
Lotus Documentationlotus-logo-big.png
Yugabyte Documentationyugabyte.svg
Curio DocumentationCurio_placeholder.webp
PDP DocumentationCurio_placeholder.webp
Filecoin Slack - #fil-pdpFilecoin.svg.png
Filecoin Slack - #fil-curio-helpFilecoin.svg.png
Filecoin Slack - #fil-lotus-helpFilecoin.svg.png
Filecoin Wallet - MetaMask SetupFilecoin.svg.png
Yugabyte Slackyugabyte.svg
- ## 🚀 Prerequisites {% hint style="warning" %} From 56f5a7afe48f161385d596f664fcb6da3bcd24ef Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 11:37:46 +0000 Subject: [PATCH 06/17] GITBOOK-3: No subject --- storage-providers/pdp/install-and-run-pdp.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index a769443d8..5c96b3474 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -9,10 +9,9 @@ description: >- {% hint style="danger" %} **ALPHA FEATURE - UNDER DEVELOPMENT** -This documentation covers the PDP (Proof of Data Possession) feature, which is currently in alpha and under active development. -This tool is intended for testing and experimental use only. +This documentation covers the PDP (Proof of Data Possession) feature, which is currently in alpha and under active development. This tool is intended for testing and experimental use only. -For production use and submitting real deals with live PDP Storage Providers, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk) and [Synapse dApp Tutorial](https://github.com/FIL-Builders/fs-upload-dapp). +For production use and submitting real deals with live PDP Storage Providers, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk). {% endhint %} ## 🚀 Prerequisites @@ -35,7 +34,7 @@ sudo apt update && sudo apt upgrade -y && sudo apt install -y \ *** -### 🐹 Install Go (v1.23.7) +### :hammer: Install Go (v1.23.7) ```sh sudo rm -rf /usr/local/go @@ -52,7 +51,7 @@ You should see something like: `go version go1.23.7 linux/amd64` *** -### 🦀 Install Rust +### :wrench: Install Rust ```sh curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh @@ -617,4 +616,4 @@ You've successfully launched a **PDP-enabled Filecoin Storage Provider** stack. ## 🔜 Next Steps * 🧭 Monitor logs and metrics -* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) [#spx-pdp](https://filecoinproject.slack.com/archives/C08JQBA7PEX) \ No newline at end of file +* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) [#spx-pdp](https://filecoinproject.slack.com/archives/C08JQBA7PEX) From fc77514801e23286f4d575b324680ccf0d696c51 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 11:43:08 +0000 Subject: [PATCH 07/17] GITBOOK-4: No subject From cbc0cababfe53d19ebf6f51b31a128ba3d0de5f6 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:44:24 +0100 Subject: [PATCH 08/17] Restore section-specific documentation link tables Added back documentation and support links for each major component: - Lotus section: Documentation and #fil-lotus-help Slack - YugabyteDB section: Documentation and Yugabyte Slack - Curio section: Documentation, PDP docs, and relevant Slack channels These section-specific resources were lost during the initial file consolidation. --- storage-providers/pdp/install-and-run-pdp.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index 5c96b3474..110dfd185 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -84,6 +84,8 @@ EOF ## ⛓️ Installing and Running Lotus - Calibration Testnet +
Lotus Documentationlotus-logo-big.png
Filecoin Slack - #fil-lotus-helpFilecoin.svg.png
+ 🧠 Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node. ### 🔧 Build Lotus Daemon @@ -182,6 +184,8 @@ Both wallets will be used during Curio initialisation. ## 🐘 Running YugabyteDB +
Yugabyte Documentationyugabyte.svg
Yugabyte Slackyugabyte.svg
+ 🧠 Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions. ### 🛠 Set ulimit configuration @@ -261,6 +265,8 @@ You can also check your Yugabyte cluster details directly in the CLI with: ## 🧱 Installing and Configuring Curio +
Curio DocumentationCurio_placeholder.webp
PDP DocumentationCurio_placeholder.webp
Filecoin Slack - #fil-curio-helpFilecoin.svg.png
Filecoin Slack - #fil-pdpFilecoin.svg.png
+ 🧠 Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. ### ⚙️ System Configuration From ab052da8b17d4aeee9de6e04afbeff6d22db54fd Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:52:03 +0100 Subject: [PATCH 09/17] Fix documentation tables to match PDF format - Removed unnecessary column headers - Added light blue background color (#e3f2fd) - Simplified table structure to match original PDF - Corrected labels to match PDF exactly --- storage-providers/pdp/install-and-run-pdp.md | 21 +++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index 110dfd185..5559b30ce 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -84,7 +84,12 @@ EOF ## ⛓️ Installing and Running Lotus - Calibration Testnet -
Lotus Documentationlotus-logo-big.png
Filecoin Slack - #fil-lotus-helpFilecoin.svg.png
+ + + + + +
Lotus Documentationhttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/
Lotus Support ChannelsFilecoin Slack - #fil-lotus-help
🧠 Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node. @@ -184,7 +189,12 @@ Both wallets will be used during Curio initialisation. ## 🐘 Running YugabyteDB -
Yugabyte Documentationyugabyte.svg
Yugabyte Slackyugabyte.svg
+ + + + + +
Yugabyte Documentationhttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/
Yugabyte Support ChannelsFilecoin Slack - #fil-curio-help - Yugabyte Slack
🧠 Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions. @@ -265,7 +275,12 @@ You can also check your Yugabyte cluster details directly in the CLI with: ## 🧱 Installing and Configuring Curio -
Curio DocumentationCurio_placeholder.webp
PDP DocumentationCurio_placeholder.webp
Filecoin Slack - #fil-curio-helpFilecoin.svg.png
Filecoin Slack - #fil-pdpFilecoin.svg.png
+ + + + + +
Curio Documentationhttps://docs.curiostorage.org/
Curio Support ChannelsFilecoin Slack - #fil-curio-help
🧠 Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. From 08d1f27c4e58745243adbeede657bda1c61432b6 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:56:09 +0100 Subject: [PATCH 10/17] Fix table placement and restore card format - Moved tables to correct position after description paragraphs - Restored GitBook card-style tables with logos - Matches original PDF layout exactly --- storage-providers/pdp/install-and-run-pdp.md | 27 +++++--------------- 1 file changed, 6 insertions(+), 21 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index 5559b30ce..592e5ca8b 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -84,15 +84,10 @@ EOF ## ⛓️ Installing and Running Lotus - Calibration Testnet - - - - - -
Lotus Documentationhttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/
Lotus Support ChannelsFilecoin Slack - #fil-lotus-help
- 🧠 Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node. +
Lotus Documentationhttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/lotus-logo-big.png
Lotus Support ChannelsFilecoin Slack - #fil-lotus-helpFilecoin.svg.png
+ ### 🔧 Build Lotus Daemon Clone and check out Lotus: @@ -189,15 +184,10 @@ Both wallets will be used during Curio initialisation. ## 🐘 Running YugabyteDB - - - - - -
Yugabyte Documentationhttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/
Yugabyte Support ChannelsFilecoin Slack - #fil-curio-help - Yugabyte Slack
- 🧠 Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions. +
Yugabyte Documentationhttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/yugabyte.svg
Yugabyte Support ChannelsFilecoin Slack - #fil-curio-help - Yugabyte Slackyugabyte.svg
+ ### 🛠 Set ulimit configuration {% hint style="warning" %} @@ -275,15 +265,10 @@ You can also check your Yugabyte cluster details directly in the CLI with: ## 🧱 Installing and Configuring Curio - - - - - -
Curio Documentationhttps://docs.curiostorage.org/
Curio Support ChannelsFilecoin Slack - #fil-curio-help
- 🧠 Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. +
Curio Documentationhttps://docs.curiostorage.org/Curio_placeholder.webp
Curio Support ChannelsFilecoin Slack - #fil-curio-helpFilecoin.svg.png
+ ### ⚙️ System Configuration Before you proceed with the installation, you should increase the UDP buffer size: From b9544f4bfcfe5bd2c3ac8c0eecfe203b71b6890e Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 13:01:04 +0100 Subject: [PATCH 11/17] Update PDP README banner to match alpha status Changed main PDP page warning from DEPRECATED to ALPHA FEATURE - UNDER DEVELOPMENT to match the Install & Run PDP page --- storage-providers/pdp/README.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/storage-providers/pdp/README.md b/storage-providers/pdp/README.md index 02aff5475..0bd1450ff 100644 --- a/storage-providers/pdp/README.md +++ b/storage-providers/pdp/README.md @@ -15,10 +15,9 @@ layout: # PDP {% hint style="danger" %} -**DEPRECATED DEVELOPER TOOL**\ -\ -This documentation refers to the legacy `pdptool`, which is intended only for low-level developer testing.\ -It is not the recommended method for onboarding or interacting with PDP Storage Providers. +**ALPHA FEATURE - UNDER DEVELOPMENT** -For current usage, including working with live PDP SPs and submitting real deals, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk) and [Synapse dApp Tutorial](https://github.com/FIL-Builders/fs-upload-dapp). +This documentation covers the PDP (Proof of Data Possession) feature, which is currently in alpha and under active development. This tool is intended for testing and experimental use only. + +For production use and submitting real deals with live PDP Storage Providers, please use the [Synapse SDK](https://github.com/FilOzone/synapse-sdk). {% endhint %} From b5de69da9e2631a1403d586a0428d3c4f78e2667 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:05:09 +0000 Subject: [PATCH 12/17] GITBOOK-5: No subject --- storage-providers/pdp/install-and-run-pdp.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index 592e5ca8b..cf37501b9 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -86,7 +86,7 @@ EOF 🧠 Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node. -
Lotus Documentationhttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/lotus-logo-big.png
Lotus Support ChannelsFilecoin Slack - #fil-lotus-helpFilecoin.svg.png
+
Lotus Documentationhttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/lotus-logo-big.png
Lotus Support ChannelsFilecoin Slack - #fil-lotus-helpFilecoin.svg.png
### 🔧 Build Lotus Daemon @@ -186,7 +186,7 @@ Both wallets will be used during Curio initialisation. 🧠 Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions. -
Yugabyte Documentationhttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/yugabyte.svg
Yugabyte Support ChannelsFilecoin Slack - #fil-curio-help - Yugabyte Slackyugabyte.svg
+
Yugabyte Documentationhttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/yugabyte.svg
Yugabyte Support ChannelsFilecoin Slack - #fil-curio-help - Yugabyte Slackyugabyte.svg
### 🛠 Set ulimit configuration @@ -267,7 +267,7 @@ You can also check your Yugabyte cluster details directly in the CLI with: 🧠 Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. -
Curio Documentationhttps://docs.curiostorage.org/Curio_placeholder.webp
Curio Support ChannelsFilecoin Slack - #fil-curio-helpFilecoin.svg.png
+
Curio Documentationhttps://docs.curiostorage.org/Curio_placeholder.webp
Curio Support ChannelsFilecoin Slack - #fil-curio-helpFilecoin.svg.png
### ⚙️ System Configuration From a00959e7fece038179b550160c805ae3fa6a7569 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:25:50 +0000 Subject: [PATCH 13/17] GITBOOK-6: No subject --- storage-providers/pdp/install-and-run-pdp.md | 40 +++++++++++++++----- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index cf37501b9..c092d1252 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -82,7 +82,7 @@ EOF *** -## ⛓️ Installing and Running Lotus - Calibration Testnet +## ⛓️ Installing and Running Lotus 🧠 Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node. @@ -98,6 +98,14 @@ cd lotus git checkout $(curl -s https://api.github.com/repos/filecoin-project/lotus/releases/latest | jq -r .tag_name) ``` +**Build and Install for Mainnet** + +```sh +make clean && make lotus +sudo make install-daemon +lotus --version +``` + **Build and Install for Calibration** ```sh @@ -116,6 +124,12 @@ You should see something like: `lotus version 1.32.2+calibnet+git.ff88d8269` Download the Snapshot +**Mainnet:** + +```sh +aria2c -x5 -o snapshot.car.zst https://forest-archive.chainsafe.dev/latest/mainnet/ +``` + **Calibration:** ```sh @@ -299,14 +313,22 @@ git checkout rename Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below. {% endhint %} -{% hint style="info" %} -This step will take a few minutes to complete. -{% endhint %} +Mainnet + +```sh +make clean build +``` + +Calibration ```sh make clean calibnet ``` +{% hint style="info" %} +This step will take a few minutes to complete. +{% endhint %} + ### ✅ Install and Verify Curio Run the following to install the compiled binary: @@ -449,6 +471,8 @@ This will launch the Curio web GUI locally. 🧠 This section enables **FWSS Proof of Data Possession (PDP)** on your SP node using Curio. These steps guide you through running a standalone PDP service using Curio and pdptool. +
Cover image
PDP Support ChannelsFilecoin.svg.pnghttps://filecoinproject.slack.com/archives/C0717TGU7V2
+ ### 📦 Attach Storage Locations With Curio running with the GUI layer: @@ -543,7 +567,7 @@ Browse to the **PDP** page of the Curio GUI and in the **Owner Address** section Your 0x wallet address - the delegated Ethereum address derived from your Filecoin delegated wallet private key - will be added to the **Owner Address** section of the Curio PDP page. {% endhint %} -Make sure to send a small amount of tFIL to your 0x wallet - we recommend 5 tFIL to ensure uninterrupted PDP operation during initial setup and testing. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). +Make sure to send a small amount of FIL or tFIL (testnet FIL) to your 0x wallet - we recommend 8 FIL for Mainnet & 5 tFIL for Calibration to ensure uninterrupted PDP operation during initial setup and testing. [Calibration test FIL faucet information](https://docs.filecoin.io/smart-contracts/developing-contracts/get-test-tokens). {% hint style="warning" %} **Important:** Secure your private key material. Don't expose or store it in plain text without protection. @@ -601,10 +625,6 @@ Expected output: Ping successful: Service is reachable and JWT token is valid. ``` -{% hint style="info" %} -Note: The first ping often fails. Try again after a short delay. -{% endhint %} - *** ## 🎉 You're Done! @@ -622,4 +642,4 @@ You've successfully launched a **PDP-enabled Filecoin Storage Provider** stack. ## 🔜 Next Steps * 🧭 Monitor logs and metrics -* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) [#spx-pdp](https://filecoinproject.slack.com/archives/C08JQBA7PEX) +* 💬 Join the community - Filecoin Slack - [#fil-pdp](https://filecoinproject.slack.com/archives/C0717TGU7V2) From 362b152f0aa64274c46c56bc96a30e13c0e16c3e Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:29:57 +0000 Subject: [PATCH 14/17] GITBOOK-7: No subject --- storage-providers/pdp/install-and-run-pdp.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index c092d1252..3c6f85c65 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -86,7 +86,7 @@ EOF 🧠 Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node. -
Lotus Documentationhttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/lotus-logo-big.png
Lotus Support ChannelsFilecoin Slack - #fil-lotus-helpFilecoin.svg.png
+
Lotus Documentationhttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/lotus-logo-big.pnghttps://lotus.filecoin.io/lotus/get-started/what-is-lotus/
Lotus Support ChannelsFilecoin Slack - #fil-lotus-helpFilecoin.svg.pnghttps://filecoinproject.slack.com/archives/CPFTWMY7N
### 🔧 Build Lotus Daemon @@ -200,7 +200,7 @@ Both wallets will be used during Curio initialisation. 🧠 Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions. -
Yugabyte Documentationhttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/yugabyte.svg
Yugabyte Support ChannelsFilecoin Slack - #fil-curio-help - Yugabyte Slackyugabyte.svg
+
Cover image
Yugabyte Documentationhttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/yugabyte.svghttps://docs.yugabyte.com/preview/tutorials/quick-start/linux/
Yugabyte Support ChannelsFilecoin Slack - #fil-curio-help - Yugabyte SlackCurio_placeholder.webphttps://filecoinproject.slack.com/archives/C06LF5YP8S3
### 🛠 Set ulimit configuration @@ -281,7 +281,7 @@ You can also check your Yugabyte cluster details directly in the CLI with: 🧠 Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs. -
Curio Documentationhttps://docs.curiostorage.org/Curio_placeholder.webp
Curio Support ChannelsFilecoin Slack - #fil-curio-helpFilecoin.svg.png
+
Curio Documentationhttps://docs.curiostorage.org/Curio_placeholder.webphttps://docs.curiostorage.org/
Curio Support ChannelsFilecoin Slack - #fil-curio-helpFilecoin.svg.pnghttps://filecoinproject.slack.com/archives/C06LF5YP8S3
### ⚙️ System Configuration From 568c30845bb24fa9d45d2b6e550f5fb2b3933810 Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 12:30:47 +0000 Subject: [PATCH 15/17] GITBOOK-8: No subject From f5aa91ced878c658ed6525227ba35507914183cf Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Wed, 24 Sep 2025 16:28:40 +0100 Subject: [PATCH 16/17] Upgrade Go installation from v1.23.7 to v1.24.0 Updated Go installation instructions to version 1.24.0. --- storage-providers/pdp/install-and-run-pdp.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index 3c6f85c65..ebe95c8cc 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -34,12 +34,12 @@ sudo apt update && sudo apt upgrade -y && sudo apt install -y \ *** -### :hammer: Install Go (v1.23.7) +### :hammer: Install Go (v1.24.0) ```sh sudo rm -rf /usr/local/go -wget https://go.dev/dl/go1.23.7.linux-amd64.tar.gz -sudo tar -C /usr/local -xzf go1.23.7.linux-amd64.tar.gz +wget https://go.dev/dl/go1.24.0.linux-amd64.tar.gz +sudo tar -C /usr/local -xzf go1.24.0.linux-amd64.tar.gz echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc source ~/.bashrc go version From 52d100794de1f3697cf5dec7db9ead31c2a8d12a Mon Sep 17 00:00:00 2001 From: TippyFlits Date: Thu, 25 Sep 2025 14:11:46 +0100 Subject: [PATCH 17/17] Change git checkout branch to 'pdpv0' Updated the git checkout command to use 'pdpv0' branch instead of 'rename'. --- storage-providers/pdp/install-and-run-pdp.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/storage-providers/pdp/install-and-run-pdp.md b/storage-providers/pdp/install-and-run-pdp.md index ebe95c8cc..c946aa02c 100644 --- a/storage-providers/pdp/install-and-run-pdp.md +++ b/storage-providers/pdp/install-and-run-pdp.md @@ -306,7 +306,7 @@ Clone the repository and switch to the PDP branch: ```sh git clone https://github.com/filecoin-project/curio.git cd curio -git checkout rename +git checkout pdpv0 ``` {% hint style="info" %}