Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions docker/builder/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,23 @@ RUN apt-get update && apt-get install -y \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*

# Install Go 1.24.7
RUN wget -c https://golang.org/dl/go1.24.7.linux-amd64.tar.gz -O - | tar -xz -C /usr/local
# Install Go 1.24.7 (auto-detect architecture)
ARG TARGETARCH
RUN GOARCH=${TARGETARCH:-amd64} && \
wget -c https://golang.org/dl/go1.24.7.linux-${GOARCH}.tar.gz -O - | tar -xz -C /usr/local
ENV PATH=$PATH:/usr/local/go/bin

# Create foc-user and foc-group with matching host UID/GID when it is run. The 1002 below is just a placeholder which will be replaced during build process. See `docker.rs`
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user && \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user && \
mkdir -p /home/foc-user/go/pkg && \
chown -R foc-user:foc-group /home/foc-user
chown -R foc-user:$GROUP_NAME /home/foc-user

# Install Rust as foc-user
USER foc-user
Expand All @@ -51,7 +56,7 @@ RUN curl -fsSL https://deb.nodesource.com/setup_24.x | bash - && \
WORKDIR /workspace

# Define volumes for external access
VOLUME ["/home/foc-user/.cargo", "/home/foc-user/.rustup", "/home/foc-user/go", "/var/tmp/filecoin-proof-parameters"]
Copy link
Collaborator

@redpanda-f redpanda-f Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed? This is good for caching.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only works when there is data present. The first time you run the mount, it will clear these directories, causing the toolchain to be lost. I added the critical mappings in a subsequent commit.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is true, but it cleans up only the first time. The caches are for when the curio or lotus codebase is actively being changed and rebuilt frequently. In those instances, re-init is not needed, and these caches would be useful.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

The cargo bin directory was overwritten, so this binary will never exist unless it is manually downloaded on the host machine.

There are two commits here: one temporarily disables everything, and the second maps according to what's actually usable. The -v parameter plays a crucial role. Therefore, I've removed it here because I think it's meaningless.

However, I must admit that some Rust binaries are not cached. The rest, such as go pkg and rust registry, are cached.

VOLUME ["/var/tmp/filecoin-proof-parameters"]

# Default command
CMD ["/bin/bash"]
23 changes: 18 additions & 5 deletions docker/builder/volumes_map.toml
Original file line number Diff line number Diff line change
@@ -1,9 +1,22 @@
# Volume mappings for builder Docker image
# Format: host_subdirectory = "container_path"
# Host paths will be relative to ~/.foc-devnet/artifacts/docker/volumes/builder/
# Host paths will be relative to ~/.foc-devnet/docker/volumes/cache/foc-c-builder/
#
# Strategy: Mount only dependency cache directories, not tool directories
# This allows caching of downloaded dependencies while keeping tools functional.

[volumes]
cargo = "/home/foc-user/.cargo"
rustup = "/home/foc-user/.rustup"
go = "/home/foc-user/go"
foundry = "/home/foc-user/.foundry"
# Cargo dependency caches (NOT the cargo binary itself)
cargo-registry = "/home/foc-user/.cargo/registry" # crates.io downloads
cargo-git = "/home/foc-user/.cargo/git" # git dependencies

# Go module cache (NOT the go toolchain)
go-pkg = "/home/foc-user/go/pkg" # go modules cache

# Note: We do NOT mount:
# - /home/foc-user/.cargo/bin (contains cargo/rustc binaries)
# - /home/foc-user/.rustup/toolchains (contains rust toolchain)
# - /home/foc-user/.rustup/downloads (rustup deletes files after use)
# - /home/foc-user/.foundry (contains forge/cast binaries)
# - /usr/local/go (Go toolchain itself)

14 changes: 9 additions & 5 deletions docker/curio/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,20 @@ RUN apt-get update && apt-get install -y \
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Create necessary directories with proper ownership
RUN mkdir -p /var/tmp/filecoin-proof-parameters \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
mkdir -p /var/tmp/filecoin-proof-parameters \
&& mkdir -p /home/foc-user/.curio \
&& mkdir -p /home/foc-user/curio/fast-storage \
&& mkdir -p /home/foc-user/curio/long-term-storage \
&& chown -R foc-user:foc-group /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:foc-group /home/foc-user
&& chown -R foc-user:$GROUP_NAME /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:$GROUP_NAME /home/foc-user

# Switch to foc-user
USER foc-user
Expand Down
14 changes: 9 additions & 5 deletions docker/lotus-miner/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,20 @@ RUN apt-get update && apt-get install -y \
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Create necessary directories with proper ownership
RUN mkdir -p /var/tmp/filecoin-proof-parameters \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
mkdir -p /var/tmp/filecoin-proof-parameters \
&& mkdir -p /home/foc-user/.lotus-miner-local-net \
&& mkdir -p /home/foc-user/.lotus-local-net \
&& mkdir -p /home/foc-user/.genesis-sectors \
&& chown -R foc-user:foc-group /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:foc-group /home/foc-user
&& chown -R foc-user:$GROUP_NAME /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:$GROUP_NAME /home/foc-user

# Switch to foc-user
USER foc-user
Expand Down
14 changes: 9 additions & 5 deletions docker/lotus/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,19 @@ RUN apt-get update && apt-get install -y \
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Create necessary directories with proper ownership
RUN mkdir -p /var/tmp/filecoin-proof-parameters \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
mkdir -p /var/tmp/filecoin-proof-parameters \
&& mkdir -p /home/foc-user/.lotus-local-net \
&& mkdir -p /home/foc-user/.genesis \
&& chown -R foc-user:foc-group /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:foc-group /home/foc-user
&& chown -R foc-user:$GROUP_NAME /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:$GROUP_NAME /home/foc-user

# Switch to foc-user
USER foc-user
Expand Down
12 changes: 8 additions & 4 deletions docker/yugabyte/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@ ENV LC_ALL=en_US.UTF-8
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Set working directory
WORKDIR /yugabyte
Expand All @@ -30,9 +33,10 @@ ARG YUGABYTE_SOURCE=yugabyte
COPY ${YUGABYTE_SOURCE} /yugabyte

# Run post-install script and set ownership
RUN /yugabyte/bin/post_install.sh && \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
/yugabyte/bin/post_install.sh && \
mkdir -p /yugabyte/data && \
chown -R foc-user:foc-group /yugabyte
chown -R foc-user:$GROUP_NAME /yugabyte

# Switch to foc-user
USER foc-user
Expand Down
3 changes: 1 addition & 2 deletions src/commands/build/docker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,7 @@ pub fn setup_docker_run_args(

let mut docker_run_args = vec![
"run".to_string(),
"-u".to_string(),
"foc-user".to_string(),
"--rm".to_string(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure if the intent really is:

  • "do not use foc-user for building"
  • "remove" the container, lose logs

I don't disagree with them, but would like to understand safety argument for removing -u foc-user.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --rm option is used because if cargo run --build lotus fails, subsequent runs will save the changes, requiring manual cleanup. Error logs will be output upon exit, so there's no need to worry.

The -u foc-user was removed because you added -u gid:uid repeatedly later, so this redundant part was removed.

"--name".to_string(),
container_name,
"-e".to_string(),
Expand Down
46 changes: 26 additions & 20 deletions src/commands/start/curio/verification.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,13 @@

use super::super::step::SetupContext;
use super::constants::TEST_FILE_SIZE_BYTES;
use crate::paths::foc_devnet_bin;
use rand::Rng;
use std::error::Error;
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::thread::sleep;
use std::time::Duration;
use tempfile::TempDir;
use tracing::info;

/// Verify a single Curio PDP SP is functioning correctly.
Expand Down Expand Up @@ -65,11 +63,15 @@
fn verify_upload_download(context: &SetupContext, sp_index: usize) -> Result<(), Box<dyn Error>> {
info!("Testing upload/download functionality via pdptool...");

// Create temporary directory for test files
let temp_dir = TempDir::new()?;
let test_file_path = create_random_test_file(&temp_dir)?;
// Create test file in curio's fast-storage (already mounted to container)
let run_id = context.run_id();
let curio_sp_dir = crate::paths::foc_devnet_curio_sp_volume(run_id, sp_index);
let test_file_dir = curio_sp_dir.join("fast-storage");
fs::create_dir_all(&test_file_dir)?;

// Upload file via pdptool
let test_file_path = create_random_test_file(&test_file_dir)?;

// Upload file via pdptool (running in container)
let piece_cid = upload_test_file(context, &test_file_path, sp_index)?;

// Wait a bit for the piece to be available for download
Expand All @@ -84,14 +86,17 @@
return Err("Downloaded data does not match original".into());
}

// Clean up test file
let _ = fs::remove_file(&test_file_path);

info!("Upload/download verified");

Ok(())
}

/// Create a random test file.
fn create_random_test_file(temp_dir: &TempDir) -> Result<PathBuf, Box<dyn Error>> {
let test_file_path = temp_dir.path().join("test_data.bin");
fn create_random_test_file(test_dir: &Path) -> Result<PathBuf, Box<dyn Error>> {
let test_file_path = test_dir.join("test_data.bin");
let mut rng = rand::thread_rng();
let random_data: Vec<u8> = (0..TEST_FILE_SIZE_BYTES).map(|_| rng.gen()).collect();

Expand All @@ -103,34 +108,35 @@
/// Upload test file using pdptool.
fn upload_test_file(
context: &SetupContext,
file_path: &Path,
_file_path: &Path,
sp_index: usize,
) -> Result<String, Box<dyn Error>> {
// Get dynamically allocated PDP port from context
let port: u16 = context
.get(&format!("curio_sp_{}_pdp_port", sp_index))
.ok_or("Curio PDP port not found in context")?
.parse()?;
// When running pdptool inside the container, use the container's internal port (4702)
// not the host-mapped port
let service_url = "http://localhost:4702";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not hardcode stuff. What's wrong with host-mapped port here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because pdptool was moved inside the container, so a fixed internal port was used.

The reason for moving pdptool internally is that the original logic allowed programs compiled inside the container to be used externally, which was not as expected and could cause problems in most environments.


let service_url = format!("http://localhost:{}", port);
// File is in fast-storage on host, which is mounted to /home/foc-user/curio/fast-storage in container
let container_file_path = "/home/foc-user/curio/fast-storage/test_data.bin";

let file_path_str = file_path
.to_str()
.ok_or("Invalid file path: contains non-UTF8 characters")?;
let run_id = context.run_id();
let container_name = format!("foc-{}-curio-{}", run_id, sp_index);

let args = [
"exec",
&container_name,
"/usr/local/bin/lotus-bins/pdptool",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure testing from within curio container constitutes "healthy" properly, especially given public. Would rather have foc-builder run pdptool for stricter testing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, I will put pdptool in foc-builder and keep testing service_url externally.

"upload-piece",
"--service-url",
&service_url,
"--service-name",
"public",
"--hash-type",
"commp",
file_path_str,
container_file_path,
"--verbose",

Check warning on line 136 in src/commands/start/curio/verification.rs

View workflow job for this annotation

GitHub Actions / fmt-clippy

Diff in /home/runner/work/foc-devnet/foc-devnet/src/commands/start/curio/verification.rs
];

let output = Command::new(foc_devnet_bin().join("pdptool"))
let output = Command::new("docker")
.args(args)
.output()?;

Expand Down
23 changes: 22 additions & 1 deletion src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -227,14 +227,35 @@ impl Default for Config {
url: "https://github.com/FilOzone/synapse-sdk.git".to_string(),
commit: "773551bf1e9cf4cdc49aeb63a47a81f8dc5cb9e1".to_string(),
},
yugabyte_download_url: "https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz".to_string(),
yugabyte_download_url: Self::get_default_yugabyte_url(),
approved_pdp_sp_count: 1,
active_pdp_sp_count: 1,
}
}
}

impl Config {
/// Get the default YugabyteDB download URL based on system architecture.
///
/// Returns the appropriate YugabyteDB tarball URL for the current platform:
/// - el8-aarch64 for ARM64 systems (Apple Silicon, AWS Graviton, etc.)
/// - linux-x86_64 for x86_64 systems
fn get_default_yugabyte_url() -> String {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This addition is a "feature update". The S3 fallback has to be changed accordingly in the code as well. Please check that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you mean by S3 fallback here? But I do see a place I forgot to update.

# Cache Yugabyte tarball

const YUGABYTE_VERSION: &str = "2.25.1.0";
const YUGABYTE_BUILD: &str = "b381";

let arch_suffix = if cfg!(target_arch = "aarch64") {
"el8-aarch64"
} else {
"linux-x86_64"
};

format!(
"https://software.yugabyte.com/releases/{}/yugabyte-{}-{}-{}.tar.gz",
YUGABYTE_VERSION, YUGABYTE_VERSION, YUGABYTE_BUILD, arch_suffix
)
}

/// Validate configuration values.
///
/// Ensures that:
Expand Down
Loading