Skip to content

Commit b4eca25

Browse files
vsochvsoch
authored
Merge pull request #10 from compspec/add-agent
wip: adding build agent to fractale
2 parents 1c90468 + 47a6385 commit b4eca25

File tree

15 files changed

+1022
-5
lines changed

15 files changed

+1022
-5
lines changed

‎examples/agent/Dockerfile

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# Dockerfile for LAMMPS (Large-scale Atomic/Molecular Massively Parallel Simulator)
2+
# Target: Production HPC environment on Google Cloud
3+
# Strategy: Multi-stage build for a lean final image with MPI support.
4+
5+
# Use ARGs at the top to easily update versions of key components globally
6+
ARG LAMMPS_VERSION=stable_2Aug2023
7+
ARG OPENMPI_VERSION=4.1.6
8+
9+
# =====================================================================
10+
# Stage 1: Builder
11+
# This stage compiles Open MPI and LAMMPS from source. It contains all
12+
# the build-time dependencies, which will be discarded later.
13+
# =====================================================================
14+
FROM debian:bullseye AS builder
15+
16+
# Inherit ARGs from the global scope
17+
ARG LAMMPS_VERSION
18+
ARG OPENMPI_VERSION
19+
20+
# Set environment variables for the Open MPI build location and path
21+
ENV OMPI_DIR=/opt/openmpi-${OPENMPI_VERSION}
22+
ENV PATH=$OMPI_DIR/bin:$PATH
23+
ENV LD_LIBRARY_PATH=$OMPI_DIR/lib
24+
25+
# Prevent interactive prompts during package installation
26+
ENV DEBIAN_FRONTEND=noninteractive
27+
28+
# Install essential build tools and libraries for both Open MPI and LAMMPS
29+
# Added ca-certificates to allow git and wget to verify SSL certificates securely.
30+
RUN apt-get update && apt-get install -y --no-install-recommends \
31+
build-essential \
32+
ca-certificates \
33+
cmake \
34+
g++ \
35+
gfortran \
36+
git \
37+
libevent-dev \
38+
libhwloc-dev \
39+
wget \
40+
&& rm -rf /var/lib/apt/lists/*
41+
42+
# --- Build Open MPI from source ---
43+
# Building from source gives control over the configuration, crucial for
44+
# containerized HPC environments. We enable PMIx for modern process management.
45+
WORKDIR /tmp
46+
RUN wget https://download.open-mpi.org/release/open-mpi/v${OPENMPI_VERSION%.*}/openmpi-${OPENMPI_VERSION}.tar.gz && \
47+
tar -xzf openmpi-${OPENMPI_VERSION}.tar.gz
48+
49+
WORKDIR /tmp/openmpi-${OPENMPI_VERSION}
50+
RUN ./configure \
51+
--prefix=${OMPI_DIR} \
52+
--with-pmix \
53+
--disable-pty-support
54+
RUN make -j$(nproc) all && make install
55+
56+
# --- Build LAMMPS from source ---
57+
# Clone a specific stable release tag for reproducibility.
58+
WORKDIR /opt
59+
RUN git clone --depth 1 --branch ${LAMMPS_VERSION} https://github.com/lammps/lammps.git lammps
60+
61+
# Use CMake to configure the LAMMPS build. Enable common packages.
62+
WORKDIR /opt/lammps/build
63+
RUN cmake ../cmake \
64+
-D CMAKE_INSTALL_PREFIX=/usr/local \
65+
-D BUILD_MPI=yes \
66+
-D PKG_KSPACE=yes \
67+
-D PKG_MOLECULE=yes \
68+
-D PKG_RIGID=yes \
69+
-D PKG_MANYBODY=yes \
70+
-D PKG_REPLICA=yes \
71+
-D CMAKE_BUILD_TYPE=Release \
72+
-D LAMMPS_EXCEPTIONS=yes
73+
74+
# Compile and install LAMMPS
75+
RUN make -j$(nproc) && make install
76+
77+
# =====================================================================
78+
# Stage 2: Final Image
79+
# This stage creates the lean, final image. It starts from a minimal
80+
# base and only copies the necessary executables, libraries, and runtime
81+
# dependencies from the builder stage.
82+
# =====================================================================
83+
FROM debian:bullseye-slim
84+
85+
# Inherit ARG for version consistency
86+
ARG OPENMPI_VERSION
87+
88+
# Set environment variables for Open MPI runtime
89+
ENV OMPI_DIR=/opt/openmpi-${OPENMPI_VERSION}
90+
ENV PATH=/usr/local/bin:$OMPI_DIR/bin:$PATH
91+
92+
# Install only the essential runtime dependencies.
93+
# libgfortran5 is required by the Fortran-compiled parts of LAMMPS.
94+
RUN apt-get update && apt-get install -y --no-install-recommends \
95+
libevent-2.1-7 \
96+
libgfortran5 \
97+
libhwloc15 \
98+
&& rm -rf /var/lib/apt/lists/*
99+
100+
# Copy the compiled Open MPI installation from the builder stage
101+
COPY --from=builder ${OMPI_DIR} ${OMPI_DIR}
102+
103+
# Copy the entire LAMMPS installation (binary, libs, potentials) from the builder stage
104+
COPY --from=builder /usr/local /usr/local
105+
106+
# Configure the dynamic linker to find Open MPI and LAMMPS libraries.
107+
# This is more robust than setting LD_LIBRARY_PATH.
108+
RUN echo "${OMPI_DIR}/lib" > /etc/ld.so.conf.d/openmpi.conf && \
109+
echo "/usr/local/lib" > /etc/ld.so.conf.d/lammps.conf && \
110+
ldconfig
111+
112+
# Create a dedicated, non-root user for running the application for security
113+
RUN useradd --create-home --shell /bin/bash lammps
114+
USER lammps
115+
WORKDIR /home/lammps
116+
117+
# Set the entrypoint to the LAMMPS executable.
118+
# Allows running the container with LAMMPS args directly, e.g., `docker run <image> -in in.lj`
119+
ENTRYPOINT ["lmp"]
120+
121+
# Provide a default command to display help if no other args are provided.
122+
CMD ["-h"]
123+
# Generated by fractale build agent

‎examples/agent/README.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Agents
2+
3+
Let's use fractale to run build, execute, and deploy agents. Right now we will run these a-la-carte, and eventually we will group them together to loop back and repeat steps if needed.
4+
5+
## Build
6+
7+
The build agent will use the Gemini API to generate a Dockerfile and then build until it succeeds. We would need subsequent agents to test it.
8+
Here is how to first ask the build agent to generate a lammps container for Google cloud.
9+
10+
```bash
11+
fractale agent build lammps --environment "google cloud" --outfile dockerfile
12+
```
13+
14+
That might generate the [Dockerfile](Dockerfile) here, and a container that defaults to the application name "lammps"
15+
16+
## Kubernetes Job
17+
18+
The kubernetes job agent agent will be asked to run a command, and will be provided the Dockerfile and name of the container. We assume that another agent (or you) have built and either pushed the image to a registry, or loaded it. Let's create our cluster and load the image:
19+
20+
```bash
21+
kind create cluster
22+
kind load docker-image lammps
23+
```
24+
25+
To start, we will assume a kind cluster running and tell the agent the image is loaded into it (and so the pull policy will be never).
26+
27+
```bash
28+
fractale agent kubernetes-job lammps --environment "google cloud CPU" --context-file ./Dockerfile --no-pull
29+
```
30+

‎fractale/agent/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
from fractale.agent.build import BuildAgent
2+
from fractale.agent.kubernetes_job import KubernetesJobAgent
3+
4+
5+
def get_agents():
6+
return {"build": BuildAgent, "kubernetes-job": KubernetesJobAgent}

‎fractale/agent/base.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
class Agent:
2+
"""
3+
A base for an agent
4+
"""
5+
6+
# name and description should be on the class
7+
8+
def add_arguments(self, subparser):
9+
"""
10+
Add arguments for the agent to show up in argparse
11+
12+
This is added by the plugin class
13+
"""
14+
pass
15+
16+
def run(self, args, extra):
17+
"""
18+
Run the agent.
19+
"""
20+
raise NotImplementedError(f"The {self.name} agent is missing a 'run' function")

‎fractale/agent/build/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
from .agent import BuildAgent

0 commit comments

Comments
 (0)