Skip to content

Commit f840455

Browse files
karatkepoleksii-donets
karatkep
and
oleksii-donets
authored
feat: buildkit to replace kaniko (#106)
Co-authored-by: Oleksii Donets <73120709+oleksii-donets@users.noreply.github.com>
1 parent 62ec4a4 commit f840455

28 files changed

+645
-333
lines changed

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ For more information about the parent AI DIAL Admin Panel, visit the [ai-dial-ad
4444
## Features
4545

4646
- **MCP Deployment Management**: Complete lifecycle management for Model Context Protocol servers
47-
- **Container Image Building**: Automated image building with Kaniko in Kubernetes jobs
47+
- **Container Image Building**: Automated image building with [BuildKit](https://github.com/moby/buildkit) in Kubernetes jobs, supporting both root and rootless containers.
4848
- **Image Definition Management**: Support for multiple image definition types (MCP, DIAL Interceptor) with versioning
4949
- **Knative-Based Deployments**: Serverless container deployments with auto-scaling and automatic HTTPS endpoints
5050
- **Real-Time Status Updates**: Server-Sent Events (SSE) for real-time build status and deployment monitoring

‎docs/Development.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ docker-compose down
9292

9393
### Core Components
9494

95-
**Pipeline-Based Image Building**: The application uses a multi-stage pipeline pattern for building container images with Kaniko in Kubernetes jobs. Key pipeline steps include base image building, image analysis, and wrapper image creation. For STDIO MCP servers, the system creates wrapper images by inserting an HTTP-to-STDIO proxy executable into user-provided base images. The proxy executable is pre-built separately (using PyInstaller for standalone executables) and the system detects Linux distributions (Alpine vs Debian) to select the appropriate proxy executable.
95+
**Pipeline-Based Image Building**: The application uses a multi-stage pipeline pattern for building container images with Buildkit in Kubernetes jobs. Key pipeline steps include base image building, image analysis, and wrapper image creation. For STDIO MCP servers, the system creates wrapper images by inserting an HTTP-to-STDIO proxy executable into user-provided base images. The proxy executable is pre-built separately (using PyInstaller for standalone executables) and the system detects Linux distributions (Alpine vs Debian) to select the appropriate proxy executable.
9696

9797
**Disposable Resource Management**: Critical pattern for tracking and cleaning up Kubernetes resources. Resources have lifecycle states (TEMPORARY, STABLE, TO_CLEANUP) and are automatically cleaned up using scheduled jobs with ShedLock for distributed coordination. All Kubernetes resources are tracked by group ID for comprehensive cleanup.
9898

‎docs/configuration.md‎

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -100,18 +100,19 @@ app:
100100

101101
#### Build and Deployment Configuration
102102

103-
| Property | Environment Variable | Default Value | Required | Applied when | Description |
104-
|-----------------------------------------|-----------------------------------|-----------------------------------------|------------------------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
105-
| `app.build-namespace` | `K8S_BUILD_NAMESPACE` | `default` | No (recommended to adjust for target environment) | - | Kubernetes namespace for build operations |
106-
| `app.git-clone-image` | `GIT_CLONE_IMAGE` | `alpine/git:latest` | No (recommended to use a specific tag in production) | - | Docker image for Git cloning in init containers. Must include git and openssh-client for SSH support. |
107-
| `app.builder-image` | - | `gcr.io/kaniko-project/executor:latest` | No (recommended to use a specific tag in production) | - | Docker image for building containers |
108-
| `app.analyser-image` | - | `anchore/syft:latest` | No (recommended to use a specific tag in production) | - | Docker image used for analyzing container images |
109-
| `app.copy-image` | - | `quay.io/skopeo/stable:latest` | No (recommended to use a specific tag in production) | - | Docker image for copying images |
110-
| `app.docker-config-path` | - | `/kaniko/.docker/config.json` | No | - | Path to the location where the Docker config file is mounted for build containers. |
111-
| `app.cilium-network-policies-enabled` | `CILIUM_NETWORK_POLICIES_ENABLED` | `false` | No | - | Flag that allows to enable Cilium network policies for image build and deployments. |
112-
| `app.image-name-format` | `IMAGE_NAME_FORMAT` | `app-%s` | No | - | Name format for images that are built using Deployment Manager. Must contain `%s` that will be replaced by image definition ID. |
113-
| `app.resource-name-prefix` | `RESOURCE_NAME_PREFIX` | - | No | - | Prefix that will be added to all resources that image build and deployments produce. Important note: do not change this value on exising setups, otherwise existing images and K8s resources will be lost. |
114-
| `app.deployment.healthcheck-enabled` | `DEPLOYMENT_HEALTHCHECK_ENABLED` | `true` | No | - | Flag that allows to enable/disable deployment healthchecks |
103+
| Property | Environment Variable | Default Value | Required | Applied when | Description |
104+
|-----------------------------------------|-----------------------------------|-----------------------------------------|------------------------------------------------------|--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
105+
| `app.build-namespace` | `K8S_BUILD_NAMESPACE` | `default` | No (recommended to adjust for target environment) | - | Kubernetes namespace for build operations |
106+
| `app.git-clone-image` | `GIT_CLONE_IMAGE` | `alpine/git:latest` | No (recommended to use a specific tag in production) | - | Docker image for Git cloning in init containers. Must include git and openssh-client for SSH support. |
107+
| `app.builder-rootless-image` | - | `moby/buildkit:v0.27.1-rootless` | No (recommended to use a specific tag in production) | - | Buildkit rootless image for building containers. |
108+
| `app.builder-root-image` | - | `moby/buildkit:v0.27.1` | No (recommended to use a specific tag in production) | - | Buildkit image for building containers. The root user is used to cover cases where rootless limitations prevent image building, but the admin is confident in the reliability and security of the built image. |
109+
| `app.analyser-image` | - | `anchore/syft:latest` | No (recommended to use a specific tag in production) | - | Docker image used for analyzing container images |
110+
| `app.copy-image` | - | `quay.io/skopeo/stable:latest` | No (recommended to use a specific tag in production) | - | Docker image for copying images |
111+
| `app.docker-config-path` | - | `/kaniko/.docker/config.json` | No | - | Path to the location where the Docker config file is mounted for build containers. |
112+
| `app.cilium-network-policies-enabled` | `CILIUM_NETWORK_POLICIES_ENABLED` | `false` | No | - | Flag that allows to enable Cilium network policies for image build and deployments. |
113+
| `app.image-name-format` | `IMAGE_NAME_FORMAT` | `app-%s` | No | - | Name format for images that are built using Deployment Manager. Must contain `%s` that will be replaced by image definition ID. |
114+
| `app.resource-name-prefix` | `RESOURCE_NAME_PREFIX` | - | No | - | Prefix that will be added to all resources that image build and deployments produce. Important note: do not change this value on exising setups, otherwise existing images and K8s resources will be lost. |
115+
| `app.deployment.healthcheck-enabled` | `DEPLOYMENT_HEALTHCHECK_ENABLED` | `true` | No | - | Flag that allows to enable/disable deployment healthchecks |
115116

116117
#### MCP Proxy Configuration
117118

‎docs/diagramms/infrastructure_component/infrastrucutre-component-diagramm.drawio‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
<mxfile host="app.diagrams.net" agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36" version="29.2.4">
1+
<mxfile host="app.diagrams.net" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36" version="29.2.7">
22
<diagram name="DIAL Infrastructure" id="dial-infrastructure">
3-
<mxGraphModel grid="1" page="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" pageScale="1" pageWidth="1400" pageHeight="1000" math="0" shadow="0">
3+
<mxGraphModel dx="1270" dy="758" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1400" pageHeight="1000" math="0" shadow="0">
44
<root>
55
<mxCell id="0" />
66
<mxCell id="1" parent="0" />
@@ -34,7 +34,7 @@
3434
<mxCell id="ns-build" parent="k8s-cluster" style="swimlane;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#000000;" value="Namespace: K8S_BUILD_NAMESPACE (DIAL deployments Image Building)" vertex="1">
3535
<mxGeometry height="140" width="590" x="200" y="50" as="geometry" />
3636
</mxCell>
37-
<mxCell id="kaniko" parent="ns-build" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1;" value="Container Image Builder (Kaniko)" vertex="1">
37+
<mxCell id="buildkit" parent="ns-build" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1;" value="Container Image Builder (Buildkit)" vertex="1">
3838
<mxGeometry height="60" width="140" x="180" y="50" as="geometry" />
3939
</mxCell>
4040
<mxCell id="ns-knative" parent="k8s-cluster" style="swimlane;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#000000;" value="Namespace: K8S_KNATIVE_DEPLOYMENT_NAMESPACE (DIAL Deployments Workloads)" vertex="1">
@@ -118,22 +118,22 @@
118118
<mxPoint x="380" y="530" as="targetPoint" />
119119
</mxGeometry>
120120
</mxCell>
121-
<mxCell id="rel-vcs-kaniko" edge="1" parent="1" source="vcs" style="endArrow=classic;html=1;entryX=1;entryY=0.25;entryDx=0;entryDy=0;exitX=0.513;exitY=0.963;exitDx=0;exitDy=0;strokeWidth=2;strokeColor=#9673a6;exitPerimeter=0;" target="kaniko" value="Source Code">
121+
<mxCell id="rel-vcs-buildkit" edge="1" parent="1" source="vcs" style="endArrow=classic;html=1;entryX=1;entryY=0.25;entryDx=0;entryDy=0;exitX=0.513;exitY=0.963;exitDx=0;exitDy=0;strokeWidth=2;strokeColor=#9673a6;exitPerimeter=0;" target="buildkit" value="Source Code">
122122
<mxGeometry height="50" relative="1" width="50" x="0.3774" y="-5" as="geometry">
123123
<mxPoint as="offset" />
124124
<mxPoint x="570" y="130" as="sourcePoint" />
125125
<mxPoint x="530" y="250" as="targetPoint" />
126126
</mxGeometry>
127127
</mxCell>
128-
<mxCell id="rel-net-kaniko" edge="1" parent="1" source="internet" style="endArrow=classic;html=1;exitX=0.07;exitY=0.4;exitDx=0;exitDy=0;strokeWidth=2;strokeColor=#9673a6;entryX=1;entryY=0.5;entryDx=0;entryDy=0;exitPerimeter=0;" target="kaniko" value="Dependencies&#xa;(PyPI/Maven/etc.)">
128+
<mxCell id="rel-net-buildkit" edge="1" parent="1" source="internet" style="endArrow=classic;html=1;exitX=0.07;exitY=0.4;exitDx=0;exitDy=0;strokeWidth=2;strokeColor=#9673a6;entryX=1;entryY=0.5;entryDx=0;entryDy=0;exitPerimeter=0;" target="buildkit" value="Dependencies&#xa;(PyPI/Maven/etc.)">
129129
<mxGeometry height="50" relative="1" width="50" x="0.212" y="12" as="geometry">
130130
<mxPoint as="offset" />
131131
<Array as="points" />
132132
<mxPoint x="1010" y="130" as="sourcePoint" />
133133
<mxPoint x="910" y="300" as="targetPoint" />
134134
</mxGeometry>
135135
</mxCell>
136-
<mxCell id="rel-kaniko-registry" edge="1" parent="1" style="endArrow=classic;html=1;entryX=0;entryY=0;entryDx=0;entryDy=52.5;strokeWidth=2;strokeColor=#9673a6;entryPerimeter=0;" target="registry" value="Push Built Image">
136+
<mxCell id="rel-buildkit-registry" edge="1" parent="1" style="endArrow=classic;html=1;entryX=0;entryY=0;entryDx=0;entryDy=52.5;strokeWidth=2;strokeColor=#9673a6;entryPerimeter=0;" target="registry" value="Push Built Image">
137137
<mxGeometry height="50" relative="1" width="50" x="-0.0866" y="-2" as="geometry">
138138
<mxPoint as="offset" />
139139
<mxPoint x="982" y="340" as="sourcePoint" />

‎docs/diagramms/infrastructure_component/infrastrucutre-component-diagramm.png‎

71.7 KB
Loading

‎src/main/java/com/epam/aidial/deployment/manager/configuration/AppProperties.java‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@ public class AppProperties {
2626

2727
private Job builderJobConfig;
2828
private Container initBuilderContainerConfig;
29-
private Container builderContainerConfig;
29+
private Container builderRootContainerConfig;
30+
private Container builderRootlessContainerConfig;
3031
private Container pushContainerConfig;
3132
private Secret builderSecretConfig;
3233

@@ -65,8 +66,12 @@ public Job cloneBuilderJobConfig() {
6566
return new JobBuilder(builderJobConfig).build();
6667
}
6768

68-
public Container cloneBuilderContainerConfig() {
69-
return new ContainerBuilder(builderContainerConfig).build();
69+
public Container cloneBuilderRootContainerConfig() {
70+
return new ContainerBuilder(builderRootContainerConfig).build();
71+
}
72+
73+
public Container cloneBuilderRootlessContainerConfig() {
74+
return new ContainerBuilder(builderRootlessContainerConfig).build();
7075
}
7176

7277
public Container clonePushContainerConfig() {

‎src/main/java/com/epam/aidial/deployment/manager/dao/entity/ImageDefinitionEntity.java‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,4 +83,8 @@ public class ImageDefinitionEntity {
8383
@JdbcTypeCode(SqlTypes.JSON)
8484
@Column(name = "allowed_domains")
8585
private List<String> allowedDomains;
86+
87+
@Column(name = "image_builder")
88+
@Enumerated(value = EnumType.STRING)
89+
private PersistenceImageBuilder imageBuilder;
8690
}

‎src/main/java/com/epam/aidial/deployment/manager/dao/entity/PersistenceImageBuilder.java‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
package com.epam.aidial.deployment.manager.dao.entity;
2+
3+
public enum PersistenceImageBuilder {
4+
BUILDKIT,
5+
BUILDKIT_ROOTLESS
6+
}

‎src/main/java/com/epam/aidial/deployment/manager/model/ImageBuilder.java‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
package com.epam.aidial.deployment.manager.model;
2+
3+
public enum ImageBuilder {
4+
BUILDKIT,
5+
BUILDKIT_ROOTLESS
6+
}

‎src/main/java/com/epam/aidial/deployment/manager/model/ImageDefinition.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,4 +35,5 @@ public abstract class ImageDefinition {
3535
private String author;
3636

3737
private List<String> allowedDomains;
38+
private ImageBuilder imageBuilder;
3839
}

0 commit comments

Comments
 (0)