|
| 1 | +# Dockerfile conversion to LLB |
| 2 | + |
| 3 | +If you want to understand how Buildkit translates Dockerfile instructions into |
| 4 | +LLB, or you want to write your own frontend, then seeing how Dockerfile maps to |
| 5 | +using the Buildkit LLB package will give you a jump start. |
| 6 | + |
| 7 | +The `llb` package from Buildkit provides a chainable state object to help |
| 8 | +construct a LLB. Then you can marshal the state object into a definition using |
| 9 | +protocol buffers, and send it off in a solve request over gRPC. |
| 10 | + |
| 11 | +In code, these transformations are performed by the [`Dockerfile2LLB()`](../../frontend/dockerfile/dockerfile2llb/convert.go) |
| 12 | +function, which takes a raw `Dockerfile`'s contents and converts it to an LLB |
| 13 | +state, and associated image config, which are then both assembled in the |
| 14 | +[`Build()`](../../frontend/dockerfile/builder/build.go) function. |
| 15 | + |
| 16 | +## Basic examples |
| 17 | + |
| 18 | +Here are a few Dockerfile instructions you should be familiar with: |
| 19 | + |
| 20 | +- Base image |
| 21 | + |
| 22 | + ```dockerfile |
| 23 | + FROM golang:1.12 |
| 24 | + ``` |
| 25 | + |
| 26 | + ```golang |
| 27 | + st := llb.Image("golang:1.12") |
| 28 | + ``` |
| 29 | + |
| 30 | +- Scratch image |
| 31 | + |
| 32 | + ```dockerfile |
| 33 | + FROM scratch |
| 34 | + ``` |
| 35 | + |
| 36 | + ```golang |
| 37 | + st := llb.Scratch() |
| 38 | + ``` |
| 39 | + |
| 40 | +- Environment variables |
| 41 | + |
| 42 | + ```dockerfile |
| 43 | + ENV DEBIAN_FRONTEND=noninteractive |
| 44 | + ``` |
| 45 | + |
| 46 | + ```golang |
| 47 | + st = st.AddEnv("DEBIAN_FRONTEND", "noninteractive") |
| 48 | + ``` |
| 49 | + |
| 50 | +- Running programs |
| 51 | + |
| 52 | + ```dockerfile |
| 53 | + RUN echo hello |
| 54 | + ``` |
| 55 | + |
| 56 | + ```golang |
| 57 | + st = st.Run( |
| 58 | + llb.Shlex("echo hello"), |
| 59 | + ).Root() |
| 60 | + ``` |
| 61 | + |
| 62 | +- Working directory |
| 63 | + |
| 64 | + ```dockerfile |
| 65 | + WORKDIR /path |
| 66 | + ``` |
| 67 | + |
| 68 | + ```golang |
| 69 | + st = st.Dir("/path") |
| 70 | + ``` |
| 71 | + |
| 72 | +## File operations |
| 73 | + |
| 74 | +This is where LLB starts to deviate from Dockerfile in features. In |
| 75 | +Dockerfiles, the run command is completely opaque to the builder and just |
| 76 | +executes the command. But in LLB, there are file operations that have better |
| 77 | +caching semantics and understanding of the command: |
| 78 | + |
| 79 | +- Copying files |
| 80 | + |
| 81 | + ```dockerfile |
| 82 | + COPY --from=builder /files/* /files |
| 83 | + ``` |
| 84 | + |
| 85 | + ```golang |
| 86 | + var CopyOptions = &llb.CopyInfo{ |
| 87 | + FollowSymlinks: true, |
| 88 | + CopyDirContentsOnly: true, |
| 89 | + AttemptUnpack: false, |
| 90 | + CreateDestPath: true, |
| 91 | + AllowWildcard: true, |
| 92 | + AllowEmptyWildcard: true, |
| 93 | + } |
| 94 | + st = st.File( |
| 95 | + llb.Copy(builder, "/files/*", "/files", CopyOptions), |
| 96 | + ) |
| 97 | + ``` |
| 98 | + |
| 99 | +- Adding files |
| 100 | + |
| 101 | + ```dockerfile |
| 102 | + ADD --from=builder /files.tgz /files |
| 103 | + ``` |
| 104 | + |
| 105 | + ```golang |
| 106 | + var AddOptions = &llb.CopyInfo{ |
| 107 | + FollowSymlinks: true, |
| 108 | + CopyDirContentsOnly: true, |
| 109 | + AttemptUnpack: true, |
| 110 | + CreateDestPath: true, |
| 111 | + AllowWildcard: true, |
| 112 | + AllowEmptyWildcard: true, |
| 113 | + } |
| 114 | + st = st.File( |
| 115 | + llb.Copy(builder, "/files.tgz", "files", AddOptions), |
| 116 | + ) |
| 117 | + ``` |
| 118 | + |
| 119 | +- Chaining file commands |
| 120 | + |
| 121 | + ```dockerfile |
| 122 | + # not possible without RUN in Dockerfile |
| 123 | + RUN mkdir -p /some && echo hello > /some/file |
| 124 | + ``` |
| 125 | + |
| 126 | + ```golang |
| 127 | + st = st.File( |
| 128 | + llb.Mkdir("/some", 0755), |
| 129 | + ).File( |
| 130 | + llb.Mkfile("/some/file", 0644, "hello"), |
| 131 | + ) |
| 132 | + ``` |
| 133 | + |
| 134 | +## Bind mounts |
| 135 | + |
| 136 | +Bind mounts allow unidirectional syncing of the host's local file system into |
| 137 | +the build environment. |
| 138 | + |
| 139 | +Bind mounts in Buildkit should not be confused with bind mounts in the linux |
| 140 | +kernel - they do not sync bidirectionally. Bind mounts are only a snapshot of |
| 141 | +your local state, which is specified through the `llb.Local` state object: |
| 142 | + |
| 143 | +- Using bind mounts |
| 144 | + |
| 145 | + ```dockerfile |
| 146 | + WORKDIR /builder |
| 147 | + RUN --mount=type=bind,target=/builder \ |
| 148 | + PIP_INDEX_URL=https://my-proxy.com/pypi \ |
| 149 | + pip install . |
| 150 | + ``` |
| 151 | + |
| 152 | + ```golang |
| 153 | + localState := llb.Local( |
| 154 | + "context", |
| 155 | + llb.SessionID(client.BuildOpts().SessionID), |
| 156 | + llb.WithCustomName("loading .") |
| 157 | + llb.FollowPaths([]string{"."}), |
| 158 | + ) |
| 159 | + |
| 160 | + execState = st.Dir("/builder").Run( |
| 161 | + llb.Shlex("pip install ."), |
| 162 | + llb.AddEnv( |
| 163 | + "PIP_INDEX_URL", |
| 164 | + "https://my-proxy.com/pypi", |
| 165 | + ), |
| 166 | + ) |
| 167 | + _ := execState.AddMount("/builder", localState) |
| 168 | + // the return value of AddMount captures the resulting state of the mount |
| 169 | + // after the exec operation has completed |
| 170 | + |
| 171 | + st := execState.Root() |
| 172 | + ``` |
| 173 | + |
| 174 | +## Cache mounts |
| 175 | + |
| 176 | +Cache mounts allow for a shared file cache location between build invocations, |
| 177 | +which allow manually caching expensive operations, such as package downloads. |
| 178 | +Mounts have options to persist between builds with different sharing modes. |
| 179 | + |
| 180 | +- Using cache mounts |
| 181 | + |
| 182 | + ```dockerfile |
| 183 | + RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ |
| 184 | + --mount=type=cache,target=/var/lib/apt \ |
| 185 | + apt-get update |
| 186 | + ``` |
| 187 | + |
| 188 | + ```golang |
| 189 | + var VarCacheAptMount = llb.AddMount( |
| 190 | + "/var/cache/apt", |
| 191 | + llb.Scratch(), |
| 192 | + llb.AsPersistentCacheDir( |
| 193 | + "some-cache-id", |
| 194 | + llb.CacheMountLocked, |
| 195 | + ), |
| 196 | + ) |
| 197 | + |
| 198 | + var VarLibAptMount = llb.AddMount( |
| 199 | + "/var/lib/apt", |
| 200 | + llb.Scratch(), |
| 201 | + llb.AsPersistentCacheDir( |
| 202 | + "another-cache-id", |
| 203 | + llb.CacheMountShared, |
| 204 | + ), |
| 205 | + ) |
| 206 | + |
| 207 | + st := st.Run( |
| 208 | + llb.Shlex("apt-get update"), |
| 209 | + VarCacheAptMount, |
| 210 | + VarLibAptMount, |
| 211 | + ).Root() |
| 212 | + ``` |
0 commit comments