chore: use pnpm for builds #19752

midzelis · 2025-07-05T19:36:38Z

No description provided.

github-actions · 2025-07-05T19:40:30Z

📖 Documentation deployed to pr-19752.preview.immich.app

Conflict Missing dependency gitignore Cli dockerfile Exiftool optional fix Update pnpm version update devcontainer for pnpm

server/Dockerfile

docker/docker-compose.dev.yml

web/Dockerfile

open-api/bin/generate-open-api.sh

mertalev

LGTM, but I think it should get more eyes before merge

.github/workflows/cli.yml

.github/workflows/docs-build.yml

.github/workflows/test.yml

danieldietzler · 2025-08-09T15:55:02Z

docker/docker-compose.dev.yml

+      - pnpm-store:/usr/src/app/.pnpm-store
+      - server-node-modules:/usr/src/app/server/node_modules
+      - github-node_modules:/usr/src/app/.github/node_modules
+      - cli-node_modules:/usr/src/app/cli/node_modules
+      - docs-node_modules:/usr/src/app/docs/node_modules
+      - e2e-node_modules:/usr/src/app/e2e/node_modules
+      - sdk-node_modules:/usr/src/app/open-api/typescript-sdk/node_modules
+      - app-node_modules:/usr/src/app/node_modules
+      - web-node_modules:/usr/src/app/web/node_modules
+      - sveltekit:/usr/src/app/web/.svelte-kit
+      - coverage:/usr/src/app/web/coverage


Why are we moving all those to named docker volumes?

A few reasons:

This is to avoid permissions issues between the view of the files inside the container vs the ones on the bind-mount on the host. If the container were to be run by a user other than the user that owns the root of the git-clone on the host, then the files would be created with the owner of the user running the container, and if that happened to be 'root' then the files can't be cleaned up with a simple make clean command - it would require sudo make clean which is a little more dangerous/annoying.

Isolation: All of dependency/package install commands are performed from within a docker container, which may be a different architecture (MacOS on arm) or libc (the web container uses alpine with musl instead of glibc) - and some of the dependent packages use native binary code, which only make sense in the architecture/libc version that is within the container it was executed inside. By using volumes, these dependencies are not created on the hosts filesystem.

pnpm performs performs a clone, hard-link, or copy of files (in that order) from the content-addressable-store (/buildcache/pnpm-store) to the virtual store (.pnpm-store). And then creates the dependency node structures in node_modules using symlinks. Using volume mounts makes it more likely that hardlinks will be successful, increasing speed to install dependencies.

oh, and named volumes vs anonymous volumes because anonymous volumes are removed when container is removed, and containers are auto-removed at the end of every make command.

Soooo, generally a nice change but kind of unrelated to this PR?

danieldietzler · 2025-08-09T15:57:28Z

server/src/constants.ts

+// eslint-disable-next-line unicorn/prefer-module
+const basePath = dirname(__filename);
+const packageFile = join(basePath, '..', 'package.json');
+const { version } = JSON.parse(readFileSync(packageFile, 'utf8'));


Why this change and why is the prefer module linting error here?

I think because we used to get the version from the package.json file and that doesn't exist in the server directory anymore.

Oh nevermind, I think it originally was removed, but it is still there in this version of the PR.

Note: package.json still exists in each of the 8 workspace modules contained in this monorepo.

However, reading it like this:

const { version } = JSON.parse(readFileSync('./package.json', 'utf8'));

uses a relative path, which is resolved against the current-working-directory, which isn't /usr/src/app/server any longer. The way to fix this is read the file from the absolute path of the server package.json. This is determined by using the CommonJS style __filename global, mostly because the ESM alternative is more verbose, and -- well, the default output of the server is CommonJS anyways, not ESM, so using import.meta.url would end up requiring even more steps to adapt back to CommonJS.

What is the CWD now?

This ends up being the WORKDIR in the various Dockerfile stages.

For production images, this is /usr/src/app (but server is at /usr/src/app/server)
For server dev image, this is /usr/src/app
For web dev image, this is /usr/src/app/web
For devcontainer images, this is /workspaces/immich

I see. Fair enough, thanks for explaining

danieldietzler · 2025-08-09T15:58:39Z

web/src/service-worker/fetch-event.ts

Is this an unrelated change?

No - the format of the the paths of individual dependencies served from node_modules when running in dev mode has changed, and this adapts the regexps to the pnpm's changes to these paths.

danieldietzler · 2025-08-09T16:01:45Z

web/vite.config.ts

@@ -60,3 +65,5 @@ export default defineConfig({
    },
  },
 });
+
+export default mergeConfig(viteConfig, vitestConfig);


Why? AFAICT using the defineConfig from vite is perfectly fine https://vitest.dev/config/#configuring-vitest

I'm not sure why (maybe because pnpm is more strict?), but I was not able to specify the 'test' property within a defineConfig from vite. I was able to specify test when importing defineConfig from vitest/config - however, then there was an type-error on the plugins property in the config. The one way that worked was what you see here - import both the vitest config and the vite config, then merge them. Also, the triple slash command in the docs did not help.

In fact, I looked at all the 'solutions' here: https://stackoverflow.com/questions/72146352/vitest-defineconfig-test-does-not-exist-in-type-userconfigexport and only the mergeConfig technique (one of the alternatives in the comments) was the only one that worked for me.

Update: I found another solution that just needs a new type import that is a little cleaner

That looks good to me, thanks!

.pnpmfile.cjs

package.json

danieldietzler · 2025-08-10T10:16:27Z

.github/workflows/fix-format.yml

-          node-version-file: './server/.nvmrc'
-          cache: 'npm'
-          cache-dependency-path: '**/package-lock.json'
+          node-version-file: './.github/.nvmrc'


Isn't this still wrong?

Looking more carefully, this is actually formatting all of the projects, not just .github. You'd technically have to run a version of setup-node for each project. This all may be a moot point, since the node version is always the same for every project.

Would it make sense to just treat the node version as a workspace level setting, instead of per-project? Then, the nvmrc file could be moved up to the root directory of the git repo.

I'll revert to just using 'server/.nvmrc' as before for now

danieldietzler

Nice! Excited for pnpm

danieldietzler · 2025-08-10T16:53:15Z

docker/docker-compose.dev.yml

+  pnpm-store:
+  github-node_modules:
+  server-node-modules:
+  cli-node_modules:
+  docs-node_modules:
+  e2e-node_modules:
+  sdk-node_modules:
+  app-node_modules:
+  web-node_modules:
+  sveltekit:
+  coverage:


Those aren't used anymore are they?

midzelis requested a review from danieldietzler as a code owner July 5, 2025 19:36

midzelis added the changelog:skip label Jul 5, 2025

midzelis requested a review from bo0tzz as a code owner July 5, 2025 19:36

midzelis mentioned this pull request Jul 5, 2025

chore: use pnpm #19570

Closed

midzelis force-pushed the chore/docker_permissions branch 2 times, most recently from 97f7e54 to 91ab806 Compare July 9, 2025 22:14

midzelis force-pushed the chore/pnpm_stacked_squashed branch from 2baf58c to b9f1b85 Compare July 9, 2025 22:39

github-actions bot added documentation Improvements or additions to documentation 📱mobile 🖥️web 🗄️server labels Jul 9, 2025

midzelis force-pushed the chore/pnpm_stacked_squashed branch 5 times, most recently from 80c0bab to 5367a86 Compare July 10, 2025 01:43

midzelis force-pushed the chore/docker_permissions branch from 91ab806 to 351266e Compare July 10, 2025 22:47

midzelis force-pushed the chore/pnpm_stacked_squashed branch from 5367a86 to 3262c4e Compare July 10, 2025 22:48

midzelis force-pushed the chore/docker_permissions branch from 351266e to 89c84cc Compare July 11, 2025 01:12

midzelis force-pushed the chore/pnpm_stacked_squashed branch 5 times, most recently from 200431d to 08865cc Compare July 11, 2025 01:42

midzelis force-pushed the chore/docker_permissions branch from 89c84cc to 3ab0dbb Compare July 11, 2025 01:50

midzelis force-pushed the chore/pnpm_stacked_squashed branch 4 times, most recently from 0d4a6fc to 6d87af1 Compare July 11, 2025 02:11

midzelis force-pushed the chore/pnpm_stacked_squashed branch 6 times, most recently from 61efa75 to 9f1fdb3 Compare August 8, 2025 03:12

chore: use pnpm for builds

687e99c

Conflict Missing dependency gitignore Cli dockerfile Exiftool optional fix Update pnpm version update devcontainer for pnpm

midzelis force-pushed the chore/pnpm_stacked_squashed branch from 9f1fdb3 to 687e99c Compare August 8, 2025 03:36

mertalev reviewed Aug 8, 2025

View reviewed changes

server/Dockerfile Show resolved Hide resolved

mertalev reviewed Aug 8, 2025

View reviewed changes

docker/docker-compose.dev.yml Outdated Show resolved Hide resolved

web/Dockerfile Outdated Show resolved Hide resolved

open-api/bin/generate-open-api.sh Outdated Show resolved Hide resolved

midzelis force-pushed the chore/pnpm_stacked_squashed branch from bb08f8a to e288f6b Compare August 8, 2025 22:01

Review comments, minor tweaks

6c81d08

midzelis force-pushed the chore/pnpm_stacked_squashed branch from e288f6b to 6c81d08 Compare August 8, 2025 22:28

midzelis requested a review from mertalev August 8, 2025 23:49

mertalev approved these changes Aug 8, 2025

View reviewed changes

midzelis requested a review from jrasm91 August 9, 2025 13:11

danieldietzler reviewed Aug 9, 2025

View reviewed changes

midzelis force-pushed the chore/pnpm_stacked_squashed branch 3 times, most recently from 2975ee8 to c302fcf Compare August 10, 2025 04:09

danieldietzler reviewed Aug 10, 2025

View reviewed changes

midzelis added 2 commits August 10, 2025 14:57

Another round of review comments

c1306bb

Use 'server/.nvmrc' for fix-format.yml GHA

c7ac088

midzelis force-pushed the chore/pnpm_stacked_squashed branch from c302fcf to c7ac088 Compare August 10, 2025 15:06

midzelis added 3 commits August 10, 2025 15:39

Delete npm locks

d306cb5

Fix devcontainer for Debian Trixie until base images are updated

8125c74

Don't isolate node_modules between host/container for now

5bc6a15

midzelis force-pushed the chore/pnpm_stacked_squashed branch from eb1b85c to 5bc6a15 Compare August 10, 2025 16:32

danieldietzler approved these changes Aug 10, 2025

View reviewed changes

unused named containers

20dffb0

Uh oh!

chore: use pnpm for builds #19752

Are you sure you want to change the base?

chore: use pnpm for builds #19752

Uh oh!

Conversation

midzelis commented Jul 5, 2025

Uh oh!

github-actions bot commented Jul 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mertalev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

midzelis Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danieldietzler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jul 5, 2025 •

edited

Loading

midzelis Aug 10, 2025 •

edited

Loading