Add debian packaging helpers

[prasannavl]

Context

• Currently, there's no way to quickly generate a debian package for all packages in one-go for quick development.
• This PR adds a script that builds all the packages given that the dependencies are already there and installed in the system.
• Note that this does NOT add the project chain of dependencies and will never be a replacement for a proper debian package build that iterates through each of the packages and builds all of them as individual packages.
• However, this makes it much much easier on development machines or for people who wish to build from git and already have all the dependencies quickly build a single package that includes all and quickly deploy it without having to use systemd sysext (which requires parts of the fs tree to be immutable) or having to do a manual install.

Example usage

./scripts/pkg-deb-gitdev.sh
dpkg -i ./_build/<pkg>.deb

That's it. Just simply running the script will automatically run just build and build a single debian package in the _build dir that's ready to be installed with dpkg.

[mmstick]

Wouldn't it be better to run dpkg-buildpackage -b (or just install) on the specific project you're developing and testing?

[prasannavl]

@mmstick - I added the other script I use for building all packages individually in one-go as well. It auto discovers every debian subproject and builds them with dpkg-buildpackage while also ensuring their deps are available to build it.

Coming to the reason for a single deb:

• The dpkg-build package is a lot more strict on how it's built and it also needs to be run from a Debian machine to package it, and also needs all of the dependencies to match exactly as provided. (for instance, I need to have debian rust packages installed, even if I have rust from my rustup env and so on for the others). This provides much better flexibility for a dev environment to build from different version, experiment and still make a package out of it to test, and also uninstall the whole thing cleanly when on a dpkg managed machine :)
• In addition, since dpkg-buildpackage is meant for complete reproducability of the build, the current way it builds will result in a fresh vendoring and then building each. This will result in every small change triggering a full rust recompile no matter what. And this can get tedious for small updates.
• dpkg-buildpackage also pollutes the source tree, including vendoring pkgs, etc. Not ideal when you prefer to use just build and let just do the building for dev.
• Also, point 3, the option to all use just to build and just package up everything as a single pkg, I often is is much nicer for quick testing and also even tracking updates from dev, rather than having to rebuild every package manually, when not targeting distribution :)

Both come in handy at different times, so I keep these on my copy. But would be great if you'd like to stash it upstream in the scripts dir optionally for people who prefer to have options. Otherwise, please let me know and happy to remove them / adapt accordingly.

[prasannavl]

To summarize:

• pkg-deb-gitdev: Simply uses just install in a contained manner, and uses dpkg-deb to package the entire output tree as single deb to easy install, uninstall and cycle. This is not candidate to upstream back into debian, but can be used for building from other environments, or testing new toolchains, etc while manually taking caring to dependencies and it makes no assumptions about it.
• pkg-debs: This uses the traditional way to use dpkg-buildpackage that builds each of the packages. This requires ensuring dependencies as provided in each deb, making sure each of them are built reproducably from scratch each time it's build, regardless of the change, and is a better candidate to prep upstreaming back into debian.

[mmstick]

The dpkg-build package is a lot more strict on how it's built and it also needs to be run from a Debian machine to package it

If you aren't on a Debian machine, there would be no need to build a debian package.

In addition, since dpkg-buildpackage is meant for complete reproducability of the build, the current way it builds will result in a fresh vendoring and then building each. This will result in every small change triggering a full rust recompile no matter what. And this can get tedious for small updates.

The vendoring is optional. If you just want to build a package locally without using a schroot or vendoring, you can use -nc to skip the cleaning step, -d to skip the build dependency check, and just to build normally. If debian/rules has an auto_build recipe defined like test -e vendor.tar && just build-vendored || just, you can run dpkg-buildpackage -d -nc

dpkg-buildpackage also pollutes the source tree, including vendoring pkgs, etc. Not ideal when you prefer to use just build and let just do the building for dev.

The files generated by debian packaging tools are already ignored via .gitignore files. You can also run dh_clean to clean up all files generated in the debian folders. You can also use just clean or git clean -fd.

Also, point 3, the option to all use just to build and just package up everything as a single pkg, I often is is much nicer for quick testing and also even tracking updates from dev, rather than having to rebuild every package manually, when not targeting distribution :)

Compiling the entire desktop environment into a single package will take a lot longer than building the specific package that you need. In general, either just run or just && sudo just install by itself is all you really need.

If you want daily updates, you can install Pop!_OS 24.04, or use Fedora with the cosmic COPR repository, or Arch with the ChaoticAUR.

[prasannavl]

@mmstick I think we probably got a misaligned on what these scripts do.

Compiling the entire desktop environment into a single package will take a lot longer than building the specific package that you need. In general, either just run or just && sudo just install by itself is all you really need.

The following basically just does the above, but instead of the polluting the system without a way to cleanly uninstall, it just packages these into a single debian package for convenience that can be removed at will.

./scripts/pkg-deb-gitdev.sh
dpkg -i ./_build/<pkg>.deb

This should be very helpful in a debian based machine for people who are not full time developers on cosmic where they are okay with doing just install (without having a native way to undo this) or have Pop repo, but still wish to follow cosmic dev and have an easy debian native way to install / uninstall without having to do actually do clean reproducable builds which is one of the goals of dpkgs-buildpackage.

Note: Reminder that Debian best practice as well as I'd recommend as general disciple to not throw things into the /usr tree dev or not, and letting that be purely always managed dpkg as a better practice. And this just attempts to make it easier to do that while just sitting on top of just install.

[prasannavl]

The other script is nothing but just a simple loop over dpkg-buildpackage to build all at once. Happy to remove that if that's what you'd prefer.

If you want daily updates, you can install Pop!_OS 24.04, or use Fedora with the cosmic COPR repository, or Arch with the ChaoticAUR.

The intent for pkg-deb-gitdev.sh and the main intent of the PR, was mainly for those not on any of these above. Eg: Debian or Ubuntu, to be able to do this without having these on repos (even if it does land, I don't expect Debian repos to be as bleeding edge as Arch), and to be able to do this without any external factors and just purely do that with just git, dpkg and the source code :)

I hope, what you're trying to imply isn't, if you want to follow daily with convenience, just switch repo out of Debian :)

Currently, all I have to do stay upto on Debian without external dependencies is:

git pull
./scripts/pkg-deb-gitdev.sh
dpkg -i ./_build/<pkg>.deb

That's it. I'm upto date. It only builds what's changed, as it simply just uses just install, and is very convenient without having to depend on anything extra and can cleanly uninstall it straight from the debian package manager unlike just install.

In addition, I'd also say that, this also helps prevents bugs like: #2287. It was caught, because I tried to package the just install cleanly. This would have left system in polluted ways.

[prasannavl]

@mmstick I'm worried the current philosophy of just checkout individually is breaking the overall builds quite often. #2396 just broke things again.

Would perhaps a Github CI pipeline to do just build help catch these? Would also be great to run pkg-deb-gitdev.sh to also quick test a full dev artifact package with it if we can get this in.