Forking the project

[cristim]

After a heated discussion on #136 generated by the stale state of this project, I proposed to fork it and run it like a meritocratic open-source project, really involving the community in development and with open governance.

I created an organization for it and forked the project at https://github.com/sedutil/sedutil
I for one won't be able to contribute much to the code but I'll do my best to set up the project and get it started. The most prominent fork and PR authors were added to the organization and have commit rights, and more authors can be added later if needed, once they have some meaningful contributions to the project.

The fork is currently based on the official code base, but the goal is that any commit passing code review by the org members will be accepted.

The current PR authors are kindly asked to submit their PRs there for review.

There is also a team chat on gitter.im where you can get in touch with the open source community.

Everyone is invited and welcome to submit issues, patches and suggest improvements.

[Manouchehri]

@cristim Could you unfork the GitHub "fork" link? Drive-Trust-Alliance/sedutil looks dead (in my opinion), we should probably make it clear that sedutil/sedutil is a new/separate project. (Also, we can't do some things like search for code on a forked repo.)

Thanks!

[cristim]

I don't know if that is possible, I still hope that DTA will eventually resume the work on their side and join the development efforts. I don't want this to be a schism-like fork, it is meant to be temporary until DTA wakes up, starts to run the project in the open and accept input from the community.

[kylemanna]

I think just forking the project and not attempting to work here with @r0m30 to find a way to add new contributors is a little dramatic. Furthermore, forking the project seems frivolous when the majority of the heavy lifting for OPAL SED support has been added to the Linux kernel. Large parts of the project aren't needed anymore and a new tool may be the most appropriate way forward.

Also, you can remove the fork link in one of two ways:

1. Delete your new Github repo, create a new repo with the same name and push the local copy of your repo to the server. This will destroy all Github metadata on the project.
2. Contact Github support and ask nicely.

But in all honesty, I think the link to this repository helps more then it hurts at this point.

[kylemanna]

That said, I have a list of features I'd like to see and would likely implement myself if given enough time:

1. The LinuxPBA and sedutil-cli tools are linked against the host machine libraries and not the Buildroot or TinyCore Linux libraries. This creates problems with shared object library mismatches. I hacked this together on my fork with a Docker build environment. Unfortunately, since the Makefiles are generated by NetBeans, the Makefiles/build system needs to be generalized first.
2. Update the Rescue image to EFI bootable image with tools like efibootmgr available to help install the syslinux bootloader in to the UEFI.
3. Improved image build scripts for building images, related to (1) and (2) above. Also, a more automated way to build the images would be desirable. Ideally Travis-CI (or whatever) could generate build artifacts and host them somewhere for people to test.
4. Explore moving away from Syslinux to something that can chain load a EFI boot loader (I use systemd-boot) to speed up boot time by not having to reboot through the BIOS/UEFI. This would also potentially resolve issue such as EFI PBA does not show in BIOS (I figured out why) #144
5. Begin integrating support to call the Linux OPAL SED ioctl()s where possible. The IOC_OPAL_SAVE ioctl is most interesting because it would enable Linux 4.11 and later to unlock the drive after S3 sleep.

These are my ambitions, but time is a huge constraint for me as I'm sure it is for @r0m30 to support reviewing PRs to add such features.

I'm hesitant to jump in and support a fork until I know it has staying power. Alot of people massively underestimate the amount of work necessary to run a popular open source project. I'd like to thanks @r0m30 for his time developing and supporting this project over years and hope that there is a sensible path forward for all parties going forward.

[cristim]

I agree this is an extreme measure, and I respect @r0m30's great work so far in this project. But the reality is people may move on and become too busy or for various reasons become unable to contribute, and when that happens they should ideally allow their work to be continued by others. If there is a community active enough, it's likely to have someone else able to pick up the torch, and surely there were various other people who kept working on their forks for long after DTA stopped the development.

As a user I had a hard time finding the best individual repo fork which would work well enough for my particular setup, so I think it's best to have a central repo where people can agree to submit their work and where the users can get the latest and greatest and well-tested code without needing to use individual forks where this may be a gamble, but at the same time have reasonably up-to-date code as well.

So all I did was to establish such a place where interested people can continue working on this project, and hopefully agree to treat it as a central community repository. I wish DTA would have continued development and opened up for contributions so this wouldn't happen, but here we are.

I have no plans to discontinue this new sedutil organization, and I will happily grant permissions to whoever makes significant contributions to the project, in order to increase the bus factor and remove PR approval bottlenecks. DTA is also welcome to join, if they are able and willing to.

Regarding the roadmap going forward, it's great to discuss such things with the community, and come to an agreement on how to proceed, share the effort and do it collaboratively which may be even more fun. I'm also willing to contribute as much as I can, perhaps also at a rewrite from scratch in a new project within this new organization.

[kylemanna]

Couldn't agree more @cristim . My main concern is to let this issue simmer for a week or two and give @r0m30 some time to respond. If he doesn't respond, then there is no alternative for those that depend on this software.

[bhack]

Do we have any contact with Distro (Fedora, Debian, Ubuntu etc...,) maintainers of this package?

[cristim]

@bhack at least on Ubuntu there are no packages for it.

[cristim]

@kylemanna (and anyone else), it's been over two weeks now, do you think we should wait more for @r0m30 to respond here or can we move on?

Please vote this comment up if we should move on and vote it down if we should keep waiting for a while more.

[kylemanna]

I think effort would be better spent updating nvme-cli to replace the sedutil-cli and then generating a more modern PBA rather then a fork of this repository.

[cristim]

I've just seen @r0m30 published a lot of code developed by DTA behind closed doors over the last year or so, and DTA is now open to receiving external contributions from people who sign an agreement.

This is very welcome and likely renders this fork obsolete on the long term, provided that DTA will begin to involve the community in the maintenance of the project and pledge for development in the open going forward.

In the meantime I will keep maintaining the fork for a while in order to see how DTA is managing their development in the open and I will try to rebase this fork on top of the current DTA codebase, as much as that makes sense.

Since the fork has no commercial interests and doesn't need a contributor agreement, it could also become an experimental/testbed/community branch where anyone unwilling to sign the DTA agreement can submit patches, as long as there is demand for this from external developers.

[r0m30]

I'm not going to get into the to fork, not to fork discussion, that's up to the individual, but I'd like to address a few things.

I understand peoples impatience but they need to understand that when I do build a new release it has been tested on two different machines. New kernel configs, both 32bit and 64bit, need to be looked at for updated/added device support. I also do a clean build from the upstream repo to ensure that the reason "it works for me" isn't that there is some tweak I made that didn't make it to the repo or a file that ended up being ignored by git. I didn't do this when I pushed the pbkdf2 function replacement and that went poorly. This is a huge chunk of time and I think it makes for a better release. It also means I need to find that huge chunk of time.

Most of the people who are here to further the development of sedutil are very Linux focused, this project currently supports Windows and there is a PR #87 to add FreeBSD support. Linux solutions do not always work for the broader project goal of multi-OS support.

@kylemanna had some goals most of which I share.

That said, I have a list of features I'd like to see and would likely implement myself if given enough
time:
The LinuxPBA and sedutil-cli tools are linked against the host machine libraries and not the Buildroot
or TinyCore Linux libraries. This creates problems with shared object library mismatches. I hacked this > together on my fork with a Docker build environment. Unfortunately, since the Makefiles are
generated by NetBeans, the Makefiles/build system needs to be generalized first.

I've added an autotools make for sedutil-cli and linuxpba that can build locally but it's primary purpose was to allow sedutil-cli and linuxpba to be built as a package in buildroot so the clib/stdlibC++ are in sync. I'm hoping this will also give the distro packagers the flexibility they need.

Update the Rescue image to EFI bootable image with tools like efibootmgr available to help install
the syslinux bootloader in to the UEFI.

The latest rescue systems (there is now a UEFI rescue system) include the PBA image files so a drive could be setup from the rescue system without additional prep. The rescue systems kernel and rootfs now match the PBA kernel and rootfs which eliminates the inconsistencies where the rescue is able to run the code but the PBA fails. Do you have a system where the efibootmgr settings persist? All of the systems I've run into don't.

Improved image build scripts for building images, related to (1) and (2) above. Also, a more
automated way to build the images would be desirable. Ideally Travis-CI (or whatever) could generate > build artifacts and host them somewhere for people to test.

The goal for the build scripts is to use buildroot as much as possible going forward. Eventually I'd like to be able to build the entire PBA with a make O=[32|64]bit and then remaster these images removing the init.d script that runs the PBA to create the rescue systems. There is also work to be done to make the build more tarball friendly.

Explore moving away from Syslinux to something that can chain load a EFI boot loader (I use
systemd-boot) to speed up boot time by not having to reboot through the BIOS/UEFI. This would also > potentially resolve issue such as #144

The current boot/unlock/reboot scenario is not ideal, the original biospba chainloaded to avoid this. I need a more generic and multi-platform solution. Unfortunately that probably means a UEFI module. I took a quick look at systemd-boot and that's another possibility but there are a lot of edge cases that would need to be addressed. Do you have pre-built boot entries for your setup or do you find the unlocked EFI partition(s) and glob them into the pba filesytem under /boot? Do you have any systems that run under CSM?

Begin integrating support to call the Linux OPAL SED ioctl()s where possible. The IOC_OPAL_SAVE
ioctl is most interesting because it would enable Linux 4.11 and later to unlock the drive after S3 sleep.

This is NOT a multi-platform solution, adding a new function that calls IOC_OPAL_SAVE in the Linux code and says sorry your OS doesn't support S3 in the others is probably the way forward for this project. You'd also want to check for the kernel support, are the distros at 4.11 yet?

===========
Finally, I'm not sure what to do with this issue, I don't want to appear vindictive or petty by closing it but it really isn't about the goals or future of this repo.

[cristim]

As the one who started the fork, I'd also like to say a few more things about how we got here and my thoughts on how to get out of this situation.

Fork history
When the fork was done, the project was more or less dormant, with the last mainline commit almost a year old, with no recent comments from anyone at DTA on the issues currently faced by the users, and with significant commits on various individual forks solving parts of the problem, but none of them complete and tested enough to become some sort of a mainline branch.

I was just a simple wanna-be user, confused on which of those forks to use, and struggling to get started since my NVMe hardware wasn't supported on the mainline codebase.

Eventually after some research I chose @kylemanna's fork, which seemed the most complete at the time, I started to join discussions on various issues and tried to help other new users get started. That's when I noticed I wasn't the only one frustrated about the state of the project, there was even a very heated discussion in which people were calling names and things got really out of hand.

So all I tried to do was to reboot the project into a community-driven initiative, in which the main fork authors would be empowered to review each other's work and collaborate towards a shared mainline which would be tested enough for anyone to use. I informed DTA about this but there was no answer signaling that development had continued and that a code drop would be imminent so this fork would be useless, so we just carried on with merging work in the individual forks and started getting some traction. The work had just started and the progress was slow, but steady, until the DTA code was published.

@r0m30 I understand it took a huge amount of work to write all this code and test it, and I really appreciate what you did, I just wish you spent a few minutes at some time over the last year to write a message that would indicate you're still alive and that something is in the works, or even better to release the work in progress in a development branch, that would have saved a lot of time.

Next steps
I would really like to remain involved in the project, I use it every day and I'm really thankful it exists,
but I also feel the pain for the suspend-to-ram and unlock/reboot issues, which I would love to see fixed and I would like to contribute as much as I can towards those. That may be hard since I'm not a C++ developer but I have some experience with building embedded Linux systems, and I can help with the build system and PBA part.

As for the fork, as I said before, I'd like to keep it for now as a community-driven effort, sync'ed as often as possible with the DTA mainline and trying out experimental things such as a different buildsystem, PBA and rescue images and so on.

But at the end of the day, the sedutil C++ code base may have little to do with the binary packaging delivered to the users, which is the PBA and Rescue image. Think of it as like the relationship between the Linux kernel or any other such project versus a Linux distribution shipping it to users.

It could all be done in different source code repos and managed by entirely different people with different interests. I'm not saying we should necessarily split it, just that some of us have different concerns. The cross-platform compatibility may be important for the (sub-)project building the C++ code base, but may matter little for the (sub-)project building the PBA image, which is Linux-only so far.