Possible new major version 2 of the guide

[ehlesp]

Motivation

In this thread we discuss all the necessary changes to update the software versions used in the guide. This also implies increasing the release number of the guide to a major version 2 to distinguish it from the v1 guide that uses older software versions.

Context

• Issue 18. sched_migration_cost_ns is not found

[ConYel]

Hello @ehlesp, and thank you for this great guide!
Just to point out that some blog posts regarding the G16, transparent hugepages, https://alexandrnikitin.github.io/blog/transparent-hugepages-measuring-the-performance-impact/ they discuss that maybe is not the best option to disable hugepages.
I will proceed to see if there are any other changes to be done downstream in case something doesn't work properly.
Just to inform you that I haven't applied all the various suggestions regarding firewalling etc, as my case is a bit different.
I also use ZFS.

[ehlesp]

Hi again @ConYel . I've been thinking about the effort this update would imply and I've realized that I won't be able to work on it.
It would require going chapter by chapter updating snapshots, testing all the commands, etc, but based on a setup (yours) that is more complex (ZFS and its implications) from the more simpler and less capable setting I used originally (and that I cannot fully replicate in a virtual environment).

Another big problem here is that I made the guide showing the after and before situations since the start, and I totally understand that you wouldn't tear down your own setup when you're probably using it. Further, since your setup is running on a more powerful hardware, uses ZFS and a different network setup, it would make more sense to make a fork of my guide and evolve it to fit your scenario.

In conclusion, the update is not just bumping version numbers, but a much more involved process for which I do not have the time right now.

How you see this on your end?

[ConYel]

Hello @ehlesp ,
I understand the possible issues, and I believe that it is better first to update the current repository and then I could make another guide (although I don't think that it will be needed as my setup is not that much different from yours and I haven't finished it yet so there are a lot of chapters for me to follow.)
For this I can restart everything in a VM if you would like to and try to replicate as much as possible from what you are showing currently.

Everything will be in a VM of my current setup so I don't know how much difference it will make but at least in that way we could be closer to the example.

Let me know if you believe that might help.

[ehlesp]

Ok @ConYel , let's update the guides. Still, I'll indicate here a few considerations so we can coordinate better:

• I've created a v2-main branch which already has its own PR.
  The idea is to create subbranches named v2-G###-sw-version-refresh, one per updated chapter and each with their own PR. I explain why in the v2-main PR, but this will allow us to keep clean independent conversations about the changes done to each guide.

• I understand you're talking about doing a virtualized setup running ProxmoxVE and everything else.
  So, I think the first thing to change will be the G001 hardware setup chapter and add a note about this. On my end, I'll use a VirtualBox VM that imitates as much as possible the original setup I used.

• Ideally, all of our changes will be on snapshots and code or config snippets.
  Still, it is possible that some option (in particular in K3s) may have changed for some reason in a newer software version. In particular, what you proposed about leaving the hugepages active might lead to greater changes beyond a simple software update. We'll have to talk about that in the corresponding PRs for the affected chapters .

By the way, do you happen to use an UPS with your Proxmox setup? I still have mine, but I no longer have the machine I used to do the original setup explained in the guides. This would be only necessary to verify that the latest version of NUT works exactly the same and gives the same outputs as those shown in the chapter G004.

[ConYel]

Hi @ehlesp, ok great!
I believe although it will take a bit of time, it might help many people.

• Sure, your branching policy looks good.
• As, I cannot reinstall everything from scratch on my current hardware, I will be using a new VM on my ProxmoxVE to follow the guide.
• I would like to suggest to use GPG signed commits -only- for the new release, and apply it as a stricter policy in the repo.
• Yes I have a UPS but it is not setup yet with NUT or something relevant, I might be able to fix that part next month though.

[ehlesp]

Yes, it will take some time (a few months probably). This might be a bit of a problem with certain software that has "fast" versioning cycles (I'm thinking about K3s and certain components used in the Kubernetes environment).
I've enabled a couple of rulesets to enforce GPG signed commits only and other protections. Let me know if you happen to have issues when using Git with the project.
I forgot to mention, title your PRs in a consistent manner. Something like V2. G###. Refresh to newer software versions should do for all of them.

[ehlesp]

Hello @ConYel . I've advanced quite a bit in the refresh of the guide, reaching the G015 and G016 chapters where you noticed issues.
You can see my changes in the v2-main branch :

• I've removed the sysctl option sched_migration_cost_ns from the Kernel optimizations sections in chapter G015, since is no longer available as you already warned me back in March.

• In chapter G016 I've decided to keep the transparent hugepages disabled. After reading the blog post you shared with me previously in this discussion and other related contents, it remains unclear what is best to do in a Proxmox VE server (specially one as small as the one used in my guide). But I've added a bit more context in that chapter G016 about what is going with the transparent hugepages, and also a warning about not discarding the feature "forever". This feature in other systems and workloads could be benefitial but requires a proper performance study in the target system.

Let me know what you think about these updates whenever you can check them out.

[ConYel]

Sure I will let you know in some days!