support running components

[stdweird]

When kerberos infrastructure is available, we can easily encrypt and decrypt data, so we could send actual work to the daemon, eg run ccm-fetch (as usual) and also run components.

Code could work similar the way ccm-fetch downloads profiles data using the trust infrastructure (if decryption works and is from trusted source, do the work). If you can't decrypt or it's not trusted, we can decide wheter to still run ccm-fetch or not.

[jouvin]

I don't understand. Why Kerberos makes the standard workflow, with cdp-listend running ccm-fetch (download the profile) and ncm-cdispd scheduling components, inappropriate?

[stdweird]

we want to (re)run components on demand, in particular spma. this is a proposal to use cdp-listend for this. but cdp-listend is insecure, anyone can notify cdp-listend to trigger ccm-fetch. this is harmless for ccm-fetch (besides possible DOS).
so doing any actual work by running components requires some form of authentication. and kerberos would make this easy to implement.

[jouvin]

It would be good to have a more precise description of the use case. I don't really get what it is and how you want to trig component execution.

[ned21]

This is because a change outside of the quattor profile (e.g. updating a yum repo) can now trigger the need for an spma run without there actually being a profile update? Or why else would you need to run spma without a profile change?

[stdweird]

yes, esp on our test infrastrucre were we have no snapshotted repos and try things out with home made tools, we are looking for an improved way to push changes.
the other scenario is rerunning previoulsy failed components due to external factors (i.e. non-profile related).

btw, this is merely a proposal at this stage, i still need to have a look how much code is needed, and i hope can reuse the CCM code as much as possible.

[jouvin]

@stdweird your choice not to have YUM snapshots is not a valid use case for me!! I don't see why to add code and complexity in Quattor rather than used a proved method of managing packages which has many other advantages. With several methods available to do the snapshot management, covering probably most if not all situations!
As for rerunning previously failed components, this is just the matter of redeploying the same profile. If this is due to a fixed external factor, this will be enough as ncm-cdispd try to rerun all the failed components everytime it receives the notification of a new profile, even if there is no content changes. At least with SCDB, it works well because a new profile file is created on the server every time the deployment is run, even if the profile content is the same (this is needed for the timestamp to be updated so that ccm-fetch downloads the profile and ncm-cdispd sees a new profile). I don't know about Aquilon but I guess it should be easy (at leas feasible) to implement something that updates the profile timestamp even if the content has not changed.

[stdweird]

@jouvin wrt the yum repos, it is an issue if you have no snapshots (wheter or not this is a good idea is another discussion), and in our case, we don not have fine control over the snapshots (but that is more of an internal matter). in testbed environmnets, i don't see the point of running snapshots (but again, different discussion).

wrt failed components, i'll have to check the exact scenario what went wrong, but afaik, cdp-listend does no --force (rightfully so), so redeploying the same profile by notification won't work, right?
(support for ccm-fetch --force could be another thing supported by this issue, btw).

[jouvin]

@stdweird you are right, cdp-listend doesn't pass --force to ccm-fetch. But you don't need it if any time you deploy, you ensure that at least the timestamp of the profile is updated. Then ccm-fetch will download it as a new profile (even if the content is the same). This is what SCDB does (at least the standard SCDB but I think you modified it).
As for YUM snapshot, you don't need any control on any infrastructure. Just a filesystem where you can create them. Using a tool like yum_snapshot (you'll find it in SCDB repo), snapshots will be created as hardlinks so it doesn't require more than a unix file system and a unix machine!