Feature flag / cherry-pick of alerting

[z0ph]

The idea is to be able to enable/disable some of the basic features of this kit.

For example, you don't want to enable the alerting when someone is triggering sts get-caller-identity to avoid alert fatigue.