See https://github.com/cert-manager/approver-policy/issues/782 - basically this issue is to do that but for trust-manager.