perf: optimize startup when there are many health-checked backends

[jranson]

This fixes a startup bottleneck with health check initialization, and separately revives the ProxyOnly backend flag from Trickster 1.0.x, removed in 1.1.x.

ProxyOnly

Back in the day, we had a ProxyOnly setting, but that was replaced eventually by just changing the backend provider type to reverseproxy. Now that we have ALB features specific to time series (namely tsm), users may want to utilize features like TSM without necessarily using the caching features. I actually needed that exact setup to help debug this bottleneck, so I added it back in the process. Since it has been useful to me, it may be useful to others, so I kept it in here. The way it works is the DeltaProxyCache and ObjectProxyCache will both check the request's corresponding backend first thing; if it is configured for ProxyOnly, the Cache function will immediately hand the request off to the Proxy handler without doing any caching.

Health Checks

At startup, the backends.StartHealthChecks routine iterates each health check that has an interval configured, and runs hc.Start() to bootstrap them. The root cause of the latency is that Start() blocks on the initial target probe response, causing the application startup to be slower and slower based on upon how many health check targets are configured and summation of network + server latency across all applicable health checks. To resolve, the initial probe is moved to inside of the probeLoop goroutine but before the ticker loop.

[coveralls]

Pull Request Test Coverage Report for Build 20150197051

Details

• 6 of 11 (54.55%) changed or added relevant lines in 3 files are covered.
• 2 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.001%) to 71.972%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/proxy/engines/objectproxycache.go	1	3	33.33%
pkg/proxy/engines/deltaproxycache.go	2	5	40.0%

Files with Coverage Reduction	New Missed Lines	%
pkg/proxy/engines/cache.go	2	91.49%

Totals
Change from base Build 20136677829:	-0.001%
Covered Lines:	14673
Relevant Lines:	20387

---

💛 - Coveralls