Metric additions for scaling services.

appservers/README.md

@@ -8,6 +8,13 @@
 * App error rate
 * Error counts on backend H/W
 * Bandwidth usage front and backend
-* System load on primary application server: CPU, memory, disk, swapping
+* System stats:
+  * cpu_idle
+  * run queue depth
+  * memory_free
+  * memory_in_use_by_application
+  * swap bytes in/out
+  * disk IO
+  * disk queue depth
 * Usage patterns:
   * which user, client time, session time, active vs idle time

databases/README.md

@@ -10,13 +10,18 @@
 * % of query time in io
 * # of locks
 * # of versions (for read consistency)
+* amount of time spent waiting for locks
+* deadlock/contention alerting
 * terminated connects
 * SQL statements
+* slow log growth rate
 * cache evictions
 * query errors by type
 * saturation: plan to execute
   * queueing on pool
 * change in number of executed plans
 * latency of last checkpoint, and on-disk representation of wall log
-  * (how much of DB to reply)
+  * how much of DB to replay
+  * how far behind the slaves are
 * checkpoint times
+* number of full table scans

distributed/README.md

@@ -11,4 +11,6 @@
     * per server
     * per backend
   * system 'flame graph'
-  * visualize traffic as graph, queue time, request flow
+  * visualize traffic as graph, queue time
+  * end-to-end request graph, with latency and queue depths
+

http/README.md

@@ -1,8 +1,10 @@
 # http (80/tcp)
 
-* duration_connect_msec (u_int32)
-* duration_firstbyte_msec (u_int32)
-* duration_total_msec (u_int32)
+* client_duration_connect_msec (u_int32)
+* client_duration_firstbyte_msec (u_int32)
+* client_duration_total_msec (u_int32)
+* backend_duration (u_int32)
+* backend_response_code (u_int32)
 * request_bytes (u_int32)
 * response_bytes (u_int32)
 * response_code (string)

https/README.md

@@ -1,3 +1,4 @@
 # https (443/tcp)
 
 * ssl_expire_in_sec (u_int64)
+* ssl_negotiation_time (u_int32)

network/README.md

@@ -12,7 +12,7 @@
   * buffer utilization
   * perf flow into:
     * app injection BW
-    * app injectiov rate
+    * app injection rate
     * app consumption rate
     * app consumption BW
 * Component:
@@ -24,3 +24,9 @@
   * app to app latency
   * app to app low
   * symmetry
+* Top Talkers Report
+  * Information about which backend services are the busiest
+  * Details about which external peers you transit with the most (broken down by egress port or ASN)
+* Internet Convergence Reporting
+  * BGPlay - http://bgplay.routeviews.org/
+  * BGPmon - http://bgpmon.netsec.colostate.edu/

webservers/README.md

@@ -6,9 +6,12 @@
 * Req size: distribution
 * Response Size: resp code, distribution
 * Responce Count: resp code, counter
-* Time To First Bite: resp code, distribution
-* Time To Last Bite: resp code, distribution
-* Active Workers: guage
-* Worker Age: guage
+* Time To First Byte: resp code, distribution
+* Time To Last Byte: resp code, distribution
+* Active Workers: gauge
+* Worker Age: gauge
+* Worker startup time: gauge
+* Worker Queue Depth: gauge
+* Syn-queue Depth: gauge
 * Connections:  counter
 * Process Metrics from host