@@ -29,28 +29,5 @@ the most friendly/intuitive format.
2929
3030## Notes
3131
32- * The "time chunking" algorithm tries to put as many events, up to the
33- limit specified in a "bucket". We start out by breaking the index into buckets
34- of 86400 seconds, or one day. If the number of events in this bucket is more
35- than our limit, we split the day into 24 buckets of one hour each. If any of the
36- hour buckets contain more events than our limit, the hour is split into 60
37- buckets of one minute each. If any of the minute buckets contain more events
38- than our limit, the minute is split into 60 buckets of one second each. A second
39- bucket is the smallest granular size.
40-
41- The code has a downsample map:
42-
43- { 86400 : 3600, 3600 : 60, 60 : 1 }
44-
45- This maps the current "bucket length in seconds" to "next bucket length in
46- seconds" if the current bucket contains more events than our limit.
47-
48- As such, it is important that the initial starttime begins on a day boundary
49- (i.e. 12:00:00 AM).
50-
51- * The goal of export.py is NOT to optimize the number of requests to splunk,
52- rather to optimize the size of the return request from splunk so that in the
53- cases of very large indices, robustness and restart are paramount.
54-
5532* When using csv or json output formats, sideband messages are not included. If
5633 you wish to capture sideband messages, the xml format should be used.
0 commit comments