|
| 1 | +A Python-based library for interacting with Chimbuko's Provenance Database after conversion to a relational database. It offers advanced query and visualization functionality, enabling a detailed analysis and comparison of performance anomalies captured by Chimbuko. |
| 2 | + |
| 3 | +# Installation |
| 4 | + |
| 5 | +- Clone the repo into the current directory (using pip) |
| 6 | +- `cd ChimbukoOfflineAnalysis` |
| 7 | +- `pip install .` |
| 8 | + |
| 9 | +# Usage |
| 10 | + |
| 11 | +1. After running your application with the [Chimbuko performance analysis tool](https://github.com/CODARcode/Chimbuko), convert the UnQlite provenance database to DuckDB format using [ChimbukoProvDBconvert](https://github.com/CODARcode/ChimbukoProvDBconvert) |
| 12 | +2. From Python |
| 13 | + ``` |
| 14 | + import chimbuko_offline_analysis as chim |
| 15 | + con = ProvenanceDatabaseConnection() #create a database connection |
| 16 | + pdb = con.connect("/path/to/your/duckdb/database") #connect to the database |
| 17 | + ``` |
| 18 | + |
| 19 | + You can also connect to multiple databases by making repeated calls to `con.connect("/some/other/database")` |
| 20 | + |
| 21 | +## Analysis functionality |
| 22 | + |
| 23 | +### Profiling |
| 24 | + |
| 25 | +Profile information can be obtained using both the 'inclusive' (timing includes child function calls) and 'exclusive' (timing comprises only time spent within the function but not within child calls). For an application profile: |
| 26 | +``` |
| 27 | +pdb.getApplicationProfile("inclusive") |
| 28 | +pdb.getApplicationProfile("exclusive") |
| 29 | +``` |
| 30 | +The profiles are sorted in descending order by the runtime. Here the exclusive profile also contains information on the anomaly **severity** (total amount of time spent in anomalous function executions) and fraction of time spent in anomalous function executions, providing a means of identifying functions that warrant more detailed investigation. |
| 31 | + |
| 32 | +Note that in this library, the function is referenced by a **function index** rather than as a string. Make note of the function indices of functions of interest. |
| 33 | + |
| 34 | +More detailed profiling information on individual functions can be obtained using, |
| 35 | +``` |
| 36 | +pdb.getFunctionProfile(function_index, 'exclusive') |
| 37 | +pdb.getFunctionProfile(function_index, 'inclusive') |
| 38 | +``` |
| 39 | + |
| 40 | +Assuming the HBOS (default) or COPOD algorithm was used to identify anomalies, it is also possible to obtain a complete histogram of exclusive execution times using |
| 41 | + |
| 42 | +``` |
| 43 | +edges, counts = pdb.getFunctionADmodelHistogram(function_index) |
| 44 | +``` |
| 45 | + |
| 46 | +which can be plotted using, e.g., |
| 47 | + |
| 48 | +``` |
| 49 | +import matplotlib as mpl |
| 50 | +mpl.pyplot.hist(edges[:-1], edges, weights=counts); |
| 51 | +mpl.pyplot.show() |
| 52 | +``` |
| 53 | + |
| 54 | + |
| 55 | +### Identifying analysis targets |
| 56 | + |
| 57 | +Functions that may deserve more detailed analysis can also be identified using |
| 58 | +```pdb.topfunctions(sort_by)``` |
| 59 | +where `sort_by` can be |
| 60 | +1. `anom_severity` - the accumulated anomaly severity - a metric of anomaly importance. |
| 61 | +2. `anom_count` - the count of anomalies. |
| 62 | +3. `total_time_excl` - the total exclusive runtime of the function. |
| 63 | + |
| 64 | +### Call stacks |
| 65 | + |
| 66 | +Anomalies and normal executions are tagged by Chimbuko based on a model indexed by the function name, but in practice there are many call paths/stacks that can execute a specific function. Summaries of which call stacks were associated with the collection of stored anomalies and/or normal executions can be obtained using |
| 67 | +``` |
| 68 | +pdb.getFunctionCallStackLabelsAndCounts(function_index, subset) |
| 69 | +``` |
| 70 | +where `subset` can be `anomalies`, `normal_execs`, `both`. The call stacks are assigned a hash index, which can be translated into a call stack using |
| 71 | +``` |
| 72 | +pdb.getLabeledCallStack(call_stack_hash) |
| 73 | +``` |
| 74 | + |
| 75 | +### Function anomalies and normal executions |
| 76 | + |
| 77 | +The anomalous and normal executions (henceforth, *events*) for a specific function can be obtained using |
| 78 | +``` |
| 79 | +pdb.getFunctionEvents(function_index, "anomalies") |
| 80 | +pdb.getFunctionEvents(function_index, "normal_execs") |
| 81 | +``` |
| 82 | + |
| 83 | +Individual events are referred to by a unique event index (**event_id**), e.g. `0:0:231:435154`. Take note of events of interest. |
| 84 | + |
| 85 | +The time distribution of anomalies on a specific program index (**pid**) and rank (**rid**) can be obtained using |
| 86 | +``` |
| 87 | +times = pdb.getAnomalyTimes(pid,rid) |
| 88 | +func_times = pdb.getAnomalyTimes(pid,rid,function_index) |
| 89 | +``` |
| 90 | +where the second version specified a specific function of interest. These can be plotted as, e.g. |
| 91 | +``` |
| 92 | +import matplotlib as mpl |
| 93 | +mpl.pyplot.hist(times, bins=100); |
| 94 | +mpl.pyplot.xlabel("application run time (s)") |
| 95 | +mpl.pyplot.xlabel("anomaly count") |
| 96 | +mpl.pyplot.show() |
| 97 | +``` |
| 98 | + |
| 99 | +### Detailed event information |
| 100 | + |
| 101 | +For a given event, a table detailing function executions occuring in a time window around the event on the same thread, can be obtained using |
| 102 | +``` |
| 103 | +pdb.getEventExecWindow(event_id) |
| 104 | +``` |
| 105 | + |
| 106 | +One can also obtain the node memory and CPU usage information using |
| 107 | +``` |
| 108 | +pdb.getEventNodeMemoryStatus(event_id) |
| 109 | +pdb.getEventNodeCPUstatus(event_id) |
| 110 | +``` |
| 111 | +Note, however, that this information is collected only periodically, and so the sampling time may not align with the function execution. |
0 commit comments