Support Modules
- All of the support modules (except the Inter-Rater Module) require the same type of input files to function.
- The Validation, Optimization, and Exploration Modules require:
- Human annotation labels (in the form of an 'hB' file)
- BehaviorDEPOT output folder (saved as '_analyzed' in the data directory)
- The Inter-Rater Module requires:
- One or more human annotation files ('hB')
- BehaviorDEPOT labels can optionally be added to the folder
- To avoid errors, it is recommended that you place rater files together in a separate folder for analysis
- The Validation Module can optionally be run on batches of files; to use batch analysis place folders with paired 'hB' files and 'analyzed' folders into a new folder; select the folder containing the individual data folders when launching the module.
Allows user to make comparisons and visualizations of behavior annotations from multiple users using an averaged-rater projection or a chosen rater as a reference; includes useful visualizations and the ability to output a report of errors between each user and the reference dataset.
- The IR Module is designed input a folder containing 'hB' files and (optionally) BehaviorDEPOT 'analyzed' folders.
- All files for analysis must be placed within a single directory
- Select the 'Inter-Rater' button from the BehaviorDEPOT GUI.
- From the pop-up menu, select all plots you want to generate.
- If you want to exclude a rater file from analysis, check 'Restrict analysis to subset of rater files' in the pop-up menu.
- Click 'Start'.
- Select the directory containing the rater files to analyze.
- Assign names to each rater file.
- Select rater to use as reference (including the option to select the average of all raters).
- Select behavior to analyze from pop-up menu
- Results will save in the same directory as the rater data.
- This plot shows the percent agreement for all raters over the session analyzed. Values range from 0-1 (i.e. 0-100%).
- Agreement scores are only shown for behavior-labeled frames. Non-behavior frames that are in agreement by all raters will be display as 0.
- This plot shows the disagreement between raters by individual rater. The value given per rater for any frame is the number of other raters who disagreed at that frame.
- This plot shows the percent overlap between each pair of raters.
- This plot shows the precision, recall, f1 score, and specificity
- This is a folder containing text files for each rater. In each file, the index start and stop values are reported for all errors found between the reference data and individual rater, organized by length of error span. Additionally, the length of each error 'bout' and the type of error are reported.
- Error types: false positive (FP) or false negative (FN)
- Structure containing all data displayed in plots
Allows user to choose a batch of BehDEPOT-analyzed files (with associated hBehavior files) and quickly generate performance statistics and visualizations for each video and average statistics for the entire set.
- Can run in single or batch mode based on the input provided to it
- When run, prompts user to select:
- SINGLE: a directory containing 1 BehaviorDEPOT output folder and 1 associated hB file.
- BATCH: a directory containing other directories in the format for single mode described above.
Allows user to choose a behavior classifier, 1-2 classifier parameters (i.e. thresholds), a range of test values, and a human annotation file to quickly examine effects of chosen values on classifier performance (reporting F1 score and ROC-based AUC values for range of tested parameters)
Allows user to choose 1 or 2 metrics from a BehDEPOT-analyzed file and an associated hBehavior file to sort data by chosen behavior and generates a report comparing the behavior subset to the complete dataset based on the selected Metrics.