LREC2026-From-Variance-to-Invariances

Source code for LREC 2026 submission: From Variance to Invariance: Qualitative Content Analysis for Narrative Graph Annotation

Data

Data Path	Description
./data/task_1/task_1_annotation.csv	Annotated data for task 1: narrative identification, including unaggregated labels
./data/task_2/task_2_annotation.csv	Annotated data for task 2: narrative identification, including unaggregated labels
./data/task_1/all_agreeing_articles.csv	A subset of data used for task 1 annotation. It includes only items where all annotators agreed on a label
./data/task_1/cause_dominant_articles.csv	A subset of data used for task 1 annotation. It includes only items where all annotators agreed on a label, and this label is "Inflation-cause-dominant"

Analysis

Data Path	Description
./compute_agreement_task_1.py	compute krippendorffs alpha for task 1 on flat label space (fast)
./compute_agreement_task_2.py	compute krippendorffs alpha for task 2 on graph annotation (slow for feature four)

Feature name map for task 2

• "feature_one": "Adjacent Events"
• "feature_two": "All Events"
• "feature_three": "Relations"
• "feature_four": "Full Story"
• "feature_six": "Adjacent Story"
• "feature_eight": "Extended Story"

How to run?

activate your favorite environment, and run the following,

pip install -r requirements.txt 
python compute_agreement_task_1.py
python compute_agreement_task_2.py # this will take a while (2-3 hours on a 2021 Macbook M2, computing graph edit distance is NP-hard! But a 1 minute timeout is implemented.)