Commonsense-Knowledge-Aware-Concept-Selection-For-Diverse-and-Informative-Visual-Storytelling

The implement of Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling

This repository consists of two parts:

1. Concept Selection
2. Visual Storytelling

Concept Selection

In the paper, we use the concept selection to first select the concepts for the story generation.

codes are involved under concept_selection/

run python train.py to train the model.

Visual Storytelling

In the paper, we use the concepts and the image as inputs to generate the story.

We use BART as pretrained model.

codes are involved under visualstorytelling/

run python train.py to train the model.

Data

All the data used in the project can be found in https://drive.google.com/file/d/1jW0fgZnCUX3EIefwTcR0l8bHDrgDSXwm/view?usp=sharing