Summarization_usecase

1. Summarizing the  the conversational data of customer  and agent to create transparency notes.
Customer support agents spend typically spends lot of time and efforty writing notesafter each call, or while transfering the case to the next level of support.



1. Topic modeling
2. Key bert extrtacting important key phrases using bert embeddings 
3. using summarization models like pegasus,BART,T5  (should be train on samsum conversational data)
Evaluated model-: bart-large-xsum-samsum
https://huggingface.co/lidiya/bart-large-xsum-samsum
after summarizing the data we can create short notes of that data (Actuak gist of the chat) for further processing 
max length is 1024 when finetuning bart

I would exclude T5 a priority, since its max input length is 512, while Bart and Pegasus can be fed with max 1024 tokens
tokenizer.model_max_lengt


above usecase -:For any Call Center, the performance metric is more focused on speed and subjective elements than the depth of problem-solving.
Now imagine how much information a call center or a customer satisfaction monitoring application has! Making sense of this unstructured information 
and utilizing it 100% to optimize your call center can be a far-fetched dream without a streamlined process.

If the evaluations have to be done by human reviewers, it will first take long working hours to conclude how to utilize it to optimize the customer service. 
Be it ineffective transfer of calls or forcing the customer to repeat their problem - summarization can help to optimize the call rerouting and save the customer 
from repeating oneself by providing a prior summary to each new representative of the call center while the call reroutes to a new representative.

This leads to improving CX,retention rates,brand image

About Dataset:-About 75% of the corpus is a conversation between two people, the rest are between three or more.
The dataset contains 14732 of training conversations, 818 for validation and 819 for testing
good reads----->https://medium.com/@ferlatti.aldo/fine-tuning-a-chat-summarizer-c18625bc817d

Initially I was thinking of using the basic BART model (the large version), but after fine-tuning it I was not satisfied with the summaries it generated
they were indeed short, but a lot of information were left out which I considered important. Because of that I decided to use the model which was fine-tuned on the 
extreme summarization dataset (xsum). The dataset was developed for evaluation of abstractive single document summarization. 
The dataset consists of 223.711 news articles with one sentence summaries