Evaluating Table Question Answering System

[MariaDavid30]

I am working on Table question answering system. I evaluated the texttableQA pipeline using the eval function and I got a recall value of 0.8 and precision value of 0.08.
What are the ways to improve the precision?
How to create better text and table labels manually?
What are the methods of evaluating a table question answering system apart from haystacks evaluation feature which uses the eval function?

[silvanocerza]

Hey @MariaDavid30, can you provide some more details? Some code snippets of what you're trying to do, some example data would be helpful too.
With this much information there's little we can do to help. 😕

[MariaDavid30]

Hey @silvanocerza , I followed the Tutorial: Open-Domain QA on Tables (https://haystack.deepset.ai/tutorials/15_tableqa). I am using a PDF file as my data.

These are the reader and retriver models used

retriever = EmbeddingRetriever(document_store=document_store, embedding_model="deepset/all-mpnet-base-v2-table")
text_reader = FARMReader("deepset/roberta-base-squad2")
table_reader = TableReader("google/tapas-base-finetuned-wtq")
route_documents = RouteDocuments()
join_answers = JoinAnswers()

This is the pipeline

text_table_qa_pipeline = Pipeline()
text_table_qa_pipeline.add_node(component=retriever, name="EmbeddingRetriever", inputs=["Query"])
text_table_qa_pipeline.add_node(component=route_documents, name="RouteDocuments", inputs=["EmbeddingRetriever"])
text_table_qa_pipeline.add_node(component=text_reader, name="TextReader", inputs=["RouteDocuments.output_1"])
text_table_qa_pipeline.add_node(component=table_reader, name="TableReader", inputs=["RouteDocuments.output_2"])
text_table_qa_pipeline.add_node(component=join_answers, name="JoinAnswers", inputs=["TextReader", "TableReader"])

[silvanocerza]

I would probably suggest to following this guide from HuggingFace on how to fine-tune a TableQA model.

That might be helpful, let us know.