Generative QA pipeline with PromptNode output incorrect/

[sb1992]

Hi I am using the tutorial : "Creating a generative QA Pipeline with PromptNode: given here:
https://haystack.deepset.ai/tutorials/22_pipeline_with_promptnode

But at the last step when i run

output = pipe.run(query="How does Rhodes Statue look like?")
print(output["results"])

it returns the below output for any query

['The query is a search for documents.']

Cant figure out whats going on. Help would be much appreciated

[ZanSara]

Hey @sb1992 did you alter anything else in the tutorial? By the way, if you want to get help faster, consider joining our Discord server. We reply much quicker there, and there are many more people that can help. https://discord.com/invite/VBpFzsgRVF

[sb1992]

No i didn't. I just ran it first locally on my laptop and got the above error/output and then tried on the colab notebook and still encountered the same output,

Sure will join discord as well

[bilgeyucel]

Thank you for raising the issue @sb1992! I could reproduce the issue but it should be solved now. Feel free to add to this thread if you notice something wrong with the tutorial :)
https://haystack.deepset.ai/tutorials/22_pipeline_with_promptnode

[astha-ts]

I modified the code to build a custom pipeline that extracts text from a pdf. I also got the same output: 'The query is a search for documents.'
Do we know what's causing it? I had run the same modified pipeline about a month ago on google colab, and I didn't have this problem. I am getting the same problem today while running on my Mac and on google colab.

[sb1992]

Yeah i reckon was caused by the Shaper. So in earlier notebook there was a shaper and now i reckon you directly pass {join(documents)} in prompt text instead using of shaper( and shaper is removed from pipeline as well). The colab notebook above has been updated with those changes and maybe that is the issue for you as well

[bilgeyucel]

Hey @astha-ts, as mentioned above, everything should be fine with the updated version. Would you like to share your pipeline so that I can take a look?
To explain what has changed in the tutorial: now there's an easier way to modify documents in prompts. So, there's no need for a Shaper if you're just joining documents.

[astha-ts]

Thanks, removing the shaper solved the problem! Returning to the older version also solved the problem.

[anthropoy]

Hey @bilgeyucel, I have also followed the tutorial. The output from
print(output["results"])

is empty though - []. The output object itself is not, it includes the answers in "documents".

[anthropoy]

Thanks @bilgeyucel, here's the output from -> pipe.run(query="How does Rhodes Statue look like?"). I didn't change the pipeline, I followed exactly what was in the tutorial.

{'results': [], 'invocation_context': {'query': 'How does Rhodes Statue look like?', 'documents': ['Within it, too, are to be seen large masses of rock, by the weight of which the artist steadied it while erecting it.[22][23]\nDestruction of the remains[edit]\nThe ultimate fate of the remains of the statue is uncertain. Rhodes has two serious earthquakes per century, owing to its location on the seismically unstable Hellenic Arc. Pausanias tells us, writing ca. 174, how the city was so devastated by an earthquake that the Sibyl oracle foretelling its destruction was considered fulfilled.[24] This means the statue could not have survived for long if it was ever repaired. By the 4th century Rhodes was Christianized, meaning any further maintenance or rebuilding, if there ever was any before, on an ancient pagan statue is unlikely. The metal would have likely been used for coins and maybe also tools by the time of the Arab wars, especially during earlier conflicts such as the Sassanian wars.[9]\nThe onset of Islamic naval incursions against the Byzantine empire gave rise to a dramatic account of what became of the Colossus. ', "Silver tetradrachm of Rhodes showing Helios and a rose (205–190 BC, 13.48 g)\nWhile scholars do not know what the statue looked like, they do have a good idea of what the head and face looked like, as it was of a standard rendering at the time. The head would have had curly hair with evenly spaced spikes of bronze or silver flame radiating, similar to the images found on contemporary Rhodian coins.[29]\n\nPossible locations[edit]\nThe old harbour entrance from inner embankment. The Fortress of St Nicholas is on right\nWhile scholars generally agree that anecdotal depictions of the Colossus straddling the harbour's entry point have no historic or scientific basis,[29] the monument's actual location remains a matter of debate. As mentioned above the statue is thought locally to have stood where two pillars now stand at the Mandraki port entrance.\nThe floor of the Fortress of St Nicholas, near the harbour entrance, contains a circle of sandstone blocks of unknown origin or purpose. "], 'results': []}, '_debug': {'prompt_node': {'runtime': {'prompts_used': []}, 'exec_time_ms': 0.33}}, 'documents': [<Document: {'content': 'Within it, too, are to be seen large masses of rock, by the weight of which the artist steadied it while erecting it.[22][23]\nDestruction of the remains[edit]\nThe ultimate fate of the remains of the statue is uncertain. Rhodes has two serious earthquakes per century, owing to its location on the seismically unstable Hellenic Arc. Pausanias tells us, writing ca. 174, how the city was so devastated by an earthquake that the Sibyl oracle foretelling its destruction was considered fulfilled.[24] This means the statue could not have survived for long if it was ever repaired. By the 4th century Rhodes was Christianized, meaning any further maintenance or rebuilding, if there ever was any before, on an ancient pagan statue is unlikely. The metal would have likely been used for coins and maybe also tools by the time of the Arab wars, especially during earlier conflicts such as the Sassanian wars.[9]\nThe onset of Islamic naval incursions against the Byzantine empire gave rise to a dramatic account of what became of the Colossus. ', 'content_type': 'text', 'score': 0.7399663750311847, 'meta': {'url': 'https://en.wikipedia.org/wiki/Colossus_of_Rhodes', '_split_id': 9}, 'id_hash_keys': ['content'], 'embedding': None, 'id': '7fc4286d8230b0749b219291c51a1426'}>, <Document: {'content': "Silver tetradrachm of Rhodes showing Helios and a rose (205–190 BC, 13.48 g)\nWhile scholars do not know what the statue looked like, they do have a good idea of what the head and face looked like, as it was of a standard rendering at the time. The head would have had curly hair with evenly spaced spikes of bronze or silver flame radiating, similar to the images found on contemporary Rhodian coins.[29]\n\nPossible locations[edit]\nThe old harbour entrance from inner embankment. The Fortress of St Nicholas is on right\nWhile scholars generally agree that anecdotal depictions of the Colossus straddling the harbour's entry point have no historic or scientific basis,[29] the monument's actual location remains a matter of debate. As mentioned above the statue is thought locally to have stood where two pillars now stand at the Mandraki port entrance.\nThe floor of the Fortress of St Nicholas, near the harbour entrance, contains a circle of sandstone blocks of unknown origin or purpose. ", 'content_type': 'text', 'score': 0.7248379215330418, 'meta': {'url': 'https://en.wikipedia.org/wiki/Colossus_of_Rhodes', '_split_id': 14}, 'id_hash_keys': ['content'], 'embedding': None, 'id': '4839aced58837e824cd75feb7836a2d9'}>], 'root_node': 'Query', 'params': {}, 'query': 'How does Rhodes Statue look like?', 'node_id': 'prompt_node'}

Regarding Discord, I've joined Discord but I couldn't seem find you on the member list, so I'm not sure how to post this question on #general and attention to you.

[anthropoy]

Interesting to note that in Tutorial: Customizing PromptNode for NLP Tasks, all the steps went fine until i came to Generating Titles for News Articles with a Custom Template. The line "prompt_node.prompt(prompt_template=title_generator, news=news)" will output an empty answer too. The same issue goes for the last task with the custom categorizing promptnode.

[anthropoy]

FYI, I have run it on Colab and the results are not empty. Very strange

[bilgeyucel]

Interesting. I'll let our team know about this 👍
I was wondering if there's a specific reason why you're running the tutorial line by line in a Python shell. It's harder for me 😅
Oh, and you can write to #haystack channel on discord. I or other people on the server will surely answer your question.

[anthropoy]

Well, I wanted to make sure that it can run locally on my machine so that I can deploy the actual application in the future. It's easier to read my local files as well.