Use json-repair package to fix LLM generated json

[willtai]

Description

This PR improves on fixing LLM-generated JSON objects for the EntityRelationExtractor with the following:

• Usage of the json-repair package to fix LLM generated JSON objects.
• Additional instructions to the LLM prompt.

The previous function to clean up LLM generated JSON objects is now removed.

This was tested using several LLMs using Ollama. During testing, the prompt and model temperature were changed to encourage the LLM to generate invalid JSON. These outputs were used to validate the new methods.

Type of Change

• New feature
• Bug fix
• Breaking change
• Documentation update
• Project configuration change

Complexity

Complexity: Low

How Has This Been Tested?

• Unit tests
• E2E tests
• Manual tests

Checklist

The following requirements should have been met (depending on the changes in the branch):

• Documentation has been updated
• Unit tests have been updated
• E2E tests have been updated
• Examples have been updated
• New files have copyright header
• CLA (https://neo4j.com/developer/cla/) has been signed
• CHANGELOG.md updated if appropriate

[stellasia]

Minor comments, but looks good! Thanks for adding this @willtai , I think it will really improve user experience!

[willtai]

@CodiumAI-Agent /update_changelog

[QodoAI-Agent]

Changelog updates: 🔄

2024-12-10

Added

• Integrated json-repair package to handle and repair invalid JSON generated by LLMs.
• Introduced InvalidJSONError exception for handling cases where JSON repair fails.

Changed

• Updated LLM prompts to include stricter instructions for generating valid JSON.
• Improved JSON handling logic in the EntityRelationExtractor component.

to commit the new content to the CHANGELOG.md file, please type:
'/update_changelog --pr_update_changelog.push_changelog_changes=true'

[QodoAI-Agent]

Changelog updates: 🔄

2024-12-10

Added

• Integrated json-repair package to handle and repair invalid JSON generated by LLMs.
• Introduced InvalidJSONError exception for cases where JSON repair fails.

Changed

• Enhanced LLM prompts with stricter instructions to ensure valid JSON generation.

to commit the new content to the CHANGELOG.md file, please type:
'/update_changelog --pr_update_changelog.push_changelog_changes=true'