Kl/scrum-132 Improved querying of year level and breadth requirement from vector db

[kevin-lann]

Description

SCRUM-132

This PR Includes:

• Documented /api/ai endpoints
• Vastly improved chatbot's ability to answer queries involving course year level and breadth requirement.
• Updated embeddings script for storing metadata in course embeddings

Type of change

• New feature (non-breaking change which adds functionality)
• [] Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings

[minhhaitran08]

I have tested the breadth requirement and year level with Morpheus.

I think the year level suggestion are pretty good and I dont find any problems there.

But for the breadth requirement

• When i ask in general course suggestion to fulfil a breadth requirement, Morpheus could provide accurate answer. But when I ask for a more narrowed suggestion than it starts to put out inaccurate suggestion

  • For example: In this case MGTA02 breath requirement is Social and Behavioural Sciences
    

• Furthermore Morpheus also seems to not being able to correctly identify the course information based on the provided course code --> Incorrect breadth requirement
  

[kevin-lann]

@minhhaitran08 Fixed the issues. The similarity search was previously failing to find the right course in some queries. Let me know if it still persists. I retried some of your queries and they are good now:

[minhhaitran08]

I have reviewed your PR again and the accuracy have been improved. Most of the course I search are accurate, although there are still some mistakes here and there especially for the upper year courses. I tried 10 course 8 of them are correct.

The recommendations are also accurate majority of time.

Another feedback I think we should implement if we can is instead of breadth requirement code we should write its name. For example Quantitative Reasoning instead of QUANT or Social and Behavioural Sciences instead of SOCIAL_SCI