Skip to content

RAG Chatbot on docs.md #160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 377 commits into
base: main
Choose a base branch
from
Open

RAG Chatbot on docs.md #160

wants to merge 377 commits into from

Conversation

Chrisyhjiang
Copy link
Contributor

@Chrisyhjiang Chrisyhjiang commented Jul 30, 2024 •
edited by github-actions bot
Loading

this is a RAG trained on pre-parsed and hard coded data on the docs.md. It Gives the 2 best responses within the database. Please take a look.

Samples Checklist

✅ All good!

raphaeltm and others added 30 commits May 29, 2024 13:05
@raphaeltm
fast api renamed
b3e4760
@raphaeltm
Merge pull request #56 from DefangLabs/rename-fastapi
8d51d64 
Rename fastapi
@Chrisyhjiang
updated the prod command
e6d5d0e
@Chrisyhjiang
Merge branch 'angular' of https://github.com/DefangLabs/samples into …
1cebf5b 
…angular
@raphaeltm
Merge branch 'main' into sample-repo-automation
fdc79ed
@raphaeltm
testrun
6f2dbb8
@raphaeltm
run another test
f9e5efd
@raphaeltm
change token
611b081
@raphaeltm
update subtree push
c9589b3
@raphaeltm
update subtree push
b6ee796
@Chrisyhjiang
took out unecessary Dockerfile.dev
ed187da
@Chrisyhjiang
took out the log files
a22a85f
@raphaeltm
update subtree push
7aa059b
@raphaeltm
update subtree push
8cd87ae
@raphaeltm
update subtree push
9fb82de
@raphaeltm
update subtree push
ce95916
@aRorschach
fastapi-postgres
14baf8c
@raphaeltm
split
fb18edd
@raphaeltm
trying something...
153810f
@raphaeltm
trying something...
f7d1afb
@raphaeltm
trying something...
3ae686d
@raphaeltm
success plz
d79ecef
@raphaeltm
success plz
6740878
@raphaeltm
this should be the last time...
225fd3c
@raphaeltm
this should be the last time...
2784d7e
@raphaeltm
this should be the last time...
53bb7df
@raphaeltm
Merge pull request #39 from DefangLabs/angular
8ddf730 
Angular Sample
@Chrisyhjiang
brought in main
0e6ebba
@Chrisyhjiang
Merge pull request #45 from DefangLabs/sailsjs-postgres
91b9cc9 
Sailsjs postgres
@aRorschach
remove .gitignore
1b117cd
@raphaeltm raphaeltm marked this pull request as draft July 31, 2024 22:06
@Chrisyhjiang
Merge branch 'main' of https://github.com/DefangLabs/samples into RAG
7903e7e
@Chrisyhjiang
README edits
6ccaeb9
@Chrisyhjiang
added some changes to make sure the context is correct
f3d8d39
@Chrisyhjiang
added a LRU cache for common queries
2022717
@Chrisyhjiang
added caching
aa6b16d
@Chrisyhjiang
improved UI
e2b9fad
@Chrisyhjiang
normalize query for caching and improved UI to match the main Defang …
39017f5 
…page
@Chrisyhjiang
README updates
d73fc30
@Chrisyhjiang
Merge branch 'main' of https://github.com/DefangLabs/samples into RAG
72ef6c2
@Chrisyhjiang
added md display as well as increased max tokens to 1024 to finish th…
f7fe8d8 
…e entire response
@Chrisyhjiang
fixed incorrect output
9e6c726
@Chrisyhjiang
enhanced knowledge base to parse directly from the docs repo with the…
0729785 
… markdown
@Chrisyhjiang
get knowledgebase implementation
f16fc46
@Chrisyhjiang
removed debugging
5eda5a2
@Chrisyhjiang
reparsed knowledge base and added in fuzzy matching logic with the ab…
e5ce451 
…out field in the json file to improve retrieval for context
@Chrisyhjiang
parsing logic
53229ba
@Chrisyhjiang
updated the knowledge base and rag system
592012b
@Chrisyhjiang
updated retrieval logic
ab26514
@Chrisyhjiang
testing
3e2be81
@Chrisyhjiang
added context limit
9b27e51
@Chrisyhjiang
changes to parsing and knowledge base
19b5ac6
@Chrisyhjiang
fixed some stuff
0bd647c
@Chrisyhjiang
Merge branch 'main' of https://github.com/DefangLabs/samples into RAG
f5abd01
@Chrisyhjiang
fixed typo and better UI (no typing when question being processed)
ea69b4c
@Chrisyhjiang
added deploy.yaml
2fe8de2
@raphaeltm raphaeltm marked this pull request as ready for review August 22, 2024 21:33
raphaeltm
raphaeltm reviewed Aug 30, 2024
View reviewed changes
samples/RAG-chatbot/README.md

## Configuration

- The knowledge base is acquired via parsing an sitemap located at "https://docs.defang.io/sitemap.xml".
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this accurate @Chrisyhjiang ? I thought you were using markdown files. Or is that just in the actual ask.defang.io implementation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that looks out of date, the docs chatbot has the final readme, I think it's because I never ended up editing and moving everything in the docs chatbot over to the real samples repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raphaeltm raphaeltm raphaeltm left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Chrisyhjiang @raphaeltm @aRorschach @lionello @Prakash-Sundaresan @edwardrf