(feat) Add initial documentation and prompt management API #40
Conversation
|
@danwritecode Needs a review |
There was a problem hiding this comment.
What's here seems reasonable so far but we need the migration scripts to create or update the relevant tables. Llmkit uses sqlite in prod with sqlx migration scripts. So whatever changes you want to make need to be a new migration script that will get applied at startup by the application.
For development, there are some sqlx cli commands that you can run to create and run migrations.
|
Sure I'll get on that |
|
Also, for docs, we actually have a shitty version of them on llmkit.xyz. i can port your doc changes over to there. For any core stuff we can add it to the README |
|
Hey Dan, thanks for getting back to me. Can you link the migration scripts to create or update the relevant tables? I can't seem to find them. Also, can you review issue #41? I would love your thoughts. I'll be working on this project long-term; I see tonnes of potential. I also wanted to ask if I could be responsible for prompt management and this feature set. I wanted more experience in the field. Thanks. |
…tory_id reference in prompts
|
@danwritecode got it. I think this ready to merge. |
let me take a look and we'll get it merged brother. i'm gearing up to release eureka in a few days here, so I can't promise it'll be this week. but i'll get it merged early next week. I appreciate your hard work. Also, DM me on discord and lets discuss your other ideas and what things you'd like to own. I want to align on that and set some direction. |
|
No problem, and thanks for the heads up. |
There was a problem hiding this comment.
A few things:
- I am hesitant to merge these changes without the accompanying UI changes. Scaffolding the controllers and DB queries is not necessarily the feature, the feature is is the full feature.
- There are dozens of compiler errors that should have been caught before this PR was opened. You need to build and test the code at the very least before opening a PR.
error: error returned from database: (code: 1) no such table: prompt_directory
--> src/db/prompts.rs:563:19
|
563 | let rec = sqlx::query!(
| ___________________^
564 | | r#"INSERT INTO prompt_directory (name, parent_id) VALUES (?, ?)"#,
565 | | name,
566 | | parent_id
567 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_directory
--> src/db/prompts.rs:574:19
|
574 | let rec = sqlx::query_as!(PromptDirectory,
| ___________________^
575 | | r#"SELECT id, name, parent_id, created_at, updated_at FROM prompt_directory WHERE id = ?"#,
576 | | id
577 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery_as` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_directory
--> src/db/prompts.rs:584:20
|
584 | let recs = sqlx::query_as!(PromptDirectory,
| ____________________^
585 | | r#"SELECT id, name, parent_id, created_at, updated_at FROM prompt_directory WHERE parent_id IS ?"#,
586 | | parent_id
587 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery_as` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_directory
--> src/db/prompts.rs:594:20
|
594 | let rows = sqlx::query!(
| ____________________^
595 | | r#"UPDATE prompt_directory SET name = ?, parent_id = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ?"#,
596 | | name, parent_id, id
597 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_directory
--> src/db/prompts.rs:605:20
|
605 | let rows = sqlx::query!(
| ____________________^
606 | | r#"DELETE FROM prompt_directory WHERE id = ?"#,
607 | | id
608 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_component
--> src/db/prompts.rs:617:19
|
617 | let rec = sqlx::query!(
| ___________________^
618 | | r#"INSERT INTO prompt_component (name, content, description) VALUES (?, ?, ?)"#,
619 | | name, content, description
620 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_component
--> src/db/prompts.rs:627:19
|
627 | let rec = sqlx::query_as!(PromptComponent,
| ___________________^
628 | | r#"SELECT id, name, content, description, created_at, updated_at FROM prompt_component WHERE id = ?"#,
629 | | id
630 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery_as` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_component
--> src/db/prompts.rs:637:20
|
637 | let recs = sqlx::query_as!(PromptComponent,
| ____________________^
638 | | r#"SELECT id, name, content, description, created_at, updated_at FROM prompt_component ORDER BY created_at DESC"#
639 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery_as` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_component
--> src/db/prompts.rs:646:20
|
646 | ... let rows = sqlx::query!(
| __________________^
647 | | ... r#"UPDATE prompt_component SET name = ?, content = ?, description = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ?"#,
648 | | ... name, content, description, id
649 | | ... )
| |_______^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_component
--> src/db/prompts.rs:657:20
|
657 | let rows = sqlx::query!(
| ____________________^
658 | | r#"DELETE FROM prompt_component WHERE id = ?"#,
659 | | id
660 | | )
| |_________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery` (in Nightly builds, run with -Z macro-backtrace for more info)
error: error returned from database: (code: 1) no such table: prompt_component
--> src/db/prompts.rs:678:28
|
678 | let comp = sqlx::query!(
| ____________________________^
679 | | r#"SELECT content FROM prompt_component WHERE name = ? LIMIT 1"#,
680 | | comp_name
681 | | )
| |_________________^
|
= note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `sqlx::qu
ery` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0432]: unresolved imports `crate::db::types::models::Prompt`, `crate::db::types::models::PromptComponent`, `crate::db::types::mod
els::PromptDirectory`
--> src/db/models.rs:2:50
|
2 | use crate::db::types::models::{ModelProviderRow, Prompt, PromptComponent, PromptDirectory};
| ^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ no `PromptDirectory` in `db::types::models
`
| | |
| | no `PromptComponent` in `db::types::models`
| no `Prompt` in `db::types::models`
|
= help: consider importing this enum instead:
async_openai::types::Prompt
error[E0432]: unresolved import `regex`
--> src/db/prompts.rs:669:13
|
669 | use regex::Regex;
| ^^^^^ use of undeclared crate or module `regex`
error[E0412]: cannot find type `PromptDirectory` in this scope
--> src/db/prompts.rs:573:65
|
573 | pub async fn get_directory(&self, id: i64) -> Result<Option<PromptDirectory>> {
| ^^^^^^^^^^^^^^^ not found in this scope
|
help: consider importing this struct
|
1 + use crate::db::models::PromptDirectory;
|
error[E0412]: cannot find type `PromptDirectory` in this scope
--> src/db/prompts.rs:583:80
|
583 | pub async fn list_directories(&self, parent_id: Option<i64>) -> Result<Vec<PromptDirectory>> {
| ^^^^^^^^^^^^^^^ not found in this scope
|
help: consider importing this struct
|
1 + use crate::db::models::PromptDirectory;
|
error[E0412]: cannot find type `PromptComponent` in this scope
--> src/db/prompts.rs:626:65
|
626 | pub async fn get_component(&self, id: i64) -> Result<Option<PromptComponent>> {
| ^^^^^^^^^^^^^^^ not found in this scope
|
help: consider importing this struct
|
1 + use crate::db::models::PromptComponent;
|
error[E0412]: cannot find type `PromptComponent` in this scope
--> src/db/prompts.rs:636:55
|
636 | pub async fn list_components(&self) -> Result<Vec<PromptComponent>> {
| ^^^^^^^^^^^^^^^ not found in this scope
|
help: consider importing this struct
|
1 + use crate::db::models::PromptComponent;
|
error[E0308]: mismatched types
--> src/db/prompts.rs:698:16
|
698 | if let Some(system) = &prompt.system {
| ^^^^^^^^^^^^ -------------- this expression has type `&std::string::String`
| |
| expected `String`, found `Option<_>`
|
= note: expected struct `std::string::String`
found enum `std::option::Option<_>`
error[E0308]: mismatched types
--> src/db/prompts.rs:699:29
|
699 | prompt.system = Some(self.resolve_components_in_text(system).await?);
| ------------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `String`, found `Option<String>`
| |
| expected due to the type of this binding
|
= note: expected struct `std::string::String`
found enum `std::option::Option<std::string::String>`
| ); | ||
|
|
||
| -- Add directory_id to prompts table | ||
| ALTER TABLE prompts ADD COLUMN directory_id INTEGER REFERENCES prompt_directories(id) ON DELETE SET NULL; |
There was a problem hiding this comment.
This statement is invalid, prompt is the name of the table not prompts
backend/src/controllers/prompts.rs
Outdated
| Ok(Json(deleted)) | ||
| } | ||
|
|
||
| // --- Prompt Component API --- |
There was a problem hiding this comment.
directories and components should be their own distinct and separate controllers.
backend/src/db/prompts.rs
Outdated
| } | ||
|
|
||
| // --- Prompt Component CRUD --- | ||
| pub async fn create_component(&self, name: &str, content: &str, description: Option<&str>) -> Result<i64> { |
There was a problem hiding this comment.
same thing here, directories and components should be their own distinct DB repos.
docs/index.md
Outdated
| @@ -0,0 +1,130 @@ | |||
| # llmkit Documentation | |||
There was a problem hiding this comment.
I like the idea of this, but I want the README to remain the primary point of documentation for now. We also have docs on llmkit.xyz that need to be updated an maintained. I don't want to add another point of documentation here.
There was a problem hiding this comment.
I like the idea of this, but I want the README to remain the primary point of documentation for now. We also have docs on llmkit.xyz that need to be updated an maintained. I don't want to add another point of documentation here.
So do I just redo the code and then remove the docs?
|
Oh, thanks. I'll check out. Sorry about that. |
|
Hey @danwritecode, I've been working on that bug for a while. I can't seem to figure it out. I'm going to create an adjacent issue to try and bring attention to it, but it's giving me quite a lot of trouble. |
Prompt Directories, Search, Pagination, and Componentization
what issues this pull request solves
#38
#39
Overview
This PR introduces several improvements to the prompt management system:
What Changed
What to Expect