fix(new-webui): Add support for querying metadata from multiple datasets (fixes #1024). #1042
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,14 @@ | ||||||||||||||||||||||||||||||||||
| import {QueryClient} from "@tanstack/react-query"; | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| const DEFAULT_REFETCH_INTERVAL_MILLIS = 10_000; | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| const queryClient = new QueryClient({ | ||||||||||||||||||||||||||||||||||
| defaultOptions: { | ||||||||||||||||||||||||||||||||||
| queries: { | ||||||||||||||||||||||||||||||||||
| refetchInterval: DEFAULT_REFETCH_INTERVAL_MILLIS, | ||||||||||||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||||||||||||
| }); | ||||||||||||||||||||||||||||||||||
|
Comment on lines
+6
to
+12
Contributor
There was a problem hiding this comment. 🧹 Nitpick (assertive) Well-structured QueryClient configuration The centralized configuration approach is excellent for maintaining consistency across the application. The current minimal configuration focusing on refetch interval is appropriate for the current refactoring. Future enhancements could include additional default options such as retry logic, cache time, or stale time if needed: const queryClient = new QueryClient({
defaultOptions: {
queries: {
refetchInterval: DEFAULT_REFETCH_INTERVAL_MILLIS,
+ retry: 3,
+ staleTime: 5_000,
},
},
});📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| export default queryClient; | ||||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,104 +1,74 @@ | ||||||||||||||
| import {useQuery} from "@tanstack/react-query"; | ||||||||||||||
| import dayjs from "dayjs"; | ||||||||||||||
|
|
||||||||||||||
| import { | ||||||||||||||
| CLP_STORAGE_ENGINES, | ||||||||||||||
| SETTINGS_STORAGE_ENGINE, | ||||||||||||||
| } from "../../../config"; | ||||||||||||||
| import {fetchDatasetNames} from "../../SearchPage/SearchControls/Dataset/sql"; | ||||||||||||||
|
Contributor
There was a problem hiding this comment. 💡 Verification agent 🧩 Analysis chainFix the unresolved import path. The ESLint error indicates that the import path 🏁 Script executed: #!/bin/bash
# Verify the import path exists and check the correct relative path
fd -t f "sql.ts" | grep -E "(SearchPage|Dataset)" | head -10Length of output: 146 #!/bin/bash
# Locate the IngestPage Details index file to confirm its path
fd -t f index.tsx | grep -E "IngestPage/Details"🏁 Script executed: #!/bin/bash
# Find the path of the IngestPage Details index file and verify the import target
file=$(fd "Details/index.tsx" | head -n1)
echo "Found index.tsx at: $file"
dir=$(dirname "$file")
import_rel="../../SearchPage/SearchControls/Dataset/sql.ts"
echo "Checking existence of import target: $dir/$import_rel"
if [ -f "$dir/$import_rel" ]; then
echo "✅ Import target exists at $dir/$import_rel"
else
echo "❌ Import target missing at $dir/$import_rel"
fi
# List all sql.ts occurrences for context
echo "All sql.ts files in project:"
fd -t f "sql.ts"Length of output: 1036 🏁 Script executed: #!/bin/bash
# 1. Locate the IngestPage Details index file
echo "Searching for Details/index.tsx…"
fd -t f index.tsx | grep -E "IngestPage/Details"
# 2. Find where fetchDatasetNames is defined in the repo
echo "Looking for fetchDatasetNames definitions…"
rg "fetchDatasetNames" -n .Length of output: 1088 #!/bin/bash
# Display the top of Details/sql.ts
echo "=== components/webui/client/src/pages/IngestPage/Details/sql.ts ==="
sed -n '1,20p' "$(fd -t f sql.ts | grep 'webui/client/src/pages/IngestPage/Details/sql.ts')"
# Display the top of api/sql.ts to see if fetchDatasetNames is exported
echo -e "\n=== components/webui/client/src/api/sql.ts ==="
sed -n '1,20p' "$(fd -t f sql.ts | grep 'webui/client/src/api/sql.ts')"#!/bin/bash
# Locate and display the Details/sql.ts file
details_sql=$(fd -t f sql.ts | grep 'webui/client/src/pages/IngestPage/Details/sql.ts')
echo "=== $details_sql ==="
sed -n '1,50p' "$details_sql"
# Locate and inspect client api/sql.ts for fetchDatasetNames export
api_sql=$(fd -t f sql.ts | grep 'webui/client/src/api/sql.ts')
echo -e "\n=== $api_sql ==="
grep -R "export.*fetchDatasetNames" -n "$api_sql" || echo "No fetchDatasetNames export found in api/sql.ts"
# Search all of webui/client for fetchDatasetNames occurrences
echo -e "\n=== All fetchDatasetNames in webui/client ==="
grep -R "fetchDatasetNames" -n components/webui/client/srcFix broken import path for fetchDatasetNames The import import {fetchDatasetNames} from "../../SearchPage/SearchControls/Dataset/sql";does not resolve because there is no • File: Suggested diff: - import {fetchDatasetNames} from "../../SearchPage/SearchControls/Dataset/sql";
+ import {fetchDatasetNames} from "../../../api/sql";Adjust the path according to where 🧰 Tools🪛 GitHub Actions: clp-lint[error] 8-8: ESLint: Unable to resolve path to module '../../SearchPage/SearchControls/Dataset/sql' (import/no-unresolved) 🤖 Prompt for AI Agents |
||||||||||||||
| import Files from "./Files"; | ||||||||||||||
| import styles from "./index.module.css"; | ||||||||||||||
| import Messages from "./Messages"; | ||||||||||||||
| import { | ||||||||||||||
| DETAILS_DEFAULT, | ||||||||||||||
| fetchClpDetails, | ||||||||||||||
| fetchClpsDetails, | ||||||||||||||
| } from "./sql"; | ||||||||||||||
| import TimeRange from "./TimeRange"; | ||||||||||||||
|
|
||||||||||||||
|
|
||||||||||||||
| /** | ||||||||||||||
| * Renders grid with compression details. | ||||||||||||||
| * | ||||||||||||||
| * @return | ||||||||||||||
| */ | ||||||||||||||
| const Details = () => { | ||||||||||||||
| const {data: datasetNames = [], isSuccess: isSuccessDatasetNames} = useQuery({ | ||||||||||||||
| queryKey: ["datasets"], | ||||||||||||||
| queryFn: fetchDatasetNames, | ||||||||||||||
|
Contributor
There was a problem hiding this comment. 💡 Verification agent 🧩 Analysis chainAddress the unsafe assignment warning. The TypeScript ESLint error suggests there's an unsafe assignment of an error-typed value. Ensure that the 🏁 Script executed: #!/bin/bash
# Check the return type of fetchDatasetNames function
ast-grep --pattern 'export $_ fetchDatasetNames'Length of output: 50 🏁 Script executed: #!/bin/bash
# Locate all references and the definition of fetchDatasetNames
rg 'fetchDatasetNames' -nLength of output: 792 🏁 Script executed: #!/bin/bash
# Display lines around the usage of fetchDatasetNames in the Details page
sed -n '1,200p' components/webui/client/src/pages/IngestPage/Details/index.tsxLength of output: 2314 Specify useQuery generics to avoid unsafe The • In - const { data: datasetNames = [], isSuccess: isSuccessDatasetNames } = useQuery({
+ const { data: datasetNames = [], isSuccess: isSuccessDatasetNames } = useQuery<string[], Error>({
queryKey: ["datasets"],
queryFn: fetchDatasetNames,
enabled: CLP_STORAGE_ENGINES.CLP_S === SETTINGS_STORAGE_ENGINE,
});Optionally, apply the same change in This ensures 📝 Committable suggestion
Suggested change
🧰 Tools🪛 GitHub Actions: clp-lint[error] 28-28: ESLint: Unsafe assignment of an error typed value (@typescript-eslint/no-unsafe-assignment) 🤖 Prompt for AI Agents |
||||||||||||||
| enabled: CLP_STORAGE_ENGINES.CLP_S === SETTINGS_STORAGE_ENGINE, | ||||||||||||||
| }); | ||||||||||||||
|
|
||||||||||||||
| const {data: details = DETAILS_DEFAULT, isPending} = useQuery({ | ||||||||||||||
| queryKey: [ | ||||||||||||||
| "details", | ||||||||||||||
| datasetNames, | ||||||||||||||
| ], | ||||||||||||||
| queryFn: async () => { | ||||||||||||||
| if (CLP_STORAGE_ENGINES.CLP === SETTINGS_STORAGE_ENGINE) { | ||||||||||||||
| return fetchClpDetails(); | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| return fetchClpsDetails(datasetNames); | ||||||||||||||
| }, | ||||||||||||||
| enabled: CLP_STORAGE_ENGINES.CLP === SETTINGS_STORAGE_ENGINE || isSuccessDatasetNames, | ||||||||||||||
| }); | ||||||||||||||
|
|
||||||||||||||
| if (CLP_STORAGE_ENGINES.CLP === SETTINGS_STORAGE_ENGINE) { | ||||||||||||||
| return ( | ||||||||||||||
| <div className={styles["detailsGrid"]}> | ||||||||||||||
| <div className={styles["timeRange"]}> | ||||||||||||||
| <TimeRange | ||||||||||||||
| beginDate={dayjs.utc(details.begin_timestamp)} | ||||||||||||||
| endDate={dayjs.utc(details.end_timestamp)} | ||||||||||||||
| isLoading={isPending}/> | ||||||||||||||
| </div> | ||||||||||||||
| <Messages | ||||||||||||||
| isLoading={isPending} | ||||||||||||||
| numMessages={details.num_messages}/> | ||||||||||||||
| <Files | ||||||||||||||
| isLoading={isPending} | ||||||||||||||
| numFiles={details.num_files}/> | ||||||||||||||
| </div> | ||||||||||||||
| ); | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| return ( | ||||||||||||||
| <div> | ||||||||||||||
| <TimeRange | ||||||||||||||
| beginDate={dayjs.utc(details.begin_timestamp)} | ||||||||||||||
| endDate={dayjs.utc(details.end_timestamp)} | ||||||||||||||
| isLoading={isPending}/> | ||||||||||||||
| </div> | ||||||||||||||
| ); | ||||||||||||||
|
Comment on lines
67
to
73
Contributor
There was a problem hiding this comment. Quick question: I might be missing some context here. Why don’t we show Messages and Files for CLP-s? @davemarco
Member
There was a problem hiding this comment. The clp-s storage engine doesn't currently keep track of the number of log events and files in the archives. |
||||||||||||||
| }; | ||||||||||||||
|
|
||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Nitpick (assertive)
Consider if 10-second refetch interval is appropriate for all queries
The constant is well-named and formatted correctly. However, a 10-second global refetch interval might be aggressive for all query types. Consider if different query types might benefit from different intervals.
🤖 Prompt for AI Agents