Conversation
- Replace manual pydantic object construction (TestCase, InteractionSpec, Scenario) with fluent builder pattern scenario().interact().check() - Update all OSS documentation files including quickstart, core concepts, single-turn, multi-turn, custom checks, tutorials, and API reference - Add asyncio.run() disclaimers in quickstart and core concepts sections to guide users on running async code - Improve code examples consistency across all documentation
- Update all examples to use scenario() as primary API - Replace trace.interactions[-1] with trace.last in Python code and JSONPath keys - Standardize JSONPath keys to use trace.last.outputs pattern (all keys start with trace.) - Fix imports: use 'from giskard.checks import' instead of 'from giskard.checks.builtin import' - Clarify Scenario vs TestCase: minimize TestCase references (internal detail) - Update core concepts, quickstart, API reference, and all tutorials - Ensure consistency across single-turn, multi-turn, and custom checks documentation
Align quickstart examples with scenario terminology and rich result rendering, and add screenshots for simple and structured outputs.
Reframe core concepts around specs, interactions, and traces, and add a clear execution flow with fluent API mapping.
Co-authored-by: Henrique Chaves <44180294+henchaves@users.noreply.github.com>
Update OpenAI model examples to gpt-5-mini to avoid deprecated gpt-4o usage.
Remove notebook mention and keep script guidance for asyncio.run().
Refresh single-turn examples with high-risk scenarios and clearer guardrail rationale to emphasize production impact.
Reframe multi-turn examples around higher-risk workflows and add rationale notes to highlight guardrail failure points across turns.
Summary of ChangesHello @kevinmessiaen, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request represents a major update to Giskard, focusing on the introduction and comprehensive documentation of the new 'Giskard Checks' library. It involves upgrading the Python environment, streamlining project dependencies, and completely restructuring the documentation to provide clear, detailed guides for AI testing with the new framework. These changes aim to modernize the project's foundation and improve the user experience for developers working with Giskard's AI testing capabilities. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a major refactoring for "Giskard v3", focusing on a new "Checks" feature for testing AI applications. It adds extensive documentation for this new feature, including quickstarts, tutorials, and API references, while removing a significant amount of old documentation related to the previous SDK, RAGET, and LLM Scan features. The changes are substantial and well-structured. My review focuses on improving code quality, fixing errors in documentation examples, and addressing potential security and accessibility issues in the configuration.
| hooks: | ||
| - id: detect-secrets | ||
| args: ["--baseline", ".secrets.baseline"] | ||
| args: ["--baseline", ".secrets.baseline", "--exclude-secrets", "your-api-key"] |
There was a problem hiding this comment.
The placeholder "your-api-key" has been added to the detect-secrets configuration. This is a significant security risk as it might be overlooked and could lead to real secrets being excluded from scans if this pattern is copied. This placeholder should be removed. If a specific secret needs to be excluded, it should be done using its ID or a more specific regex.
args: ["--baseline", ".secrets.baseline"]There was a problem hiding this comment.
"your-api-key" is not a real api key and is used in example code.
| docs_version = os.getenv("READTHEDOCS_VERSION", "latest") | ||
| if docs_version == "latest" or docs_version == "stable": | ||
| branch = "main" | ||
| else: | ||
| branch = docs_version.replace("-", "/") | ||
| branch = "main" |
There was a problem hiding this comment.
The branch variable is unconditionally set to "main" on line 141, which makes the preceding if/else block that also sets branch redundant. This dead code should be removed to improve clarity.
# Do not execute the notebooks when building the docs
branch = "main"| from sphinx.environment.adapters.toctree import TocTree | ||
| from sphinx import addnodes |
There was a problem hiding this comment.
The imports for TocTree and addnodes are inside a nested function toctree_from_doc. According to PEP 8, imports should usually be at the top of the file. While this might be done to avoid polluting the global namespace, it's better to move them to the top of conf.py for consistency and readability.
Refresh check references, expand comparison checks, and add semantic similarity details. Co-authored-by: Cursor <cursoragent@cursor.com>
2 participants
No description provided.