Skip to content

Comments

🚀 Feat: seeders blocks + launch validation#27

Merged
nicofretti merged 20 commits intodevelopfrom
26-feat-seeders-blocks-launch-validation
Nov 11, 2025
Merged

🚀 Feat: seeders blocks + launch validation#27
nicofretti merged 20 commits intodevelopfrom
26-feat-seeders-blocks-launch-validation

Conversation

@nicofretti
Copy link
Owner

@nicofretti nicofretti commented Nov 4, 2025

Description

Find it in the related issue. Key changes:

  • Added MarkdownMultiplierBlock using LlamaIndex for markdown parsing and chunking
  • Introduced BaseMultiplierBlock abstract class and dual-mode pipeline execution logic
  • Enhanced seed validation API endpoint with template variable extraction
  • Updated Q&A Generation template to process markdown documents via chunking
  • Modified frontend to support markdown file uploads and real-time seed validation

Related Issue

Closes #26

Checklist

  • Code follows project style guidelines
  • Comments explain "why" not "what"
  • Documentation updated (if needed)
  • No debug code or console statements
  • make format passes
  • make pre-merge passes
  • PR update from develop branch
  • Copilot review run and addressed

@nicofretti nicofretti linked an issue Nov 4, 2025 that may be closed by this pull request
@nicofretti nicofretti changed the base branch from main to develop November 4, 2025 19:19
@nicofretti nicofretti self-assigned this Nov 4, 2025
@nicofretti nicofretti added the enhancement New feature or request label Nov 4, 2025
@nicofretti nicofretti marked this pull request as ready for review November 7, 2025 18:25
@nicofretti nicofretti requested a review from Copilot November 7, 2025 18:25
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request introduces a Markdown Multiplier Block that enables chunk-based processing of markdown documents, transforming single markdown files into multiple pipeline executions for each chunk. The key architectural change allows pipelines to have two execution modes: traditional single-input/single-output and multiplier fan-out mode.

Key Changes:

  • Added MarkdownMultiplierBlock using LlamaIndex for markdown parsing and chunking
  • Introduced BaseMultiplierBlock abstract class and dual-mode pipeline execution logic
  • Enhanced seed validation API endpoint with template variable extraction
  • Updated Q&A Generation template to process markdown documents via chunking
  • Modified frontend to support markdown file uploads and real-time seed validation

Reviewed Changes

Copilot reviewed 42 out of 42 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
lib/blocks/builtin/markdown_multiplier.py New multiplier block for markdown chunking with configurable parser types
lib/blocks/base.py Added BaseMultiplierBlock base class for fan-out blocks
lib/workflow.py Implemented dual execution modes with _execute_multiplier_pipeline method
lib/job_processor.py Added conditional logic to handle multiplier vs normal pipeline execution
app.py Added /api/seeds/validate endpoint and markdown file support in /generate
models.py Added SeedValidationRequest model for validation API
tests/blocks/test_markdown_multiplier.py Comprehensive test suite for new multiplier block
tests/test_workflow.py Added tests for multiplier validation rules and execution
tests/test_api.py Added extensive seed validation endpoint test coverage
lib/templates/qa_generation.yaml Updated to use markdown multiplier with chunk-based generation
lib/templates/__init__.py Enhanced template loader to support markdown seed files
lib/blocks/builtin/text_generator.py Added get_required_fields method for template variable extraction
lib/blocks/builtin/structured_generator.py Renamed prompt parameter to user_prompt with template variable extraction
frontend/src/pages/Generator.tsx Added markdown file support, validation UI, and multiplier pipeline detection
frontend/src/components/pipeline-editor/BlockPalette.tsx Updated to use backend-provided category field with new "seeders" category
frontend/src/components/pipeline-editor/BlockNode.tsx Updated block categorization to use category field from block data
frontend/src/pages/Pipelines.tsx Added markdown seed download support for templates
pyproject.toml Added llama-index-core>=0.14.7 dependency
docs/templates.md Updated Q&A Generation documentation to reflect markdown chunking
.github/ISSUE_TEMPLATE/* Cleaned up comment formatting in issue templates

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 14 to 15
2.
3.
Copy link

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing step 1 in reproduction steps. The list should start with "1." before step 2.

Suggested change
2.
3.
1.
2.

Copilot uses AI. Check for mistakes.
}
};
revalidate();
}, [selectedPipeline]);
Copy link

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This effect re-runs whenever selectedPipeline changes and attempts to revalidate the file. However, it doesn't include file in the dependency array, which could lead to stale closure issues. The effect uses file but doesn't list it as a dependency. This is a React Hook dependency warning that should be addressed by either adding file to the dependencies or using a different approach.

Suggested change
}, [selectedPipeline]);
}, [selectedPipeline, file]);

Copilot uses AI. Check for mistakes.
Comment on lines 108 to 109
except Exception:
pass
Copy link

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
except Exception:
pass
except Exception as e:
logger.warning(f"Failed to parse Jinja2 template for required fields: {e} (prompt: {prompt!r})")

Copilot uses AI. Check for mistakes.
@nicofretti nicofretti requested a review from Copilot November 8, 2025 11:31
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 46 out of 46 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nicofretti nicofretti requested a review from Copilot November 11, 2025 19:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 57 out of 57 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

app.py:1

  • Line 238: seed.get('repetitions', 1) is called twice in the isinstance check. The second call should be seed.get('repetitions') without the default value of 1. Current code: isinstance(seed.get('repetitions', 1), int) will always return True because if 'repetitions' is missing, it defaults to 1 (an int), masking potential type errors. Should be: isinstance(seed.get('repetitions'), int) if 'repetitions' in seed else True
import json

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nicofretti nicofretti merged commit d5f6ab4 into develop Nov 11, 2025
8 checks passed
@nicofretti nicofretti deleted the 26-feat-seeders-blocks-launch-validation branch November 15, 2025 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🚀 Feat: seeders blocks + launch validation

1 participant