promptify v3.0

Author: monk1337

.DS_Store

@@ -5,7 +5,7 @@
 <h2 align="center">Promptify</h2> -->
 
 <p align="center">
-  <p align="center">Prompt Engineering, Solve NLP Problems with LLM's & Easily generate different NLP Task prompts for popular generative models like GPT, PaLM, and more with Promptify
+  <p align="center">Task-based NLP engine with Pydantic structured outputs, built-in evaluation, and LiteLLM as the universal LLM backend. Think "scikit-learn for LLM-powered NLP".
 </p>
 </p>
 
@@ -33,104 +33,149 @@
 
 ### With pip
 
-This repository is tested on Python 3.7+, openai 0.25+.
-
-You should install Promptify using Pip command
+Requires Python 3.9+.
 
 ```bash
-pip3 install promptify
+pip install promptify
 ```
 
 or
 
 ```bash
-pip3 install git+https://github.com/promptslab/Promptify.git
+pip install git+https://github.com/promptslab/Promptify.git
+```
+
+For evaluation metrics support:
+```bash
+pip install promptify[eval]
 ```
 
-## Quick tour
+## Quick Tour
 
-To immediately use a LLM model for your NLP task, we provide the `Pipeline` API.
+### 3-Line NER
 
 ```python
-from promptify import Prompter,OpenAI, Pipeline
+from promptify import NER
 
-sentence     =  """The patient is a 93-year-old female with a medical  				 
-                history of chronic right hip pain, osteoporosis,					
-                hypertension, depression, and chronic atrial						
-                fibrillation admitted for evaluation and management				
-                of severe nausea and vomiting and urinary tract				
-                infection"""
+ner = NER(model="gpt-4o-mini", domain="medical")
+result = ner("The patient is a 93-year-old female with a medical history of chronic right hip pain, osteoporosis, hypertension, depression, and chronic atrial fibrillation admitted for evaluation and management of severe nausea and vomiting and urinary tract infection")
+```
 
-model        = OpenAI(api_key) # or `HubModel()` for Huggingface-based inference or 'Azure' etc
-prompter     = Prompter('ner.jinja') # select a template or provide custom template
-pipe         = Pipeline(prompter , model)
+**Output:**
+```python
+NERResult(entities=[
+    Entity(text="93-year-old", label="AGE"),
+    Entity(text="chronic right hip pain", label="CONDITION"),
+    Entity(text="osteoporosis", label="CONDITION"),
+    Entity(text="hypertension", label="CONDITION"),
+    Entity(text="depression", label="CONDITION"),
+    Entity(text="chronic atrial fibrillation", label="CONDITION"),
+    Entity(text="severe nausea and vomiting", label="SYMPTOM"),
+    Entity(text="urinary tract infection", label="CONDITION"),
+])
+```
 
+### Classification
 
-result = pipe.fit(sentence, domain="medical", labels=None)
+```python
+from promptify import Classify
+
+clf = Classify(model="gpt-4o-mini", labels=["positive", "negative", "neutral"])
+result = clf("Amazing product! Best purchase I've ever made.")
+# Classification(label="positive", confidence=0.95)
+```
 
+### Question Answering
 
-### Output
+```python
+from promptify import QA
 
-[
-    {"E": "93-year-old", "T": "Age"},
-    {"E": "chronic right hip pain", "T": "Medical Condition"},
-    {"E": "osteoporosis", "T": "Medical Condition"},
-    {"E": "hypertension", "T": "Medical Condition"},
-    {"E": "depression", "T": "Medical Condition"},
-    {"E": "chronic atrial fibrillation", "T": "Medical Condition"},
-    {"E": "severe nausea and vomiting", "T": "Symptom"},
-    {"E": "urinary tract infection", "T": "Medical Condition"},
-    {"Branch": "Internal Medicine", "Group": "Geriatrics"},
-]
- 
+qa = QA(model="gpt-4o-mini")
+answer = qa("Einstein was born in Ulm in 1879.", question="Where was Einstein born?")
+# Answer(answer="Ulm", evidence="Einstein was born in Ulm", confidence=0.98)
 ```
 
-<p float="left">
-  <img src="https://raw.githubusercontent.com/promptslab/Promptify/main/assets/ner.png" width="250" />
-  <img src="https://raw.githubusercontent.com/promptslab/Promptify/main/assets/multilabel.png" width="250" /> 
-  <img src="https://raw.githubusercontent.com/promptslab/Promptify/main/assets/qa_gen.png" width="250" />
-</p>
-<h4 align="center">GPT-3 Example with NER, MultiLabel, Question Generation Task</h3>
+### Custom Task with Any Pydantic Schema
+
+```python
+from promptify import Task
+from pydantic import BaseModel
 
+class MovieReview(BaseModel):
+    sentiment: str
+    rating: float
+    key_themes: list[str]
 
-<h2>Features 🎮 </h2>
-<ul>
-  <li> Perform NLP tasks (such as NER and classification) in just 2 lines of code, with no training data required</li>
-  <li> Easily add one shot, two shot, or few shot examples to the prompt</li>
-  <li> Handling out-of-bounds prediction from LLMS (GPT, t5, etc.)</li>
-  <li> Output always provided as a Python object (e.g. list, dictionary) for easy parsing and filtering. This is a major advantage over LLMs generated output, whose unstructured and raw output makes it difficult to use in business or other applications.</li>
-  <li> Custom examples and samples can be easily added to the prompt</li>
-  <li> 🤗 Run inference on any model stored on the Huggingface Hub (see <a href="https://github.com/promptslab/Promptify/blob/main/notebooks/huggingface.ipynb">notebook guide</a>).</li>
-  <li> Optimized prompts to reduce OpenAI token costs (coming soon)</li>
-</ul>
+task = Task(model="gpt-4o", output_schema=MovieReview, instruction="Analyze this movie review.")
+review = task("Nolan's best work. Stunning visuals but the plot drags.")
+# MovieReview(sentiment="mostly positive", rating=7.5, key_themes=["visuals", "pacing"])
+```
 
+### Any Provider - Just Change the Model String
 
-### Supporting wide-range of Prompt-Based NLP tasks :
+```python
+ner_openai = NER(model="gpt-4o-mini")
+ner_claude = NER(model="claude-sonnet-4-20250514")
+ner_local  = NER(model="ollama/llama3")
+```
 
-| Task Name | Colab Notebook | Status |
-|-------------|-------|-------|
-| Named Entity Recognition | [NER Examples with GPT-3](https://colab.research.google.com/drive/16DUUV72oQPxaZdGMH9xH1WbHYu6Jqk9Q?usp=sharing) | ✅  |
-| Multi-Label Text Classification | [Classification Examples with GPT-3](https://colab.research.google.com/drive/1gNqDxNyMMUO67DxigzRAOa7C_Tcr2g6M?usp=sharing) | ✅    |
-| Multi-Class Text Classification | [Classification Examples with GPT-3](https://colab.research.google.com/drive/1gNqDxNyMMUO67DxigzRAOa7C_Tcr2g6M?usp=sharing) | ✅    |
-| Binary Text Classification  | [Classification Examples with GPT-3](https://colab.research.google.com/drive/1gNqDxNyMMUO67DxigzRAOa7C_Tcr2g6M?usp=sharing) | ✅    |
-| Question-Answering | [QA Task Examples with GPT-3](https://colab.research.google.com/drive/1Yhl7iFb7JF0x89r1L3aDuufydVWX_VrL?usp=sharing) | ✅    |
-| Question-Answer Generation | [QA Task Examples with GPT-3](https://colab.research.google.com/drive/1Yhl7iFb7JF0x89r1L3aDuufydVWX_VrL?usp=sharing) | ✅    |
-| Relation-Extraction | [Relation-Extraction Examples with GPT-3](https://colab.research.google.com/drive/1iW4QNjllc8ktaQBWh3_04340V-tap1co?usp=sharing) | ✅    |
-| Summarization  | [Summarization Task Examples with GPT-3](https://colab.research.google.com/drive/1PlXIAMDtrK-RyVdDhiSZy6ztcDWsNPNw?usp=sharing) | ✅    |
-| Explanation    | [Explanation Task Examples with GPT-3](https://colab.research.google.com/drive/1PlXIAMDtrK-RyVdDhiSZy6ztcDWsNPNw?usp=sharing) | ✅    |
-| SQL Writer    | [SQL Writer Example with GPT-3](https://colab.research.google.com/drive/1JNUYCTdqkdeIAxiX-NzR-4dngdmWj0rV?usp=sharing) | ✅    |
-| Tabular Data | |    |
-| Image Data | |     |
-| More Prompts | |     |
+### Batch Processing
 
+```python
+results = ner.batch(["text1", "text2", "text3"], max_concurrent=10)
+```
 
-## Docs
+### Async Support
 
-[Promptify Docs](https://promptify.readthedocs.io/)
+```python
+result = await ner.acall("Patient has diabetes")
+```
+
+### Built-in Evaluation
+
+```python
+from promptify.eval import evaluate
+
+scores = evaluate(task=ner, dataset=labeled_data, metrics=["precision", "recall", "f1"])
+# {"precision": 0.92, "recall": 0.88, "f1": 0.90}
+```
 
-## Community 
+## Features
+
+- **2-3 lines of code** for any NLP task -no training data required
+- **Pydantic structured outputs** -type-safe results, not raw strings
+- **Any LLM provider** via LiteLLM -OpenAI, Anthropic, Google, Ollama, Azure, and 100+ more
+- **Built-in tasks** -NER, Classification (binary/multiclass/multilabel), QA, Summarization, Relation Extraction, SQL Generation, and more
+- **Custom tasks** -bring your own Pydantic schema for any structured output
+- **Few-shot examples** -easily add examples to improve accuracy
+- **Domain specialization** -pass `domain="medical"` or any domain for context-aware prompts
+- **Batch processing** -async concurrency under the hood for processing multiple texts
+- **Async support** -native `await` support with `acall()`
+- **Evaluation framework** -precision, recall, F1, accuracy, exact match, ROUGE metrics
+- **Safe parser** -fallback JSON completion for providers without native structured outputs (no `eval()`)
+- **Cost tracking** -built-in token usage and cost monitoring via `get_cost_summary()`
+
+### Supported NLP Tasks
+
+| Task | Class | Output Schema |
+|------|-------|---------------|
+| Named Entity Recognition | `NER` | `NERResult` (list of `Entity`) |
+| Binary Classification | `Classify` | `Classification` |
+| Multiclass Classification | `Classify` | `Classification` |
+| Multilabel Classification | `Classify(multi_label=True)` | `MultiLabelResult` |
+| Question Answering | `QA` | `Answer` |
+| Summarization | `Summarize` | `Summary` |
+| Relation Extraction | `ExtractRelations` | `ExtractionResult` |
+| Tabular Extraction | `ExtractTable` | `ExtractionResult` |
+| Question Generation | `GenerateQuestions` | list of `GeneratedQuestion` |
+| SQL Generation | `GenerateSQL` | `SQLQuery` |
+| Text Normalization | `NormalizeText` | normalized text |
+| Topic Modelling | `ExtractTopics` | list of topics |
+| Custom Task | `Task` | any Pydantic `BaseModel` |
+
+## Community
 <div align="center">
-If you are interested in Prompt-Engineering, LLMs, ChatGPT and other latest research discussions, please consider joining <a href="https://discord.gg/m88xfYMbK6">PromptsLab</a></div>
+If you are interested in Prompt-Engineering, LLMs, and NLP, please consider joining <a href="https://discord.gg/m88xfYMbK6">PromptsLab</a></div>
 <div align="center">
 <img alt="Join us on Discord" src="https://img.shields.io/discord/1069129502472556587?color=5865F2&logo=discord&logoColor=white">
 </div>

.bumpversion.cfg

@@ -1,17 +1,37 @@
-__version__ = "2.0.3"
-from .parser.parser import Parser
-from .prompter.nlp_prompter import Prompter
-from .prompter.prompt_cache import PromptCache
-from .prompter.template_loader import TemplateLoader
-from .prompter.conversation_logger import ConversationLogger
-from .models.text2text.api.openai_models import OpenAI
-from .models.text2text.api.anthropic import AnthropicModel
-from .models.text2text.api.cohere import CohereModel
-from .models.text2text.api.azure_openai import Azure
-from .models.text2text.api.hub_model import HubModel
-from .models.text2text.api.mock_model import MockModel
-from .models.text2text.api.base_model import Model
-from .utils.file_utils import *
-from .utils.data_utils import *
-from .utils.conversation_utils import *
-from .pipelines import Pipeline
+"""Promptify v3 — Task-based NLP engine with structured outputs."""
+
+from promptify._version import __version__
+from promptify.core.config import ModelConfig
+from promptify.core.logging import setup_logging
+from promptify.engine.cost import get_cost_summary
+from promptify.tasks import (
+    NER,
+    QA,
+    Classify,
+    ExtractRelations,
+    ExtractTable,
+    ExtractTopics,
+    GenerateQuestions,
+    GenerateSQL,
+    NormalizeText,
+    Summarize,
+    Task,
+)
+
+__all__ = [
+    "__version__",
+    "NER",
+    "Classify",
+    "QA",
+    "Summarize",
+    "Task",
+    "ExtractRelations",
+    "ExtractTable",
+    "GenerateQuestions",
+    "GenerateSQL",
+    "NormalizeText",
+    "ExtractTopics",
+    "ModelConfig",
+    "setup_logging",
+    "get_cost_summary",
+]

README.md

@@ -0,0 +1 @@
+__version__ = "3.0.0"

promptify/prompts/text2text/.DS_Store promptify/.DS_Storepromptify/prompts/text2text/.DS_Store renamed to promptify/.DS_Store

@@ -0,0 +1,5 @@
+from promptify.core.config import ModelConfig, CacheConfig
+from promptify.core.exceptions import PromptifyError
+from promptify.core.logging import setup_logging
+
+__all__ = ["ModelConfig", "CacheConfig", "PromptifyError", "setup_logging"]

promptify/VERSION

@@ -0,0 +1,42 @@
+"""Pydantic configuration models."""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Literal, Optional, Union
+
+from pydantic import BaseModel, Field, field_validator
+
+
+class ModelConfig(BaseModel):
+    """Configuration for the LLM engine."""
+
+    model: str = Field(description="Model identifier (e.g. 'gpt-4o-mini', 'claude-sonnet-4-20250514')")
+    api_key: Optional[str] = Field(default=None, description="API key (falls back to env var)")
+    temperature: float = Field(default=0.0, ge=0.0, le=2.0)
+    top_p: float = Field(default=1.0, ge=0.0, le=1.0)
+    max_tokens: Optional[int] = Field(default=None, gt=0)
+    stop: Optional[Union[str, List[str]]] = None
+    presence_penalty: float = Field(default=0.0, ge=-2.0, le=2.0)
+    frequency_penalty: float = Field(default=0.0, ge=-2.0, le=2.0)
+    timeout: Optional[float] = Field(default=None, gt=0)
+    max_retries: int = Field(default=3, ge=0, le=20)
+    extra_params: Dict[str, Any] = Field(default_factory=dict)
+
+    model_config = {"frozen": False}
+
+
+class CacheConfig(BaseModel):
+    """Configuration for response caching."""
+
+    enabled: bool = True
+    backend: Literal["memory", "disk", "redis"] = "memory"
+    maxsize: int = Field(default=128, gt=0)
+    ttl: Optional[int] = Field(default=3600, gt=0, description="TTL in seconds")
+    redis_url: Optional[str] = None
+
+    @field_validator("redis_url")
+    @classmethod
+    def validate_redis_url(cls, v: Optional[str], info: Any) -> Optional[str]:
+        if info.data.get("backend") == "redis" and not v:
+            raise ValueError("redis_url is required when backend is 'redis'")
+        return v

promptify/__init__.py

@@ -0,0 +1,53 @@
+"""Promptify exception hierarchy."""
+
+
+class PromptifyError(Exception):
+    """Base exception for all Promptify errors."""
+
+
+class ConfigurationError(PromptifyError):
+    """Invalid configuration."""
+
+
+class ModelError(PromptifyError):
+    """Base for model-related errors."""
+
+
+class ModelConnectionError(ModelError):
+    """Failed to connect to model provider."""
+
+
+class ModelAuthenticationError(ModelError):
+    """Invalid API key or authentication failure."""
+
+
+class ModelRateLimitError(ModelError):
+    """Rate limit exceeded."""
+
+
+class ModelResponseError(ModelError):
+    """Invalid or unexpected model response."""
+
+
+class TemplateError(PromptifyError):
+    """Base for template-related errors."""
+
+
+class TemplateNotFoundError(TemplateError):
+    """Template file not found."""
+
+
+class TemplateMissingVariableError(TemplateError):
+    """Required template variable not provided."""
+
+
+class ParserError(PromptifyError):
+    """Failed to parse model output."""
+
+
+class PipelineError(PromptifyError):
+    """Error in task pipeline execution."""
+
+
+class EvaluationError(PromptifyError):
+    """Error during evaluation."""