v12.0.1

What's Changed

• fix: Fix wrong timezone issue in BenchmarkExecution

Full Changelog: v12.0.0...v12.0.1

v12.0.0

Breaking Changes

• StudioClient raises ValueError when instantiated with project name that belongs to multiple projects.

Full Changelog: v11.0.0...v12.0.0

v11.0.0

11.0.0

Breaking Changes

• QdrantInMemoryRetriever and HybridQdrantInMemoryRetriever now use pharia-1-embedding-4608-control as embedding model with an embedding size of 4608.

Full Changelog: v10.0.0...v11.0.0

v10.0.0

10.0.0

Features

• Introduced AsyncDocumentIndexClient and AsyncDocumentIndexRetriever as drop-in replacements for their blocking counterparts, enabling coroutine-based, non-blocking document indexing and retrieval.

Fixes

• InMemoryDatasetRepository now has a more descriptive error message when creating a dataset fails due to an ID clash.
• StudioClient now deserializes and serializes examples while maintaining type information, which was previously dropped.
• RunRepository and EvaluationRepository now more accurately reflect their actual return types in their signatures. Previously, it was not obvious that failed examples could be returned.
• FileTracer.traces now uses "utf-8" encoding to read the persisted trace file. Previously, the machines default encoding was used, which could lead to a mismatch between the encoding used for writing and reading.

Breaking Changes

• StudioClient now handles project_id as a string instead of an integer. This is only relevant when you handle project ids (not names) manually.
• InMemoryDatasetRepository now returns the exact types given by users when retrieving Example. Previously, it disregarded the types it was given and returned what was saved.
  • This is in line with how the other repositories work.
• InMemoryDatasetRepository, InMemoryRunRepository, InMemoryEvaluationRepository, and InMemoryAggregationRepository now either return the exact types given by users when retrieving example-related data or fail. Specifically, this means that passing the wrong type when retrieving data will now fail with a ValidationError. Previously, the repositories disregarded the types they were given and returned whatever object was saved.
  • This is in line with how the other repositories work.
• EloQaEvaluationLogic now has an expected output type of None instead of SingleChunkQaOutput. The information was unused.
  • If you have pipelines that define data to be processed by this logic, or if you subclass from this specific logic, you may need to adapt it.
  • log_probs in the CompletionInput of the do_run method been set to 20 instead of the prior value of 30
• The legacy Trace Viewer has now been removed along with all references to it.

Full changelog: v9.1.0...v10.0.0

v9.1.0

9.1.0

Features

• New Pharia Kernel connector (KernelTask) for calling Skills from a Task
• Add HybridQdrantInMemoryRetriever enabling hybrid search for in-memory Qdrant collections

Fixes

• Add warning to PromptBasedClassify and PromptBasedClassifyWithDefinitions to be cautious when using them with model families other than luminous

Full changelog: v9.0.2...v9.1.0

v9.0.2

9.0.2

Features

• Update dependencies

Full changelog: v9.0.1...v9.0.2

v9.0.1

9.0.1

Fixes

• Fixes an incompatibility where models with tokenizer with no whitespace prefix could not be used for qa examples. Now, no error will be thrown.

Full changelog: v9.0.0...v9.0.1

v9.0.0

9.0.0

Features

• Introduce Benchmark and StudioBenchmark
  • Benchmark allows you to evaluate and compare the performance of different Tasks with a fixed evaluation logic, aggregation logic and Dataset.
  • Add how_to_execute_a_benchmark.ipynb to how-tos
  • Add studio.ipynb to notebooks to show how one can debug a Task with Studio
• Introduce BenchmarkRepositoryand StudioBenchmarkRepository
• Add create_project bool to StudioClient.__init__() to enable users to automatically create their Studio projects
• Add progressbar to the Runner to be able to track the Run
• Add StudioClient.submit_benchmark_lineages function and include it in StudioClient.submit_benchmark_execution

DocumentIndexClient

• Add method DocumentIndexClient.chunks() for retrieving all text chunks of a document.
• Add metadata filter FilterOps.IS_NULL, that allows to filter fields based on whether their value is null.

Fixes

• The Document Index SearchQuery now correctly allows searches with a negative min_score.

Deprecations

...

Breaking Changes

• The env variable POSTGRES_HOST is split into POSTGRES_HOST and POSTGRES_PORT. This affects all classes interacting with Studio and the InstructionFinetuningDataRepository.
• The following env variables now need to be set (previously pointed to defaults)
  • CLIENT_URL - URL of your inference stack
  • DOCUMENT_INDEX_URL - URL of the document index

Full changelog: v8.0.0...v9.0.0

v8.0.0

v8.0.0

Features

• You can now customise the embedding model when creating an index using the DocumentIndexClient.
• You can now use the InstructableEmbed embedding strategy when creating an index using the DocumentIndexClient. See the document_index.ipynb notebook for more information and an example.

Breaking Changes

• The way you configure indexes in the DocumentIndexClient has changed. See the document_index.ipynb notebook for more information.
• The EmbeddingType alias has been renamed to Representation to better align with the underlying API.
• The embedding_type field has been removed from the IndexConfiguration class. You now configure embedding-related parameters via the embedding field.
• You now always need to specify an embedding model when creating an index. Previously, this was always luminous-base.

Full changelog: v7.3.1...v8.0.0

v7.3.1

7.3.1

Features

• Update dependencies

Full changelog: v0.7.3.0...v7.3.1