You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/concepts/freeform_views.md
+35-3Lines changed: 35 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,9 +2,41 @@
2
2
3
3
Freeform views are a type of [view](views.md) that provides a way for developers using db-ally to define what they need from the LLM without requiring a fixed response structure. This flexibility is beneficial when the data structure is unknown beforehand or when potential queries are too diverse to be covered by a structured view. Though freeform views offer more flexibility than structured views, they are less predictable, efficient, and secure, and may be more challenging to integrate with other systems. For these reasons, we recommend using [structured views](./structured_views.md) when possible.
4
4
5
-
Unlike structured views, which define a response format and a set of operations the LLM may use in response to natural language queries, freeform views only have one task - to respond directly to natural language queries with data from the datasource. They accomplish this by implementing the [`ask`][dbally.views.base.BaseView] method. This method takes a natural language query as input and returns a response. The method also has access to the LLM model (via the `llm` attribute), which is typically used to retrieve the correct data from the source (for example, by generating a source-specific query string). To learn more about implementing freeform views, refer to the [How to: Custom Freeform Views](../how-to/custom_freeform_views.md) guide.
5
+
Unlike structured views, which define a response format and a set of operations the LLM may use in response to natural language queries, freeform views only have one task - to respond directly to natural language queries with data from the datasource. They accomplish this by implementing the [`ask`][dbally.views.base.BaseView.ask] method. This method takes a natural language query as input and returns a response. The method also has access to the LLM model (via the `llm` attribute), which is typically used to retrieve the correct data from the source (for example, by generating a source-specific query string).
6
6
7
-
## Security
7
+
!!! example
8
+
For instance, this is a simple view that uses SQLAlchemy to query candidate table in a SQL database. It contains a single table definition, which LLM will use to generate the appropriate SQL query to answer the question:
9
+
10
+
```python
11
+
class CandidateView(BaseText2SQLView):
12
+
"""
13
+
A view for retrieving candidates from the database.
14
+
"""
15
+
16
+
def get_tables(self) -> List[TableConfig]:
17
+
"""
18
+
Defines the tables available in the database.
19
+
"""
20
+
return [
21
+
TableConfig(
22
+
name="candidate",
23
+
columns=[
24
+
ColumnConfig(
25
+
name="name",
26
+
data_type="VARCHAR",
27
+
),
28
+
ColumnConfig(
29
+
name="country",
30
+
data_type="VARCHAR",
31
+
),
32
+
...
33
+
],
34
+
),
35
+
...
36
+
]
37
+
```
38
+
39
+
To learn more about implementing freeform views, refer to the [How to: Create text-to-SQL Freeform View](../how-to/custom_freeform_views.md) guide.
8
40
9
41
!!! warning
10
-
When using freeform views, the LLM typically gets raw access to the data source and can execute arbitrary operations on it using the query language of the data source (e.g., SQL). This can be powerful but also necessitates that the developer be extremely cautious about securing the data source outside of db-ally. For instance, in the case of Relational Databases, the developer should ensure that the database user used by db-ally has read-only access to the database, and that the database does not contain any sensitive data that shouldn't be exposed to the LLM.
42
+
When using freeform views, the LLM typically gets raw access to the data source and can execute arbitrary operations on it using the query language of the data source (e.g., SQL). This can be powerful but also necessitates that the developer be extremely cautious about securing the data source outside of db-ally. For instance, in the case of Relational Databases, the developer should ensure that the database user used by db-ally has read-only access to the database, and that the database does not contain any sensitive data that shouldn't be exposed to the LLM.
Copy file name to clipboardExpand all lines: docs/concepts/structured_views.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ Given different natural language queries, a db-ally view will produce different
10
10
Each structured view can contain one or more “filters”, which the LLM may decide to choose and apply to the extracted data so that it meets the criteria specified in the natural language query. Given such a query, LLM chooses which filters to use, provides arguments to the filters, and connects the filters with Boolean operators. The LLM expresses these filter combinations using a special language called [IQL](iql.md), in which the defined view filters provide a layer of abstraction between the LLM and the raw syntax used to query the data source (e.g., SQL).
11
11
12
12
!!! example
13
-
For instance, this is a simple [view that uses SQLAlchemy](../how-to/sql_views.md) to select data from specific columns in a SQL database. It contains a single filter, that the LLM may optionally use to control which table rows to fetch:
13
+
For instance, this is a simple [view that uses SQLAlchemy](../how-to/views/sql.md) to select data from specific columns in a SQL database. It contains a single filter, that the LLM may optionally use to control which table rows to fetch:
Copy file name to clipboardExpand all lines: docs/how-to/views/custom.md
+12-12Lines changed: 12 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
# How-To: Use custom data sources with db-ally
2
2
3
3
!!! note
4
-
This is an advanced topic. If you're looking to create a view that retrieves data from an SQL database, please refer to the [SQL Views](sql_views.md) guide instead.
4
+
This is an advanced topic. If you're looking to create a view that retrieves data from an SQL database, please refer to the [SQL Views](sql.md) guide instead.
5
5
6
-
In this guide, we'll show you how to create [structured views](../concepts/structured_views.md) that connect to custom data sources. This could be useful if you need to retrieve data from a REST API, a NoSQL database, or any other data source not supported by the built-in base views.
6
+
In this guide, we'll show you how to create [structured views](../../concepts/structured_views.md) that connect to custom data sources. This could be useful if you need to retrieve data from a REST API, a NoSQL database, or any other data source not supported by the built-in base views.
7
7
8
-
#Summary
8
+
## Intro
9
9
Firstly, we will create a custom base view called `FilteredIterableBaseView` that retrieves data from a Python iterable and allows it to be filtered. It forms the base that implements data source-specific logic and lets other views inherit from it in order to define filters for specific use cases (similar to how `SqlAlchemyBaseView` is a base view provided by db-ally).
10
10
11
11
Then, we will create a view called `CandidatesView` that inherits from `FilteredIterableBaseView` and represents a use case of retrieving a list of job candidates. We will define filters for this view.
@@ -53,7 +53,7 @@ class CandidateView(FilteredIterableBaseView):
53
53
54
54
Lastly, we will illustrate how to use the `CandidatesView` like any other view in db-ally. We will create an instance of the view, add it to a collection, and start querying it.
55
55
56
-
## Types of Custom Views
56
+
## Types of custom views
57
57
There are two main ways to create custom structured views:
58
58
59
59
* By subclassing the `MethodsBaseView`: This is the most common method. These views expect filters to be defined as class methods and help manage them. All the built-in db-ally views use this method.
@@ -62,12 +62,12 @@ There are two main ways to create custom structured views:
62
62
If you're not sure which method to choose, we recommend starting with the `MethodsBaseView`. It's simpler and easier to use, and you can switch to the `BaseStructuredView` later if you find you need more control over filter management. For this guide, we'll focus on the `MethodsBaseView`.
63
63
64
64
!!! note
65
-
Both are methods of creating [structured views](../concepts/structured_views.md). If you're looking to create a [freeform view](../concepts/freeform_views.md), refer to the [Freeform Views](custom_freeform_views.md) guide instead.
65
+
Both are methods of creating [structured views](../../concepts/structured_views.md). If you're looking to create a [freeform view](../../concepts/freeform_views.md), refer to the [Freeform Views](custom_freeform_views.md) guide instead.
66
66
67
-
## The Example
67
+
## Example
68
68
Throughout the guide, we'll use an example of creating a custom base view called `FilteredIterableBaseView`. To keep things simple, the "data source" it uses is a list defined in Python. The goal is to demonstrate how to create a custom view and define filters for it. In most real-world scenarios, data would usually come from an external source, like a REST API or a database.
69
69
70
-
Next, we are going to create a view that inherits from `FilteredIterableBaseView` and implements a use case of retrieving a list of job candidates. This is the same use case from the [Quickstart](../quickstart/index.md) guide - but this time we'll use a custom view instead of the built-in `SqlAlchemyBaseView`. For comparison, you can refer to the Quickstart guide.
70
+
Next, we are going to create a view that inherits from `FilteredIterableBaseView` and implements a use case of retrieving a list of job candidates. This is the same use case from the [Quickstart](../../quickstart/index.md) guide - but this time we'll use a custom view instead of the built-in `SqlAlchemyBaseView`. For comparison, you can refer to the Quickstart guide.
71
71
72
72
Before we start, let's define a simple data class to represent a candidate:
73
73
@@ -83,8 +83,8 @@ class Candidate:
83
83
country: str
84
84
```
85
85
86
-
## Creating a Custom View
87
-
In db-ally, the typical approach is to have a base view that inherits from `MethodsBaseView` and implements elements specific to a type of data source (for example, [`SqlAlchemyBaseView`](./sql_views.md) is a base view provided by db-ally). Subsequently, you can create views that inherit from this base view, reflecting business logic specific to given use cases, including defining the filter methods.
86
+
## Creating a custom view
87
+
In db-ally, the typical approach is to have a base view that inherits from `MethodsBaseView` and implements elements specific to a type of data source (for example, [`SqlAlchemyBaseView`](sql.md) is a base view provided by db-ally). Subsequently, you can create views that inherit from this base view, reflecting business logic specific to given use cases, including defining the filter methods.
88
88
89
89
For our example, let's create a base class called `FilteredIterableBaseView`:
90
90
@@ -100,7 +100,7 @@ class FilteredIterableBaseView(MethodsBaseView):
100
100
For now, this class is empty, but we'll build upon it in the following sections.
101
101
102
102
## Specifying how filters should be applied
103
-
Classes that inherit from `FilteredIterableBaseView` can define filters as methods. The LLM will choose which of these filters to use, feed them with arguments, and determine how to combine them using boolean operators. To achieve this, the LLM uses its own language, [IQL](../concepts/iql.md). db-ally translates the IQL expressions and passes the parsed IQL tree to your view via the `apply_filters` method. This method is responsible for identifying the selected filter methods and applying them to the data (which could vary based on the data source).
103
+
Classes that inherit from `FilteredIterableBaseView` can define filters as methods. The LLM will choose which of these filters to use, feed them with arguments, and determine how to combine them using boolean operators. To achieve this, the LLM uses its own language, [IQL](../../concepts/iql.md). db-ally translates the IQL expressions and passes the parsed IQL tree to your view via the `apply_filters` method. This method is responsible for identifying the selected filter methods and applying them to the data (which could vary based on the data source).
104
104
105
105
Let's implement the required `apply_filters` method in our `FilteredIterableBaseView`. Additionally, we'll create a helper method, `build_filter_node`, responsible for handling a single node of the IQL tree (either a filter or a logical operator). We'll use recursion to handle these nodes since they can have children (a logical operator can have two children that are filters, for instance).
The `execute` function gets the data (by calling the `get_data` method) and applies the combined filters to it. We're using the [`filter`](https://docs.python.org/3/library/functions.html#filter) function from Python's standard library to accomplish this. The filtered data is then returned as a list.
172
172
173
-
## Creating a View
173
+
## Creating a view
174
174
Now that `FilteredIterableBaseView` is complete, we can create a view that inherits from it and represents the use case of retrieving a list of job candidates. We'll name this view `CandidatesView`:
Copy file name to clipboardExpand all lines: docs/how-to/views/pandas.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,10 @@
1
1
# How-To: Use Pandas DataFrames with db-ally
2
2
3
-
In this guide, you will learn how to write [structured views](../concepts/structured_views.md) that use [Pandas](https://pandas.pydata.org/) DataFrames as their data source. You will understand how to define such a view, create filters that operate on the DataFrame, and register it while providing it with the source DataFrame.
3
+
In this guide, you will learn how to write [structured views](../../concepts/structured_views.md) that use [Pandas](https://pandas.pydata.org/) DataFrames as their data source. You will understand how to define such a view, create filters that operate on the DataFrame, and register it while providing it with the source DataFrame.
4
4
5
-
The example used in this guide is a DataFrame containing information about candidates. The DataFrame includes columns such as `id`, `name`, `country`, `years_of_experience`. This is the same use case as the one in the [Quickstart](../quickstart/index.md) and [Custom Views](./custom_views.md) guides. Please feel free to compare the different approaches.
5
+
The example used in this guide is a DataFrame containing information about candidates. The DataFrame includes columns such as `id`, `name`, `country`, `years_of_experience`. This is the same use case as the one in the [Quickstart](../../quickstart/index.md) and [Custom Views](./custom.md) guides. Please feel free to compare the different approaches.
6
6
7
-
## The DataFrame
7
+
## Data
8
8
Here is an example of a DataFrame containing information about candidates:
To use the view, you need to create a [Collection](../concepts/collections.md) and register the view with it. This is done in the same manner as registering other types of views, but you need to provide the view with the DataFrame on which it should operate:
72
+
##Registering the view
73
+
To use the view, you need to create a [Collection](../../concepts/collections.md) and register the view with it. This is done in the same manner as registering other types of views, but you need to provide the view with the DataFrame on which it should operate:
0 commit comments