refactor(targets): rename storage to target (#625)

* docs(targets): rename `storage` to `target`

* refactor(targets): update Python to use `storages` instead of `targets`

* refactor(targets): rename `storages` to `targets` for remaining code

* examples: revert upgrades for now - will bring back after release

Author: badmonster0

README.md

@@ -111,7 +111,7 @@ def text_embedding_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoind
     # Export collected data to a vector index.
     doc_embeddings.export(
         "doc_embeddings",
-        cocoindex.storages.Postgres(),
+        cocoindex.targets.Postgres(),
         primary_key_fields=["filename", "location"],
         vector_indexes=[
             cocoindex.VectorIndexDef(

docs/docs/core/basics.md

@@ -9,15 +9,15 @@ An **index** is a collection of data stored in a way that is easy for retrieval.
 
 CocoIndex is an ETL framework for building indexes from specified data sources, a.k.a. **indexing**. It also offers utilities for users to retrieve data from the indexes.
 
-An **indexing flow** extracts data from specified data sources, upon specified transformations, and puts the transformed data into specified storage for later retrieval.
+An **indexing flow** extracts data from specified data sources, upon specified transformations, and puts the transformed data into specified target for later retrieval.
 
 ## Indexing flow elements
 
 An indexing flow has two aspects: data and operations on data.
 
 ### Data
 
-An indexing flow involves source data and transformed data (either as an intermediate result or the final result to be put into storage). All data within the indexing flow has **schema** determined at flow definition time.
+An indexing flow involves source data and transformed data (either as an intermediate result or the final result to be put into targets). All data within the indexing flow has **schema** determined at flow definition time.
 
 Each piece of data has a **data type**, falling into one of the following categories:
 
@@ -36,8 +36,8 @@ An **operation** in an indexing flow defines a step in the flow. An operation is
 *   **Action**, which defines the behavior of the operation, e.g. *import*, *transform*, *for each*, *collect* and *export*.
     See [Flow Definition](flow_def) for more details for each action.
 
-*   Some actions (i.e. "import", "transform" and "export") require an **Operation Spec**, which describes the specific behavior of the operation, e.g. a source to import from, a function describing the transformation behavior, a target storage to export to (as an index).
-    *   Each operation spec has a **operation type**, e.g. `LocalFile` (data source), `SplitRecursively` (function), `SentenceTransformerEmbed` (function), `Postgres` (storage).
+*   Some actions (i.e. "import", "transform" and "export") require an **Operation Spec**, which describes the specific behavior of the operation, e.g. a source to import from, a function describing the transformation behavior, a target to export to (as an index).
+    *   Each operation spec has a **operation type**, e.g. `LocalFile` (data source), `SplitRecursively` (function), `SentenceTransformerEmbed` (function), `Postgres` (target).
     *   CocoIndex framework maintains a set of supported operation types. Users can also implement their own.
 
 "import" and "transform" operations produce output data, whose data type is determined based on the operation spec and data types of input data (for "transform" operation only).
@@ -62,11 +62,11 @@ This shows schema and example data for the indexing flow:
 
 ## Life cycle of an indexing flow
 
-An indexing flow, once set up, maintains a long-lived relationship between data source and data in target storage. This means:
+An indexing flow, once set up, maintains a long-lived relationship between data source and target. This means:
 
-1.  The target storage created by the flow remain available for querying at any time
+1.  The target created by the flow remain available for querying at any time
 
-2.  As source data changes (new data added, existing data updated or deleted), data in the target storage are updated to reflect those changes,
+2.  As source data changes (new data added, existing data updated or deleted), data in the target are updated to reflect those changes,
     on certain pace, according to the update mode:
 
     *   **One time update**: Once triggered, CocoIndex updates the target data to reflect the version of source data up to the current moment.

docs/docs/core/cli.mdx

@@ -61,7 +61,7 @@ The following subcommands are available:
 | ---------- | ----------- |
 | `ls` | List all flows present in the given file/module. Or list all persisted flows under the current app namespace if no file/module specified. |
 | `show` | Show the spec and schema for a specific flow. |
-| `setup` | Check and apply backend setup changes for flows, including the internal and target storage (to export). |
+| `setup` | Check and apply backend setup changes for flows, including the internal storage and target (to export). |
 | `drop` | Drop the backend setup for specified flows. |
 | `update` | Update the index defined by the flow. |
 | `evaluate` | Evaluate the flow and dump flow outputs to files.  Instead of updating the index, it dumps what should be indexed to files. Mainly used for evaluation purpose. |

docs/docs/core/data_types.mdx

@@ -13,7 +13,7 @@ This makes schema of data processed by CocoIndex clear, and easily determine the
 
 You don't need to spell out data types in CocoIndex, when you define the flow using existing operations (source, function, etc).
 These operations decide data types of fields produced by them based on the spec and input data types.
-All you need to do is to make sure the data passed to functions and storage targets are accepted by them.
+All you need to do is to make sure the data passed to functions and targets are accepted by them.
 
 When you define [custom functions](/docs/core/custom_function), you need to specify the data types of arguments and return values.
 
@@ -40,7 +40,7 @@ This is the list of all basic types supported by CocoIndex:
 | Vector[*T*, *Dim*?] | *T* can be a basic type or a numeric type. *Dim* is a positive integer and optional. | `cocoindex.Vector[T]` or `cocoindex.Vector[T, Dim]` | `numpy.typing.NDArray[T]` or `list[T]` |
 
 Values of all data types can be represented by values in Python's native types (as described under the Native Python Type column).
-However, the underlying execution engine and some storage system (like Postgres) has finer distinctions for some types, specifically:
+However, the underlying execution engine has finer distinctions for some types, specifically:
 
 *   *Float32* and *Float64* for `float`, with different precision.
 *   *LocalDateTime* and *OffsetDateTime* for `datetime.datetime`, with different timezone awareness.
@@ -50,7 +50,7 @@ However, the underlying execution engine and some storage system (like Postgres)
 
 The native Python type is always more permissive and can represent a superset of possible values.
 *   Only when you annotate the return type of a custom function, you should use the specific type,
-    so that CocoIndex will have information about the precise type to be used in the execution engine and storage system.
+    so that CocoIndex will have information about the precise type to be used in the execution engine and target.
 *   For all other purposes, e.g. to provide annotation for argument types of a custom function, or used internally in your custom function,
     you can choose whatever to use.
     The native Python type is usually simpler.

docs/docs/core/flow_def.mdx

@@ -1,14 +1,14 @@
 ---
 title: Flow Definition
-description: Define a CocoIndex flow, by specifying source, transformations and storages, and connect input/output data of them.
+description: Define a CocoIndex flow, by specifying source, transformations and targets, and connect input/output data of them.
 ---
 
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 
 # CocoIndex Flow Definition
 
-In CocoIndex, to define an indexing flow, you provide a function to import source, transform data and put them into target storage (sinks).
+In CocoIndex, to define an indexing flow, you provide a function to import source, transform data and put them into targets.
 You connect input/output of these operations with fields of data scopes.
 
 ## Entry Point
@@ -246,14 +246,14 @@ and generates a `id` field with UUID and remains stable when `filename` and `sum
 
 ### Export
 
-The `export()` method exports the collected data to an external storage.
+The `export()` method exports the collected data to an external target.
 
-A *storage spec* needs to be provided for any export operation, to describe the storage and parameters related to the storage.
+A *target spec* needs to be provided for any export operation, to describe the target and parameters related to the target.
 
 Export must happen at the top level of a flow, i.e. not within any child scopes created by "for each row". It takes the following arguments:
 
 *   `name`: the name to identify the export target.
-*   `target_spec`: the storage spec as the export target.
+*   `target_spec`: the target spec as the export target.
 *   `setup_by_user` (optional):
      whether the export target is setup by user.
      By default, CocoIndex is managing the target setup (surfaced by the `cocoindex setup` CLI subcommand), e.g. create related tables/collections/etc. with compatible schema, and update them upon change.
@@ -270,22 +270,22 @@ def demo_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataSco
     demo_collector = data_scope.add_collector()
     ...
     demo_collector.export(
-        "demo_storage", DemoStorageSpec(...),
+        "demo_target", DemoTargetSpec(...),
         primary_key_fields=["field1"],
         vector_indexes=[cocoindex.VectorIndexDef("field2", cocoindex.VectorSimilarityMetric.COSINE_SIMILARITY)])
 ```
 
 </TabItem>
 </Tabs>
 
-The target storage is managed by CocoIndex, i.e. it'll be created by [CocoIndex CLI](/docs/core/cli) when you run `cocoindex setup`, and the data will be automatically updated (including stale data removal) when updating the index.
-The `name` for the same storage should remain stable across different runs.
-If it changes, CocoIndex will treat it as an old storage removed and a new one created, and perform setup changes and reindexing accordingly.
+The target is managed by CocoIndex, i.e. it'll be created by [CocoIndex CLI](/docs/core/cli) when you run `cocoindex setup`, and the data will be automatically updated (including stale data removal) when updating the index.
+The `name` for the same target should remain stable across different runs.
+If it changes, CocoIndex will treat it as an old target removed and a new one created, and perform setup changes and reindexing accordingly.
 
 ## Storage Indexes
 
-Many storage supports indexes, to boost efficiency in retrieving data.
-CocoIndex provides a common way to configure indexes for various storages.
+Many targets are storage systems supporting indexes, to boost efficiency in retrieving data.
+CocoIndex provides a common way to configure indexes for various targets.
 
 ### Primary Key
 
@@ -330,7 +330,7 @@ For example,
 ```python
 doc_embeddings.export(
     "doc_embeddings",
-    cocoindex.storages.Qdrant(
+    cocoindex.targets.Qdrant(
         collection_name=cocoindex.get_app_namespace(trailing_delimiter='__') + "doc_embeddings",
         ...
     ),
@@ -345,8 +345,8 @@ It will use `Staging__doc_embeddings` as the collection name if the current app
 
 ### Target Declarations
 
-Most time a target storage is created by calling `export()` method on a collector, and this `export()` call comes with configurations needed for the target storage, e.g. options for storage indexes.
-Occasionally, you may need to specify some configurations for target storage out of the context of any specific data collector.
+Most time a target is created by calling `export()` method on a collector, and this `export()` call comes with configurations needed for the target, e.g. options for storage indexes.
+Occasionally, you may need to specify some configurations for the target out of the context of any specific data collector.
 
 For example, for graph database targets like `Neo4j` and `Kuzu`, you may have a data collector to export data to relationships, which will create nodes referenced by various relationships in turn.
 These nodes don't directly come from any specific data collector (consider relationships from different data collectors may share the same nodes).
@@ -359,7 +359,7 @@ To specify configurations for these nodes, you can *declare* spec for related no
 
 ```python
 flow_builder.declare(
-    cocoindex.storages.Neo4jDeclarations(...)
+    cocoindex.targets.Neo4jDeclarations(...)
 )
 ```
 
@@ -389,7 +389,7 @@ You can add an auth entry by `cocoindex.add_auth_entry()` function, which return
 ```python
 my_graph_conn = cocoindex.add_auth_entry(
     "my_graph_conn",
-    cocoindex.storages.Neo4jConnectionSpec(
+    cocoindex.targets.Neo4jConnectionSpec(
             uri="bolt://localhost:7687",
             user="neo4j",
             password="cocoindex",
@@ -403,7 +403,7 @@ Then reference it when building a spec that takes an auth entry:
     ```python
     demo_collector.export(
         "MyGraph",
-        cocoindex.storages.Neo4jRelationship(connection=my_graph_conn, ...)
+        cocoindex.targets.Neo4jRelationship(connection=my_graph_conn, ...)
     )
     ```
 
@@ -412,7 +412,7 @@ Then reference it when building a spec that takes an auth entry:
     ```python
     demo_collector.export(
         "MyGraph",
-        cocoindex.storages.Neo4jRelationship(connection=cocoindex.ref_auth_entry("my_graph_conn"), ...))
+        cocoindex.targets.Neo4jRelationship(connection=cocoindex.ref_auth_entry("my_graph_conn"), ...))
     ```
 
 </TabItem>

docs/docs/core/flow_methods.mdx

@@ -1,7 +1,7 @@
 ---
 title: Run a Flow
 toc_max_heading_level: 4
-description: Run a CocoIndex Flow, including build / update data in the target storage and evaluate the flow without changing the target storage.
+description: Run a CocoIndex Flow, including build / update data in the target and evaluate the flow without changing the target.
 ---
 
 import Tabs from '@theme/Tabs';
@@ -37,7 +37,7 @@ It creates a `demo_flow` object in `cocoindex.Flow` type.
 
 ## Build / update target data
 
-The major goal of a flow is to perform the transformations on source data and build / update data in the target storage (the index).
+The major goal of a flow is to perform the transformations on source data and build / update data in the target.
 This action has two modes:
 
 *   **One time update.**
@@ -53,7 +53,7 @@ This action has two modes:
 :::info
 
 For both modes, CocoIndex is performing *incremental processing*,
-i.e. we only perform computations and storage mutations on source data that are changed, or the flow has changed.
+i.e. we only perform computations and target mutations on source data that are changed, or the flow has changed.
 This is to achieve best efficiency.
 
 :::
@@ -63,7 +63,7 @@ This is to achieve best efficiency.
 
 #### CLI
 
-The `cocoindex update` subcommand creates/updates data in the target storage.
+The `cocoindex update` subcommand creates/updates data in the target.
 
 Once it's done, the target data is fresh up to the moment when the function is called.
 
@@ -76,7 +76,7 @@ cocoindex update main.py
 <Tabs>
 <TabItem value="python" label="Python">
 
-The `update()` async method creates/updates data in the target storage.
+The `update()` async method creates/updates data in the target.
 
 Once the function returns, the target data is fresh up to the moment when the function is called.
 
@@ -207,7 +207,7 @@ CocoIndex also provides asynchronous versions of APIs for blocking operations, i
 
 ## Evaluate the flow
 
-CocoIndex allows you to run the transformations defined by the flow without updating the target storage.
+CocoIndex allows you to run the transformations defined by the flow without updating the target.
 
 ### CLI
 

docs/docs/getting_started/quickstart.md

@@ -87,7 +87,7 @@ def text_embedding_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoind
     # Export collected data to a vector index.
     doc_embeddings.export(
         "doc_embeddings",
-        cocoindex.storages.Postgres(),
+        cocoindex.targets.Postgres(),
         primary_key_fields=["filename", "location"],
         vector_indexes=[
             cocoindex.VectorIndexDef(
@@ -214,7 +214,7 @@ from pgvector.psycopg import register_vector
 
 def search(pool: ConnectionPool, query: str, top_k: int = 5):
     # Get the table name, for the export target in the text_embedding_flow above.
-    table_name = cocoindex.utils.get_target_storage_default_name(text_embedding_flow, "doc_embeddings")
+    table_name = cocoindex.utils.get_target_default_name(text_embedding_flow, "doc_embeddings")
     # Evaluate the transform flow defined above with the input query, to get the embedding.
     query_vector = text_to_embedding.eval(query)
     # Run the query and get the results.
@@ -237,7 +237,7 @@ There're two CocoIndex-specific logic:
 1.  Get the table name from the export target in the `text_embedding_flow` above.
     Since the table name for the `Postgres` target is not explicitly specified in the `export()` call,
     CocoIndex uses a default name.
-    `cocoindex.utils.get_target_storage_default_name()` is a utility function to get the default table name for this case.
+    `cocoindex.utils.get_target_default_name()` is a utility function to get the default table name for this case.
 
 2.  Evaluate the transform flow defined above with the input query, to get the embedding.
     It's done by the `eval()` method of the transform flow `text_to_embedding`.