v0.2.21

What's Changed

• Support source text files encoded in UTF-16/UTF-32
  • feat(file-decode): support files in UTF-16/UTF-32 by detecting BOM by @georgeh0 in #1185
  • fix: let bytes_to_string() return Cow to avoid unnecessary copy by @georgeh0 in #1186
• Support custom source
  • chore: always pass PyExecutionContext to the analyzer by @georgeh0 in #1195
  • feat(custom-source): add custom source support by @georgeh0 in #1197
  • feat(custom-source): make custom source API more robust and expose more error information by @georgeh0 in #1198
  • chore: allow source get API returns no ordinal by @georgeh0 in #1199
  • docs(custom-source): add docs for custom source by @georgeh0 in #1201
• fix: show proper error message when failed in loading module by @georgeh0 in #1196

Full Changelog: v0.2.20...v0.2.21

v0.2.20

What's Changed

• feat: Add AWS Bedrock LLM Support by @belloibrahv in #1173
• documentation - sources link & kuzu by @badmonster0 in #1179
• feat: improve logic fp: avoid source changes and count fn behavior ver by @georgeh0 in #1182
• cleanup: remove unused modules by @georgeh0 in #1183
• [doc] clean up sources subsections from the sources/index by @badmonster0 in #1184

Full Changelog: v0.2.19...v0.2.20

v0.2.19

What's Changed

• docs: create subpages for each source (#1126) by @esther-anierobi in #1136
• feat: Neo4j vector index method support by @belloibrahv in #1111
• feat(pg-source-filter): support filter for Postgres source by @georgeh0 in #1178

New Contributors

• @esther-anierobi made their first contribution in #1136

Full Changelog: v0.2.18...v0.2.19

v0.2.18

What's Changed

• feat(halfvec): support halfvec for postgres target by @georgeh0 in #1171
• Adjust SplitRecrsively
  • fix(split/markdown): treat code blocks as terminal nodes by @georgeh0 in #1157
  • chore: add more punctuations for pure text split by @georgeh0 in #1159
  • chore: use chunk_overlap as minimum atom size, if specified by @georgeh0 in #1160
• feat: add DetectProgrammingLanguage function by @georgeh0 in #1165
• fix: log instead of error out on live mode source-level error by @georgeh0 in #1156
• fix: auto retry explicitly for OpenAI requests by @georgeh0 in #1172
• fix: compatible with null value during foreach and get field by @georgeh0 in #1174
• docs: clarify pool limit for Supabase by @georgeh0 in #1175

Full Changelog: v0.2.17...v0.2.18

v0.2.17

What's Changed

• Python SDK improvement
  • Support pydantic with field-level description
    • support binding classes on Pydantic Models to CocoIndex Struct by @Davda-James in #1072
    • feat: Add field-level descriptions to FieldSchema by @belloibrahv in #1087
  • fix: auto-apply safe defaults for missing fields in load_engine_object by @princyballabh in #1104
  • feat: implemented str and repr methods for python schema classes by @shresthashim in #1095
• reorganize package structure
  • sources moved to sources subpackage in python sdk by @Davda-James in #1081
  • reafactored functions module in python sdk to subpackage by @Davda-James in #1082
• CLI improvement
  • feat: add --reset flag for setup, update, and server commands by @thisisharsh7 in #1106
  • generate full CLI docs from click's metadata and also added it in precommit hooks by @Davda-James in #1096
  • feat: add pretty error when no flows registered in server command by @thisisharsh7 in #1070
• Documentation updates
  • Update documentation to use cocoindex update --setup main with modu… by @mensonones in #1093
  • docs: setup dev env should happen first, also clarify pre-commit etc. by @georgeh0 in #1097
  • Update documentation to use cocoindex update module_name by @badmonster0 in #1103
• Documentation reorganization
  • organize targets in documentation by @badmonster0 in #1101
  • feat(docs): move built-in source as top level navigation by @TheVijayVignesh in #1119
  • [documentation] source overview by @badmonster0 in #1125
• feat: collect additional counter for number of rows that are being processed for stats by @Davda-James in #1105
• feat: split convert.py into 2 modules engine_object and engine_value along with splitting of tests by @Davda-James in #1120
• Target attachment / Postgres SQL command attachment
  • feat(attachment): support target attachment in the core setup engine by @georgeh0 in #1139
  • feat(attachment): create TargetSpecificAttachmentFactoryBase by @georgeh0 in #1140
  • feat(postgres-sql-attachments): add PostgresSqlAttachment by @georgeh0 in #1141
  • feat(attachment): update tracked states even if no update action needed by @georgeh0 in #1142
  • feat(postgres-sql-attachments): allow multiple statements executed by @georgeh0 in #1143
  • docs(postgres-sql-cmd): add docs for PostgresSqlCommand attachment by @georgeh0 in #1146
  • chore(postgres-sql-cmd): rename by @georgeh0 in #1147
• More comprehensive tests
  • Feature:Add tests for json_schema.rs by @aryasoni98 in #1133
• Isolate logic for states backward compatible into Cargo feature
  • feat: Add legacy-states-v0 Cargo feature for backward compatibility #1076 by @skalwaghe-56 in #1135
• Feat/postgres schema support by @thisisharsh7 in #1138
• Fix: Flatten recursion in SplitRecursively to prevent stack overflow by @siddharthbaleja7 in #1127
• fix(vertexai-retry): retry properly for Vertex AI by @georgeh0 in #1150

New Contributors

• @Davda-James made their first contribution in #1072
• @mensonones made their first contribution in #1093
• @shresthashim made their first contribution in #1095
• @belloibrahv made their first contribution in #1087
• @princyballabh made their first contribution in #1104
• @TheVijayVignesh made their first contribution in #1119
• @aryasoni98 made their first contribution in #1133
• @skalwaghe-56 made their first contribution in #1135
• @siddharthbaleja7 made their first contribution in #1127

Full Changelog: v0.2.16...v0.2.17

v0.2.16

What's Changed

• fix: avoid mypy glitches by @georgeh0 in #1064
• fix: make example paths os-friendly using os.path.join() by @thisisharsh7 in #1066
• fix: make sure a row is reexported on schema change by @georgeh0 in #1069

New Contributors

• @thisisharsh7 made their first contribution in #1066

Full Changelog: v0.2.15...v0.2.16

v0.2.15

What's Changed

• feat(ops): SplitBySeparators — regex-only splitter (Rust+Py) by @MrAnayDongre in #1010
• refactor: extract common logic out of split functions with style fix by @georgeh0 in #1058
• fix: make sure pyo3 runtime is initialized before all others by @georgeh0 in #1063

New Contributors

• @MrAnayDongre made their first contribution in #1010

Full Changelog: v0.2.14...v0.2.15

v0.2.14

What's Changed

• feat: fix Gemini embedding config and expose Postgres index tuning by @banrovegrie in #1050
• fix: raise errors when indexing option methods are not supported by @georgeh0 in #1056

New Contributors

• @banrovegrie made their first contribution in #1050

Full Changelog: v0.2.13...v0.2.14

v0.2.13

What's Changed

• fix(query-handler): allow QueryOutput without type var by @georgeh0 in #1041
• fix(shared-error): carry backtrace on the first cast to anyhow error by @georgeh0 in #1044
• fix(gemini): support output_dimension for Gemini by @georgeh0 in #1046
• fix(reqwest-send): always do error_for_status()? for send() by @georgeh0 in #1047
• docs: update LLM docs to add output_dimensions example by @georgeh0 in #1048

Full Changelog: v0.2.12...v0.2.13

v0.2.12

What's Changed

• Support LanceDB target
  • feat: add strong-typed representation of cocoindex type in Python by @georgeh0 in #1019
  • fix: always treat target as not-compatible when key schema changed by @georgeh0 in #1020
  • feat(py-target): pass optional fields schema to target connector by @georgeh0 in #1021
  • feat(py-sdk-spec): add the full fledged load_engine_object() by @georgeh0 in #1022
  • chore: use dmypy instead of mypy in pre-commit by @georgeh0 in #1023
  • build: upgrade Rust version by @georgeh0 in #1024
  • cleanup: run pre-commit hooks to cleanup files by @georgeh0 in #1025
  • feat(custom-target): pass indexing options to target connector by @georgeh0 in #1026
  • cleanup: clear a few build warnings by @georgeh0 in #1027
  • refactor: move targets in Python SDK to sub module by @georgeh0 in #1028
  • feat(custom-target): support compatibility and custom setup key type by @georgeh0 in #1029
  • feat(encode-types): encode coco type's python classes and test roundtrip by @georgeh0 in #1030
  • feat(auth-registry): add get_auth_registry() for Python by @georgeh0 in #1032
  • feat(lancedb): preliminary integration for lancedb target by @georgeh0 in #1031
  • fix: strip attributes for export targets by @georgeh0 in #1033
  • fix(lancedb): use fixed size list for vector by @georgeh0 in #1034
  • fix(lancedb): unsupport Union types for lancedb target by @georgeh0 in #1035
  • feat(lancedb): expose a shared connection for query by @georgeh0 in #1036
  • fix: dump NamedTuple to dict instead of list by @georgeh0 in #1037
  • feat(lancedb): create index by @georgeh0 in #1038
  • docs(lancedb): add documentation for lancedb by @georgeh0 in #1039

Full Changelog: v0.2.11...v0.2.12