Merge pull request #133 from cipherstash/update-docs

Update all the docs

Author: tobyhede

DEVELOPMENT.md

@@ -318,6 +318,17 @@ This produces two SQL files in `releases/`:
 
 ## Structure
 
+### Adding SQL
+
+When adding new SQL files to the project, follow these guidelines:
+
+- Never drop the configuration table as it may contain customer data and needs to live across EQL versions
+- Everything else should have a `DROP IF EXISTS`
+- Functions should be `DROP` and `CREATE`, instead of `CREATE OR REPLACE`
+  - Data types cannot be changed once created, so dropping first is more flexible
+- Keep `DROP` and `CREATE` together in the code
+- Types need to be dropped last, add to the `666-drop_types.sql`
+
 ### Schema
 
 EQL is installed into the `eql_v2` PostgreSQL schema.

README.md

@@ -23,7 +23,10 @@ Store encrypted data alongside your existing data:
   - [Enable encrypted columns](#enable-encrypted-columns)
 - [Encrypt configuration](#encrypt-configuration)
 - [CipherStash integrations using EQL](#cipherstash-integrations-using-eql)
-- [Developing](#developing)
+- [Versioning](#versioning)
+  - [Upgrading](#upgrading)
+- [Troubleshooting](#troubleshooting)
+- [Contributing](#contributing)
 
 ---
 
@@ -169,7 +172,7 @@ You can find the EQL extension on [dbdev's extension catalog](https://database.d
 
 ## Getting started
 
-Once the custom types and functions are installed in your PostgreSQL database, you can start using EQL in your queries.
+Once EQL is installed in your PostgreSQL database, you can start using encrypted columns in your tables.
 
 ### Enable encrypted columns
 
@@ -178,12 +181,22 @@ Define encrypted columns using the `eql_v2_encrypted` type, which stores encrypt
 **Example:**
 
 ```sql
+-- Step 1: Create a table with an encrypted column
 CREATE TABLE users (
     id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
     encrypted_email eql_v2_encrypted
 );
+
+-- Step 2: Configure the column for encryption/decryption
+SELECT eql_v2.add_column('users', 'encrypted_email', 'text');
+
+-- Step 3: (Optional) Add search indexes
+SELECT eql_v2.add_search_config('users', 'encrypted_email', 'unique', 'text');
 ```
 
+> [!NOTE]
+> You must use [CipherStash Proxy](https://github.com/cipherstash/proxy) or [Protect.js](https://github.com/cipherstash/protectjs) to encrypt and decrypt data. EQL provides the database functions and types, while these tools handle the actual cryptographic operations.
+
 ## Encrypt configuration
 
 In order to enable searchable encryption, you will need to configure your CipherStash integration appropriately.
@@ -232,6 +245,28 @@ To upgrade to the latest version of EQL, you can simply run the install script a
 
 Follow the instructions in the [dbdev documentation](https://database.dev/cipherstash/eql) to upgrade the extension to your desired version.
 
-## Developing
+## Troubleshooting
+
+### Common Errors
+
+**Error: "Some pending columns do not have an encrypted target"**
+- **Cause**: Trying to configure a column that doesn't exist as `eql_v2_encrypted` type
+- **Solution**: First create the column: `ALTER TABLE table_name ADD COLUMN column_name eql_v2_encrypted;`
+
+**Error: "Config exists for column: table_name column_name"**
+- **Cause**: Attempting to add a column configuration that already exists
+- **Solution**: Use `eql_v2.add_search_config()` to add indexes, or `eql_v2.remove_column()` first to reconfigure
+
+**Error: "No configuration exists for column: table_name column_name"**
+- **Cause**: Trying to add search configuration before configuring the column
+- **Solution**: Run `eql_v2.add_column()` first, then add search indexes
+
+### Getting Help
+
+- Check the [full documentation](./docs/README.md)
+- Review [CipherStash Proxy configuration guide](./docs/tutorials/proxy-configuration.md)
+- Report issues at [https://github.com/cipherstash/encrypt-query-language/issues](https://github.com/cipherstash/encrypt-query-language/issues)
+
+## Contributing
 
-See the [development guide](./DEVELOPMENT.md).
+See the [development guide](./DEVELOPMENT.md) for information on developing and extending EQL.

docs/README.md

@@ -8,9 +8,11 @@ This directory contains the documentation for the Encrypt Query Language (EQL).
 
 ## Reference
 
-- [EQL index configuration for CipherStash Proxy](reference/index-configuration.md)
+- [EQL Functions Reference](reference/eql-functions.md) - Complete API reference for all EQL functions
+- [Database Indexes for Encrypted Columns](reference/database-indexes.md) - PostgreSQL B-tree index creation and usage
+- [EQL index configuration for CipherStash Proxy](reference/index-config.md)
 - [EQL with JSON and JSONB](reference/json-support.md)
-- [EQL payload data format](reference/eql-payload.md)
+- [EQL payload data format](reference/PAYLOAD.md)
 
 ## Tutorials