Merge pull request #109 from sqlitecloud/jacob/update-architecture

Staging test - Jacob/update architecture

Author: TizianoT

sqlite-cloud/platform/scaling.mdx _architecture.mdxsqlite-cloud/platform/scaling.mdx renamed to _architecture.mdx

@@ -1,11 +1,13 @@
----
-title: Scaling
-description: How to scale your SQLite Cloud cluster.
-category: platform
-status: publish
-slug: scaling
----
 
+
+## Architecture
+SQLite Cloud uses the [Raft](https://raft.github.io) consensus algorithm to distribute your data changes across a cluster of computing systems, ensuring that each node in the cluster agrees upon the same series of state transitions. Raft implements consensus with a leader approach.
+
+SQLite Cloud is written in ANSI C and GO, and it works on most POSIX systems (Linux, *BSD, Mac OS X) and Windows. 
+
+SQLite Cloud supports all the SQLite features without any limitations, including ACID compliance and non-deterministic SQL statements.
+
+## Scaling your cluster
 SQLite Cloud leverages a customized Raft algorithm to maintain a robust and highly available database cluster. Here’s an essential guide on the node types within SQLite Cloud and strategic tips for scaling your cluster effectively.
 
 ## Overview of Node types

sqlite-cloud/_nav.ts

@@ -2,9 +2,8 @@ import type { SidebarNavStruct } from "@docs-website/types/sidebar-navigation";
 
 const sidebarNav: SidebarNavStruct = [
 	{ title: "", type: "primary" },
-	{ title: "Getting Started", filePath:"getting-started", type: "secondary", icon: "docs-star" },
-	{ title: "Introduction", filePath:"introduction", type: "inner", level: 0 }, // should be index page for /docs and highlight "introduction" in nav
-	{ title: "Fundamentals", type: "inner", level: 0 },
+	{ title: "Introduction", type: "secondary", icon: "docs-star" },
+	{ title: "Getting Started", href: "/docs/sqlite-cloud", type: "inner", level: 0 },
 	{ title: "Connecting", filePath: "connect-cluster", type: "inner", level: 1 },
 	{ title: "Creating a database", filePath: "create-database", type: "inner", level: 1 },
 	{ title: "Writing data", filePath: "write-data", type: "inner", level: 1 },
@@ -17,11 +16,14 @@ const sidebarNav: SidebarNavStruct = [
 	{ title: "Next.js", filePath: "quick-start-next", type: "inner", level: 1 },
 	{ title: "Django", filePath: "quick-start-django", type: "inner", level: 1 },
 	{ title: "Flask", filePath: "quick-start-flask", type: "inner", level: 1 },
+	{ title: "SQLAlchemy", filePath: "quick-start-sqlalchemy-orm", type: "inner", level: 1 },
 	{ title: "Streamlit", filePath: "quick-start-streamlit", type: "inner", level: 1 },
 	{ title: "PHP / Laravel", filePath: "quick-start-php-laravel", type: "inner", level: 1 },
 	{ title: "Gin", filePath: "quick-start-gin", type: "inner", level: 1 },
 	{ title: "Tutorials", type: "inner", level: 0 },
 	{ title: "Geopoly", filePath: "tutorial-geopoly", type: "inner", level: 1 },
+	{ title: "Integrations", type: "inner", level: 0 },
+	{ title: "Knex.js", filePath: "knex-integration", type: "inner", level: 1 },
 
 	{ title: "Platform", filePath:"platform", type: "secondary", icon: "docs-plat" },
 	{ title: "Edge Functions", filePath: "edge-functions", type: "inner", level: 0 },
@@ -36,7 +38,6 @@ const sidebarNav: SidebarNavStruct = [
 	{ title: "Weblite", filePath: "weblite", type: "inner", level: 0 },
 		// { title: "Storage", type: "inner", level: 0 },
 	// { title: "Partitioning", type: "inner", level: 0 },
-	{ title: "Settings", filePath: "settings", type: "inner", level: 0 },
 
 	{ title: "SDKs", filePath:"sdks", type: "secondary", icon: "docs-sdk" },
 	{ title: "C/C++", type: "inner", level: 0 },
@@ -162,6 +163,7 @@ const sidebarNav: SidebarNavStruct = [
 	{ title: 'Introduction', type: "inner", filePath: "sdk-python-introduction", level: 1 },
 	{ title: "Django", ref: "/docs/quick-start-django", type: "inner", level: 1 },
 	{ title: "Flask", ref: "/docs/quick-start-flask", type: "inner", level: 1 },
+	{ title: "SQLAlchemy", ref: "/docs/quick-start-sqlalchemy-orm", type: "inner", level: 1 },
 
 	{ title: "Go",  type: "inner", level: 0 },
 	{ title: 'Introduction', type: "inner", filePath: "sdk-go-introduction", level: 1 },

sqlite-cloud/connect-cluster.mdx

@@ -8,10 +8,7 @@ slug: connect-cluster
 
 SQLite databases in SQLite Cloud are distributed across a cluster of nodes. Each cluster comes with a multi-region load balancer that routes traffic to the nearest appropriate node.
 
-For this reason, we strongly recommend connecting to your cluster via your project connection string. To retrieve your project connection string, navigate to the **Nodes** page and click on any node. 
-{/* ![Project connection string modal](@docs-website-assets/connect-cluster-1.png) */}
-
-Copy the connection string and use it with a client library to connect to your cluster.
+Click "Connect" in the bottom left-hand corner of your dashboard to get your connection string to use with a SQLite Cloud client library.
 
 ## Connecting with JavaScript
 Here's an example of how you can connect to your cluster using the `@sqlitecloud/drivers` JavaScript client library:
@@ -63,4 +60,5 @@ conn.close()
 ```
 
 ## Next Steps
+- [Creating a database](/docs/create-database)
 - [Writing data](/docs/write-data)

sqlite-cloud/create-database.mdx

@@ -6,13 +6,25 @@ status: publish
 slug: create-database
 ---
 
-SQLite Cloud allows you to import existing SQLite Databases, or create a new database in SQLite Cloud by importing an existing SQLite database, or using the SQLite Cloud UI, API, or client libraries.
+You can import an existing SQLite databases, or create new databases using the SQLite Cloud UI, API, or client libraries.
 
-## Importing an existing SQLite database
-SQLite Cloud allows you to import existing SQLite databases into the platform. 
+## Uploading an existing SQLite Database
+### Via HTTP API
+You can upload an existing SQLite database to your cluster using the SQLite Cloud UI or the Weblite API.
 
-Note that you can download, modify, and re-upload the database file at any time. You can also upload encrypted SQLite databases if you used the official SEE SQLite encryption extension.
+To upload a local SQLite database via weblite, make a POST request to the `/v2/weblite/<database-name>.sqlite` endpoint.
 
+```bash
+curl -X 'POST' \
+  'https://<your-project-id>.sqlite.cloud:8090/v2/weblite/<database-name>.sqlite' \
+  -H 'accept: application/json' \
+  -H 'Authorization: Bearer sqlitecloud://<your-project-id>.sqlite.cloud:8860?apikey=<your-api-key>' \
+  -d ''
+```
+
+To upload a local SQLite database via the SQLite Cloud UI, navigate to the Database tab in the left-hand navigation. Click the "Upload Database" button and select your local SQLite database.
+
+### Via Dashboard UI
 To import a database from the UI, navigate to the Databases tab and click the "Upload Database" button.
 ![Dashbord Upload Database](@docs-website-assets/introduction/dashboard_upload_db.png)
 
@@ -27,7 +39,7 @@ To create a new database from the SQLite Cloud UI, navigate to the Databases tab
 The default encoding is set to UTF-8, and the default page size is 4096KB.
 
 ### From the API
-To create a new database or upload an existing database via [Weblite](#), our REST API, you can make a request with the following parameters:
+To create a new database or upload an existing database via [Weblite](/docs/weblite), our REST API, you can make a request with the following parameters:
 ```bash
 curl -X 'POST' \
   'https://<your-project-id>.sqlite.cloud:8090/v2/weblite/<database-name>.sqlite' \
@@ -37,17 +49,32 @@ curl -X 'POST' \
 ```
 
 ### From client libraries
-To create a new database from a client library, use the CREATE DATABASE command. 
+To create a new database from a client library, connect to your cluster using a connection string without a specified database.
+
+Then, use the CREATE DATABASE command to create a new database. 
+
+To start using the database within the connection, you can use the `USE DATABASE` command.
 
 ```javascript
 import { Database } from '@sqlitecloud/drivers';
-
+// note that no database name is specified in the connection string path
 const db = new Database('sqlitecloud://<your-project-id>.sqlite.cloud:<your-host-port>?apikey=<your-api-key>')
 
 const createDatabase = async () => await db.sql`CREATE DATABASE <database-name>;`;
 
 createDatabase().then((res) => console.log(res));
 
 // "OK"
+
+db.exec('USE DATABASE <database-name>;')
+
+// now you can use the database
+const fetchAlbums = async () => await db.exec`SELECT * FROM albums;`;
+
+fetchAlbums().then((albums) => console.log(albums));
+
+// [{ Title: 'For Those About To Rock We Salute You', ... }, ...]
 ```
 
+## Next Steps
+- [Writing data](/docs/write-data)

sqlite-cloud/index.mdx

@@ -1,35 +1,29 @@
 ---
-title: Getting Started 
-description: Index page for getting started section
+title: Getting Started with SQLite Cloud
+description: SQLite Cloud is a distributed relational database system built on top of the SQLite database engine.
 category: getting-started
 status: publish
-icon: docs-star
-slug: getting-started
 ---
-import IndexPage from "@docs-website-components/Docs/IndexPage.astro"
 
-export const introduction = "SQLite Cloud is a distributed relational database system built on top of the SQLite database engine. It has been specifically designed from the ground up to ensure the strong consistency of your data across all nodes in a cluster while simultaneously managing the technical aspects of scaling, security, and data distribution."
+## Overview
+**SQLite Cloud** is a managed, distributed relational database system built on top of the SQLite database engine. 
 
-export const sections = [
-    {
-        icon: "curvedArrow",
-        title: "Introduction",
-        description: "SQLite Cloud introduction and getting started guide.",
-        href: "/docs/introduction",
-    },
-    {
-        icon: "twoColsGrid",
-        title: "Fundamentals",
-        description: "Learn how to connect to a cluster and start using SQLite Cloud.",
-        href: "/docs/connect-cluster",
-    },
-    {
-        icon: "puzzle",
-        title: "Quick start guide",
-        description: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.",
-        href: "/docs/quick-start-cdn",
-    }
-]
+It has been designed from the ground up to ensure strong consistency across all nodes in a cluster while simultaneously managing the technical aspects of scaling, security, and data distribution. This ensures that you can focus on your core tasks while relying on SQLite Cloud to handle the complexities of managing your databases.
 
+SQLite Cloud is built on the open source SQLite engine, ensuring complete feature parity. You get all of SQLite's core strengths: ACID compliance, support for complex SQL operations, and compatibility with the rich SQLite extension ecosystem.
 
-<IndexPage introduction={introduction} sections={sections} />
+You can access SQLite Cloud from the most popular programming languages or its REST API. 
+
+Like SQLite, each database in SQLite Cloud is a separate file, giving you flexible deployment options:
+
+* Create separate databases for each customer in a multi-tenant application
+* Share a single database among multiple users with built-in access controls
+* Mix both approaches based on your application's needs
+
+### Features
+SQLite Cloud provides a comprehensive suite of tools for building realtime, local-first, edge AI applications.
+* **[Webhooks](/docs/webhooks)**: Trigger edge functions or send change payloads via HTTP, Websockets, or on database events like INSERT, UPDATE, and DELETE.
+* **[Edge Functions](/docs/edge-functions)**: Run serverless functions on the same nodes that store your data for lightning-fast data access.
+* **[Pub/Sub](/docs/pub-sub)**: Subscribe to changes in your database to replicate data, power notifications, and build multiplayer experiences.
+* **[Weblite](/docs/weblite)**: Autogenerated REST APIs to interact with the SQLite Cloud platform.
+* **[Query Analyzer](/docs/analyzer)**: Receive optimization recommendations for your queries to improve performance.

sqlite-cloud/knex-integration.mdx

@@ -0,0 +1,154 @@
+---
+title: Knex.js Integration
+description: Integrate SQLite Cloud with Knex.js, a popular SQL query builder.
+category: getting-started
+status: publish
+slug: knex-integration
+---
+
+In this tutorial, we'll show you how to connect your TypeScript application to a SQLite Cloud database using the popular SQL builder, [Knex.js](https://knexjs.org/).
+
+---
+
+**Prerequisites**
+
+- Node.js and npm installed on your system
+- A SQLite Cloud account (you can sign up for a free account [here](https://sqlitecloud.io/register))
+
+1. **How to connect**
+
+- Create a Knex.js instance that uses the SQLite Cloud JavaScript driver to connect to your database.
+
+```typescript
+import 'dotenv/config'
+import { knex } from 'knex'
+
+const Client_SQLite3 = require('knex/lib/dialects/sqlite3')
+
+// client will have sqlite3 dialect, but will use sqlitecloud-js driver
+class Client_Libsql extends Client_SQLite3 {
+  _driver() {
+    return require('@sqlitecloud/drivers')
+  }
+}
+
+// Create a Knex.js instance with the custom SQLite3 client
+const db = knex({
+    client: Client_Libsql as any,
+    connection: {
+        filename: process.env.DATABASE_URL as string
+    }
+})
+```
+
+2. **Basic Usage**
+
+In this example, we will use the sample datasets that come pre-loaded with SQLite Cloud.
+
+- Initialize a new Node project:
+
+```bash
+npm init -y
+```
+
+- Install the required dependencies:
+
+```bash
+npm install @sqlitecloud/drivers knex dotenv --save
+```
+
+- Install the necessary development dependencies:
+
+```bash
+npm install @types/node nodemon ts-node typescript --save-dev
+```
+
+- Create a `.env` file in the root of your project and add your SQLite Cloud connection string:
+
+```bash
+DATABASE_URL="sqlitecloud://{USER}:{PASSWORD}@{HOST}.sqlite.cloud:8860"
+```
+
+Replace `{USER}`, `{PASSWORD}`, and `{HOST}` with your actual SQLite Cloud credentials and server hostname.
+
+- Create a `tsconfig.json` file to configure your TypeScript compiler:
+
+```bash
+tsc --init
+```
+
+- Create a new file called `example.ts` and add the following code:
+
+```typescript
+import 'dotenv/config'
+import { knex } from 'knex'
+
+const Client_SQLite3 = require('knex/lib/dialects/sqlite3')
+
+class Client_Libsql extends Client_SQLite3 {
+  _driver() {
+    return require('@sqlitecloud/drivers')
+  }
+}
+
+console.assert(process.env.DATABASE_URL, 'Define DATABASE_URL environment variable')
+
+const db = knex({
+  client: Client_Libsql as any,
+  connection: {
+    filename: process.env.DATABASE_URL as string
+  }
+})
+
+db.raw('USE DATABASE chinook.sqlite; SELECT * FROM customers')
+  .then(result => {
+    console.log(`Connected to database via knex and received ${result.length} rows`)
+    console.log(JSON.stringify(result, null, 2))
+    db.destroy()
+  })
+  .catch(err => {
+    console.error(err)
+    db.destroy()
+  })
+```
+
+- Update your `package.json` file to include a script for running the example:
+
+```bash
+{
+  "scripts": {
+    "dev": "nodemon --exec ts-node example.ts"
+  }
+}
+```
+
+- Start the development server:
+
+```bash
+npm run dev
+```
+
+This will run the `example.ts` file using `ts-node` and will automatically restart the server when you make changes to your code.
+
+- Observe the output in the console, which should display the customer data fetched from the SQLite Cloud database.
+```bash
+ [
+  {
+    "CustomerId": 1,
+    "FirstName": "Luís",
+    "LastName": "Gonçalves",
+    "Company": "Embraer - Empresa Brasileira de Aeronáutica S.A.",
+    "Address": "Av. Brigadeiro Faria Lima, 2170",
+    "City": "São José dos Campos",
+    "State": "SP",
+    "Country": "Brazil",
+    "PostalCode": "12227-000",
+    "Phone": "+55 (12) 3923-5555",
+    "Fax": "+55 (12) 3923-5566",
+    "Email": "[email protected]",
+    "SupportRepId": 3
+  },
+ ]
+```
+
+And that's it! You've successfully connected your TypeScript application to a SQLite Cloud database using Knex.js.

sqlite-cloud/platform/backups.mdx

@@ -7,7 +7,7 @@ slug: backups
 ---
 
 ## Overview
-Backups provide a robust solution for mitigating data loss and resolving data corruption issues. 
+Backups provide a robust solution for mitigating data loss and resolving data corruption issues. Backups are available for databases in all [Pro and Scale](https://www.sqlitecloud.io/pricing) projects.
 
 SQLite Cloud creates a full snapshot backup of your data once a day, and stores incremental changes once per second, on commodity object storage.