Merge pull request #354 from HarperFast/4.7

4.7 release

Author: heskew

docusaurus.config.ts

@@ -78,19 +78,19 @@ const config: Config = {
 						return `https://github.com/HarperDB/documentation/blob/main/docs/${docPath}`;
 					}
 				},
-				lastVersion: '4.6',
-				includeCurrentVersion: true,
+				lastVersion: '4.7',
+				includeCurrentVersion: false,
 				versions: {
-					'current': {
-						label: '4.7-beta',
-						banner: 'unreleased',
+					'4.7': {
+						// No banner for 4.7 as it's the latest stable version
+						banner: 'none',
 					},
 					'4.6': {
-						// No banner for 4.6 as its the latest stable version
+						// No banner for 4.6 as it's still actively maintained
 						banner: 'none',
 					},
 					'4.5': {
-						// No banner for 4.5 as its still actively maintained
+						// No banner for 4.5 as it's still actively maintained
 						banner: 'none',
 					},
 				},

package.json

@@ -8,7 +8,7 @@
 		"start": "npm run prebuild && docusaurus start",
 		"dev": "npm run prebuild && docusaurus start",
 		"build": "npm run prebuild && docusaurus build && node scripts/postbuild.js",
-		"version": "node scripts/cut-version.js",
+		"version": "docusaurus docs:version",
 		"swizzle": "docusaurus swizzle",
 		"deploy": "docusaurus deploy",
 		"clear": "npm run prebuild -- clean && docusaurus clear",

release-notes/v4-tucker/4.7.0.md

@@ -1,14 +1,10 @@
 ---
-title: 4.7.0 alpha
+title: 4.7.0
 ---
 
-# 4.7.0 alpha
+# 4.7.0
 
-8/15/2025
-
-:::warning Alpha Release
-This is an alpha release available on npm with the 'next' tag. This version should not be considered stable and is intended for testing and evaluation purposes only. Use in production environments is not recommended. See the [current tags on npm](https://www.npmjs.com/package/harperdb?activeTab=versions#current-tags) for details.
-:::
+10/16/2025
 
 - A new component status monitoring collects status from each component from loading and any registered notification of status changes.
 - OCSP is now supported, and can be used to invalidate TLS certificates used for replication and HTTP through an OCSP server.

release-notes/v4-tucker/index.mdx

@@ -8,7 +8,7 @@ import LatestPatchLink from '@site/src/components/LatestPatchLink';
 
 HarperDB version 4 ([Tucker release](v4-tucker/tucker)) represents major step forward in database technology. This release line has ground-breaking architectural advancements including:
 
-## <LatestPatchLink major={4} minor={7} label="4.7 alpha" />
+## <LatestPatchLink major={4} minor={7} />
 
 - Component status monitoring for tracking loading and status changes across all components
 - OCSP support for TLS certificate validation in replication and HTTP connections

versioned_docs/version-4.7/administration/_category_.json

@@ -0,0 +1,10 @@
+{
+	"label": "Administration",
+	"position": 2,
+	"link": {
+		"type": "generated-index",
+		"title": "Administration Documentation",
+		"description": "Guides for managing and administering HarperDB instances",
+		"keywords": ["administration"]
+	}
+}

versioned_docs/version-4.7/administration/administration.md

@@ -0,0 +1,32 @@
+---
+title: Best Practices and Recommendations
+---
+
+# Best Practices and Recommendations
+
+Harper is designed for minimal administrative effort, and with managed services these are handled for you. But there are important things to consider for managing your own Harper servers.
+
+### Data Protection and (Backup and) Recovery
+
+As a distributed database, data protection and recovery can benefit from different data protection strategies than a traditional single-server database. But multiple aspects of data protection and recovery should be considered:
+
+- Availability: As a distributed database Harper is intrinsically built for high-availability and a cluster will continue to run even with complete server(s) failure. This is the first and primary defense for protecting against any downtime or data loss. Harper provides fast horizontal scaling functionality with node cloning, which facilitates ease of establishing high availability clusters.
+- [Audit log](administration/logging/audit-logging): Harper defaults to tracking data changes so malicious data changes can be found, attributed, and reverted. This provides security-level defense against data loss, allowing for fine-grained isolation and reversion of individual data without the large-scale reversion/loss of data associated with point-in-time recovery approaches.
+- Snapshots: When used as a source-of-truth database for crucial data, we recommend using snapshot tools to regularly snapshot databases as a final backup/defense against data loss (this should only be used as a last resort in recovery). Harper has a [`get_backup`](./developers/operations-api/databases-and-tables#get-backup) operation, which provides direct support for making and retrieving database snapshots. An HTTP request can be used to get a snapshot. Alternatively, volume snapshot tools can be used to snapshot data at the OS/VM level. Harper can also provide scripts for replaying transaction logs from snapshots to facilitate point-in-time recovery when necessary (often customization may be preferred in certain recovery situations to minimize data loss).
+
+### Horizontal Scaling with Node Cloning
+
+Harper provides rapid horizontal scaling capabilities through [node cloning functionality described here](administration/cloning).
+
+### Monitoring
+
+Harper provides robust capabilities for analytics and observability to facilitate effective and informative monitoring:
+
+- Analytics provides statistics on usage, request counts, load, memory usage with historical tracking. The analytics data can be [accessed through querying](./reference/analytics).
+- A large variety of real-time statistics about load, system information, database metrics, thread usage can be retrieved through the [`system_information` API](./developers/operations-api/system-operations).
+- Information about the current cluster configuration and status can be found in the [cluster APIs](./developers/operations-api/clustering).
+- Analytics and system information can easily be exported to Prometheus with our [Prometheus exporter component](https://github.com/HarperDB-Add-Ons/prometheus_exporter), making it easy visualize and monitor Harper with Graphana.
+
+### Replication Transaction Logging
+
+Harper utilizes NATS for replication, which maintains a transaction log. See the [transaction log documentation for information on how to query this log](administration/logging/transaction-logging).

versioned_docs/version-4.7/administration/cloning.md

@@ -0,0 +1,153 @@
+---
+title: Clone Node
+---
+
+# Clone Node
+
+Clone node is a configurable node script that when pointed to another instance of Harper will create a clone of that
+instance's config, databases and setup full replication. If it is run in a location where there is no existing Harper install,
+it will, along with cloning, install Harper. If it is run in a location where there is another Harper instance, it will
+only clone config, databases and replication that do not already exist.
+
+Clone node is triggered when Harper is installed or started with certain environment or command line (CLI) variables set (see below).
+
+**Leader node** - the instance of Harper you are cloning.\
+**Clone node** - the new node which will be a clone of the leader node.
+
+To start clone run `harperdb` in the CLI with either of the following variables set:
+
+#### Environment variables
+
+- `HDB_LEADER_URL` - The URL of the leader node's operation API (usually port 9925).
+- `HDB_LEADER_USERNAME` - The leader node admin username.
+- `HDB_LEADER_PASSWORD` - The leader node admin password.
+- `REPLICATION_HOSTNAME` - _(optional)_ The clones replication hostname. This value will be added to `replication.hostname` on the clone node. If this value is not set, replication will not be set up between the leader and clone.
+
+For example:
+
+```
+HDB_LEADER_URL=https://node-1.my-domain.com:9925 REPLICATION_HOSTNAME=node-2.my-domain.com HDB_LEADER_USERNAME=... HDB_LEADER_PASSWORD=... harperdb
+```
+
+#### Command line variables
+
+- `--HDB_LEADER_URL` - The URL of the leader node's operation API (usually port 9925).
+- `--HDB_LEADER_USERNAME` - The leader node admin username.
+- `--HDB_LEADER_PASSWORD` - The leader node admin password.
+- `--REPLICATION_HOSTNAME` - _(optional)_ The clones clustering host. This value will be added to `replication.hostname` on the clone node. If this value is not set, replication will not be set up between the leader and clone.
+
+For example:
+
+```
+harperdb --HDB_LEADER_URL https://node-1.my-domain.com:9925 --REPLICATION_HOSTNAME node-2.my-domain.com --HDB_LEADER_USERNAME ... --HDB_LEADER_PASSWORD ...
+```
+
+Each time clone is run it will set a value `cloned: true` in `harperdb-config.yaml`. This value will prevent clone from
+running again. If you want to run clone again set this value to `false`. If Harper is started with the clone variables
+still present and `cloned` is true, Harper will just start as normal.
+
+Clone node does not require any additional configuration apart from the variables referenced above.
+However, if you wish to set any configuration during clone this can be done by passing the config as environment/CLI
+variables or cloning overtop of an existing `harperdb-config.yaml` file.
+
+More can be found in the Harper config documentation [here](../deployments/configuration).
+
+### Excluding database and components
+
+To set any specific (optional) clone config, including the exclusion of any database and/or replication, there is a file
+called `clone-node-config.yaml` that can be used.
+
+The file must be located in the `ROOTPATH` directory of your clone (the `hdb` directory where you clone will be installed.
+If the directory does not exist, create one and add the file to it).
+
+The config available in `clone-node-config.yaml` is:
+
+```yaml
+databaseConfig:
+  excludeDatabases:
+    - database: null
+  excludeTables:
+    - database: null
+      table: null
+componentConfig:
+  exclude:
+    - name: null
+```
+
+_Note: only include the configuration that you are using. If no clone config file is provided nothing will be excluded,
+unless it already exists on the clone._
+
+`databaseConfig` - Set any databases or tables that you wish to exclude from cloning.
+
+`componentConfig` - Set any components that you do not want cloned. Clone node will not clone the component code,
+it will only clone the component reference that exists in the leader harperdb-config file.
+
+### Cloning configuration
+
+Clone node will not clone any configuration that is classed as unique to the leader node. This includes `replication.hostname`, `replication.url`,`clustering.nodeName`,
+`rootPath` and any other path related values, for example `storage.path`, `logging.root`, `componentsRoot`,
+any authentication certificate/key paths.
+
+### Cloning system database
+
+Harper uses a database called `system` to store operational information. Clone node will only clone the user and role
+tables from this database. It will also set up replication on this table, which means that any existing and future user and roles
+that are added will be replicated throughout the cluster.
+
+Cloning the user and role tables means that once clone node is complete, the clone will share the same login credentials with
+the leader.
+
+### Replication
+
+If clone is run with the `REPLICATION_HOSTNAME` variable set, a fully replicating clone will be created.
+
+If any databases are excluded from the clone, replication will not be set up on these databases.
+
+### JWT Keys
+
+If cloning with replication, the leader's JWT private and public keys will be cloned. To disable this, include `CLONE_KEYS=false` in your clone variables.
+
+### Cloning overtop of an existing Harper instance
+
+Clone node will not overwrite any existing config, database or replication. It will write/clone any config database or replication
+that does not exist on the node it is running on.
+
+An example of how this can be useful is if you want to set Harper config before the clone is created. To do this you
+would create a harperdb-config.yaml file in your local `hdb` root directory with the config you wish to set. Then
+when clone is run it will append the missing config to the file and install Harper with the desired config.
+
+Another useful example could be retroactively adding another database to an existing instance. Running clone on
+an existing instance could create a full clone of another database and set up replication between the database on the
+leader and the clone.
+
+### Cloning steps
+
+Clone node will execute the following steps when ran:
+
+1. Look for an existing Harper install. It does this by using the default (or user provided) `ROOTPATH`.
+1. If an existing instance is found it will check for a `harperdb-config.yaml` file and search for the `cloned` value. If the value exists and is `true` clone will skip the clone logic and start Harper.
+1. Clone harperdb-config.yaml values that don't already exist (excluding values unique to the leader node).
+1. Fully clone any databases that don't already exist.
+1. If classed as a "fresh clone", install Harper. An instance is classed as a fresh clone if there is no system database.
+1. If `REPLICATION_HOSTNAME` is set, set up replication between the leader and clone.
+1. Clone is complete, start Harper.
+
+### Cloning with Docker
+
+To run clone inside a container add the environment variables to your run command.
+
+For example:
+
+```
+docker run -d \
+  -v <host directory>:/home/harperdb/hdb \
+  -e HDB_LEADER_PASSWORD=password \
+  -e HDB_LEADER_USERNAME=admin \
+  -e HDB_LEADER_URL=https://1.123.45.6:9925 \
+  -e REPLICATION_HOSTNAME=1.123.45.7 \
+  -p 9925:9925 \
+  -p 9926:9926 \
+  harperdb/harperdb
+```
+
+Clone will only run once, when you first start the container. If the container restarts the environment variables will be ignored.

versioned_docs/version-4.7/administration/compact.md

@@ -0,0 +1,60 @@
+---
+title: Compact
+---
+
+# Compact
+
+Database files can grow quickly as you use them, sometimes impeding performance. Harper has multiple compact features that can be used to reduce database file size and potentially improve performance. The compact process does not compress your data, it instead makes your database file smaller by eliminating free-space and fragmentation.
+
+There are two options that Harper offers for compacting a Database.
+
+_Note: Some of the storage configuration (such as compression) cannot be updated on existing databases, this is where the following options are useful. They will create a new compressed copy of the database with any updated configuration._
+
+More information on the storage configuration options can be [found here](../deployments/configuration#storage)
+
+### Copy compaction
+
+It is recommended that, to prevent any record loss, Harper is not running when performing this operation.
+
+This will copy a Harper database with compaction. If you wish to use this new database in place of the original, you will need to move/rename it to the path of the original database.
+
+This command should be run in the [CLI](../deployments/harper-cli)
+
+```bash
+harperdb copy-db <source-database> <target-database-path>
+```
+
+For example, to copy the default database:
+
+```bash
+harperdb copy-db data /home/user/hdb/database/copy.mdb
+```
+
+### Compact on start
+
+Compact on start is a more automated option that will compact **all** databases when Harper is started. Harper will not start until compact is complete. Under the hood it loops through all non-system databases, creates a backup of each one and calls copy-db. After the copy/compaction is complete it will move the new database to where the original one is located and remove any backups.
+
+Compact on start is initiated by config in `harperdb-config.yaml`
+
+_Note: Compact on start will switch `compactOnStart` to `false` after it has run_
+
+`compactOnStart` - _Type_: boolean; _Default_: false
+
+`compactOnStartKeepBackup` - _Type_: boolean; _Default_: false
+
+```yaml
+storage:
+  compactOnStart: true
+  compactOnStartKeepBackup: false
+```
+
+Using CLI variables
+
+```bash
+--STORAGE_COMPACTONSTART true --STORAGE_COMPACTONSTARTKEEPBACKUP true
+```
+
+```bash
+STORAGE_COMPACTONSTART=true
+STORAGE_COMPACTONSTARTKEEPBACKUP=true
+```

versioned_docs/version-4.7/administration/harper-studio/create-account.md

@@ -0,0 +1,27 @@
+---
+title: Create a Studio Account
+---
+
+# Create a Studio Account
+
+Start at the [Harper Studio sign up page](https://fabric.harper.fast/#/sign-up).
+
+1. Provide the following information:
+   - First Name
+   - Last Name
+   - Email Address
+   - Subdomain
+
+     _Part of the URL that will be used to identify your Harper Cloud Instances. For example, with subdomain "demo" and instance name "c1" the instance URL would be: [https://c1-demo.harperdbcloud.com](https://c1-demo.harperdbcloud.com)._
+
+   - Coupon Code (optional)
+
+1. Review the Privacy Policy and Terms of Service.
+1. Click the sign up for free button.
+1. You will be taken to a new screen to add an account password. Enter your password.
+   _Passwords must be a minimum of 8 characters with at least 1 lower case character, 1 upper case character, 1 number, and 1 special character._
+1. Click the add account password button.
+
+You will receive a Studio welcome email confirming your registration.
+
+Note: Your email address will be used as your username and cannot be changed.

versioned_docs/version-4.7/administration/harper-studio/enable-mixed-content.md

@@ -0,0 +1,9 @@
+---
+title: Enable Mixed Content
+---
+
+# Enable Mixed Content
+
+If you want to connect insecure HTTP instances from the secure HTTPS Fabric Studio, you can enable mixed content temporarily. This isn't recommended in production systems. It would be better to add HTTPS / SSL Termination in front of your instances. But if you understand the risks, you can enable mixed content. Enabling mixed content is required in cases where you would like to connect the Harper Studio to Harper Instances via HTTP. This should not be used for production systems, but may be convenient for development and testing purposes. Doing so will allow your browser to reach HTTP traffic, which is considered insecure, through an HTTPS site like the Studio.
+
+A comprehensive guide is provided by Adobe [here](https://experienceleague.adobe.com/docs/target/using/experiences/vec/troubleshoot-composer/mixed-content.html).