Merge pull request #1155 from MinaProtocol/dkijania/mesa_preflight

documentation for preflight network for mesa

Author: deepthiskumar

docs/mesa-upgrade/archive-upgrade.mdx

@@ -0,0 +1,158 @@
+---
+title: Archive Upgrade
+sidebar_label: Archive Upgrade
+hide_title: true
+description: A guide on how to upgrade your Mina archive node to the Mesa version.
+keywords:
+  - Mesa
+  - upgrade
+  - archive
+  - mina archive node
+  - archive node
+---
+
+# Archive Upgrade
+
+To successfully upgrade the archive database into the Mesa version of the Mina network, you must ensure that your environment meets the foundational requirements.
+
+## Migration host
+
+- PostgreSQL database for database server
+- If you use Docker, then any of the supported OS by Mina (bullseye, focal, noble, bookworm or jammy) with at least 32 GB of RAM
+- gsutil application from Google Cloud Suite in version 5 or later
+- (Optional) Docker in version 23.0 or later
+
+## Archive database
+
+One of the most obvious prerequisites is a Mainnet database. If you don't have an existing database with Devnet/Mainnet archive data,
+you can always download it from the O1Labs Google Cloud bucket.
+
+## Upgrade process
+
+### Upgrade script
+
+Assuming that you have a PostgreSQL database with Mainnet archive data, in order to upgrade it to Mesa version, you need to run SQL upgrade script.
+We put all efforts to make the upgrade process as smooth as possible. Script can be run on archive node which is online or offline.
+Script can be run multiple times, it will skip steps that were already completed. It also performs sanity checks before each step to ensure that the upgrade process is successful.
+Finally it creates new table (version) in the database to keep track of the upgrade process.
+
+#### Getting the script
+
+You can find the SQL upgrade script in the Mina repository on GitHub. Make sure to download the latest version of the script before proceeding.
+You can download the script directly using the following command:
+
+```bash
+curl -O https://raw.githubusercontent.com/MinaProtocol/mina/refs/heads/mesa/src/app/archive/upgrade_to_mesa.sql
+```
+
+We also ship the script in the Mina archive Docker image and Debian package.
+
+```bash
+docker run --rm gcr.io/o1labs-192920/mina-archive-mesa:4.0.0-preflight1-b649c79-bookworm-mesa cat /etc/mina/archive/upgrade-to-mesa.sql > upgrade-to-mesa.sql
+```
+
+```bash
+# Setup the Mina repository and install the archive package
+# See preflight-network.mdx for detailed repository setup instructions
+
+apt-get install mina-archive-mesa=4.0.0-preflight1-b649c79
+
+# View the upgrade and downgrade scripts
+cat /etc/mina/archive/upgrade-to-mesa.sql
+cat /etc/mina/archive/downgrade-to-berkeley.sql
+```
+
+#### Running the script
+
+:::caution Database Backup
+
+Before running the upgrade script, **backup your archive database**. The upgrade modifies the database schema.
+
+```bash
+pg_dump -U <username> <database> > berkeley-archive-backup.sql
+```
+
+:::
+
+To run the upgrade script, execute the following command:
+
+```bash
+psql -U <username> -d <database> -f upgrade-to-mesa.sql
+```
+
+Make sure to replace `<username>` and `<database>` with your actual PostgreSQL username and database name.
+
+
+#### Rollback
+
+You can rollback the upgrade process by restoring the database from a backup taken before running the upgrade script.
+Another is to run rollback script which is part of the upgrade script. It will drop all tables and other database objects created by the upgrade script.
+It will also update the version table to reflect the rollback.
+
+##### Running the rollback script
+
+To run the rollback script, you need to execute the following command:
+
+```bash
+psql -U <username> -d <database> -f /etc/mina/archive/downgrade-to-berkeley.sql
+```
+
+Make sure to replace `<username>` and `<database>` with your actual PostgreSQL username and database name.
+
+
+### Post-upgrade steps
+
+After successfully running the upgrade script, you DO NOT need to restart your archive node or Rosetta API.
+Changes in upgrade script are backward compatible and will be picked up by the archive node and Rosetta API automatically.
+
+### Verification
+To verify that the upgrade was successful, you can check the version table in the PostgreSQL database.
+
+You can do this by running the following command:
+```bash
+psql -U <username> -d <database> -c "SELECT * FROM version;"
+```
+Make sure to replace `<username>` and `<database>` with your actual PostgreSQL username and database name.
+
+If the upgrade was successful, you should see the new version number in the output.
+
+We put a lot of effort into making the upgrade process as smooth as possible.
+However, if you encounter any issues or need assistance, please reach out to the Mina community on [Discord](https://discord.gg/minaprotocol) or [GitHub Discussions](https://github.com/MinaProtocol/mina/discussions).
+
+## Appendix: Database Schema Changes
+
+Below we present details of what was changed in the archive node database schema between Berkeley and Mesa versions.
+
+### Zkapp_state_nullable Additional Columns
+
+The `zkapp_state_nullable` table has been modified to include new columns `element8` through `element31` which are nullable and can store additional state information for zkApps.
+
+```sql
+, element8          int     REFERENCES zkapp_field(id)
+...
+, element31         int     REFERENCES zkapp_field(id)
+);
+```
+
+This expansion allows zkApps to store up to 31 state elements instead of the previous 8, significantly increasing the state storage capacity for complex smart contracts.
+
+### Version Table
+
+We also introduced a new table `version` to keep track of the database schema version.
+The purpose of this table is to help with future database migrations. The table tracks which migration scripts were applied and when.
+Ultimately it helps to determine the current version of the database schema and helps to avoid applying the same migration script multiple times.
+
+This table is created if it does not exist already. Rollback and upgrade scripts will insert a new row with the version number and timestamp when the script was applied.
+
+```sql
+CREATE TABLE IF NOT EXISTS version (
+    version_num INT PRIMARY KEY,
+    applied_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+The version table provides:
+- **Migration tracking**: Records which migrations have been applied
+- **Timestamp tracking**: Shows when each migration was executed
+- **Idempotency**: Prevents duplicate migration runs
+- **Version identification**: Easily identify the current database schema version