You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: user/enterprise/tcie-3.x-migrating-db-from-2.x-to-3.x.md
+19-19Lines changed: 19 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,35 +8,35 @@ This document describes the steps necessary for database migration from an exist
8
8
9
9
## Overview
10
10
11
-
The TCIE 3.x runs over Postgresql v11 and features a bit different database schema, synchronized with Travis CI Hosted solution. Since TCIE 3.x is deployed as Kubernetes cluster, PostgreSQL is now one of the pods (TCIE 2.x had every dependency embedded in one large Docker image).
11
+
The TCIE 3.x runs over Postgresql v11 and features a slightly different database schema, synchronized with the Travis CI Hosted solution. Since TCIE 3.x is deployed as a Kubernetes cluster, PostgreSQL is now one of the pods (TCIE 2.x had every dependency embedded in one large Docker image).
12
12
13
13
The main schema differences are:
14
14
* In TCIE 3 databases for *logs* and *main* are now separated.
15
15
* The *_configs* schema in the new TCIE 3 *main* database is a bit different from the one in TCIE 2.2, and thus, this part will not be migrated during the following process; that means for builds done before the migration, the users won't have the `.travis.yml` contents relevant to these builds available in the Web User Interface.
16
16
17
17
## Migration Preparation
18
18
19
-
1.First, but optional step is removing unnecessary data to make your dump lighter. You can do it using [travis-backup gem](https://rubygems.org/gems/travis-backup). We recommend running the following commands after installing the gem:
19
+
1.The first but optional step is removing unnecessary data to make your dump lighter. You can do it using [travis-backup gem](https://rubygems.org/gems/travis-backup). We recommend running the following commands after installing the gem:
20
20
- to remove orphaned data: `travis_backup 'postgres://your_db_url' --remove_orphans`
21
21
- to remove requests, builds, jobs, and logs older than 6 months: `travis_backup 'postgres://your_db_url' --threshold 6`
22
-
2. You will need to dump existing TCIE 2.x (single docker image) PostgreSQL database into two files:
23
-
1. main database which should be a copy of current existing DB from Travis CI Enterprise 2.2.x (TCI E 2.2.x - one docker solution)
24
-
2. logs database which should contain only two tables: logs and log_parts from the main database
22
+
2. You will need to dump the existing TCIE 2.x (single docker image) PostgreSQL database into two files:
23
+
1. main database, which should be a copy of the current existing DB from Travis CI Enterprise 2.2.x (TCI E 2.2.x - one docker solution)
24
+
2. logs database, which should contain only two tables: logs and log_parts from the main database
25
25
3. For both aforementioned databases you need to have following credentials which will be used during Travis CI Enterprise 3.0.x (TCI E 3.0.x - k8s solution) installation process:
26
26
- DB name
27
27
- DB user
28
28
- DB password
29
29
- DB host
30
30
- DB port
31
-
4. You will need encryption key which can be found in current TCI E 2.2.x installation:
31
+
4. You will need an encryption key, which can be found in the current TCI E 2.2.x installation:
32
32
- ssh to platform
33
33
- call:
34
34
```bash
35
35
travis bash call: cat /usr/local/travis/etc/travis/config/travis.yml | grep -A 1 "encryption:"
36
36
```
37
-
- note it down on a piece of paper or at different computer
37
+
- note it down on a piece of paper or a different computer
38
38
39
-
> Before migrating the DB, we **strongly recommend**to store a DB snapshot or make additional copy of DB dump.
39
+
> Before migrating the DB, we **strongly recommend**storing a DB snapshot or making an additional copy of the DB dump.
As a result, you should have both database files available for further processing.
70
70
71
-
> This is the last moment you can make a backup of the dump files!
71
+
> This is the last moment you can back up the dump files!
72
72
73
73
### Alter the Database Dumps
74
74
@@ -80,9 +80,9 @@ sed -i '' 's/OWNER TO travis/OWNER TO postgres/g' TCI_E_2_0_db_schema_dump_main_
80
80
```
81
81
82
82
**PLEASE NOTE**:
83
-
1. Please reconfirm existing database owner name directly via PSQL query or command line tool. If required, unify/change it to a single user using psql tooling/queries before any further operation is executed.
84
-
1. If you use external database (not from the Travis CI Enterprise 2.2 release) and wish to continue so: you do not have to perform above step. Just use the database owner credentials from Travis CI Enterprise 2.2 in Travis Ci Enterprsie 3.x configuration.
85
-
2. If you use external database (not from the Travis CI Enterprise 2.2 release) and wish to cease to do so and use database delivered as a pod in Travis CI Enterprsie 3.3, in above bash commands change following fragments
83
+
1. Please reconfirm the existing database owner name directly via PSQL query or command line tool. If required, unify/change it to a single user using psql tooling/queries before any further operation is executed.
84
+
1. If you use an external database (not from the Travis CI Enterprise 2.2 release) and wish to continue so you do not have to perform the above step. Just use the database owner credentials from Travis CI Enterprise 2.2 in Travis CI Enterprise 3.x configuration.
85
+
2. If you use an external database (not from the Travis CI Enterprise 2.2 release) and wish to cease to do so and use a database delivered as a pod in Travis CI Enterprise 3.3, inthe above bash commands, change the following fragments
86
86
87
87
`Owner: travis` to `Owner: {your external tcie 2.2 database owner}`
88
88
@@ -96,7 +96,7 @@ Make your dump files available under any `http` address visible for your target
96
96
97
97
### Install TCIE 3.x Providing the Dumps and Key
98
98
99
-
Make sure you have provided the encryption key from existing TCIE 2.x installation in the TCIE 3.x GUI:
99
+
Make sure you have provided the encryption key from the existing TCIE 2.x installation in the TCIE 3.x GUI:
@@ -111,11 +111,11 @@ SQL Schema from 2.2 -> set to yes ->
111
111
Once the installer completes the work, your database should be successfully migrated.
112
112
113
113
> Important!
114
-
Due to the fact that TCIE 3.x uses original configs `raw_configs` (as in`.travis.yml`) and there is no way to restore such `raw_configs` from the old normalized versions from `[request|build|job]_configs` - these will be not migrated. The drawback is that no historical configs are available in the travis-web app for the end user.
114
+
Because TCIE 3.x uses original configs `raw_configs` (as in`.travis.yml`) and there is no way to restore such `raw_configs` from the old normalized versions from `[request|build|job]_configs` - these will be not migrated. The drawback is that no historical configs are available forthe end userin the travis-web app.
115
115
116
116
## Summary
117
-
- main DB from TCI E 2.X is split into two databases in TCIE 3.x: *main* and *logs*
118
-
- old configs from logs, builds, requests (column: `config`) are not migrated into new tables `[request|build|job]_configs`
119
-
- old not used config columns from `[requests|builds|jobs]` are dropped
120
-
- new tables are created for configs for future build runs: `[request|build|job]_configs`
121
-
- new table for raw config is created for future build runs `request_raw_configs`
117
+
- main DB from TCI E 2.X is split into two databases in TCIE 3.x: *main* and *logs*.
118
+
- old configs from logs, builds, and requests (column: `config`) are not migrated into new tables `[request|build|job]_configs`.
119
+
- old not used config columns from `[requests|builds|jobs]` are dropped.
120
+
- new tables are created for configs for future build runs: `[request|build|job]_configs`.
121
+
- new table for raw config is created for future build runs `request_raw_configs`.
0 commit comments