STAND-109: Replace the embedded mysql with mariadb4j by Muta-Jonathan · Pull Request #68 · openmrs/openmrs-standalone

Muta-Jonathan · 2025-05-01T19:07:53Z

See https://openmrs.atlassian.net/browse/STAND-109

Description

The change migrates the embedded database used by openmrs-standalone from mysql mxj to MariaDB4j (see https://github.com/MariaDB4j/MariaDB4j)

In order to keep the PR simple I opted to continue using the mysql driver instead of a MariaDB driver as mysql drivers are compatible with MariaDB.

on this PR, uses java version to 8 as the minimum java version. Also using MariaDB4J (3.2.0v) latest version with mac silicon (M1,M2+) support

Testing

In addition to the new unit tests I also performed the below manual tests and not sure if I need to do any validation on the other artifacts that get generated from packaging.

NOTE: Some liquidbase files in script testing are outdated in line with the current openmrs tables thats why are commented out and unused

Use cases

Am using a mac M2 chip and the following are the steps needed to run the openmrs-standalone-2.4.0

After clone, run mvn clean
then mvn package
Copy generated openmrs-standalone/target/openmrs-standalone-2.4.0-SNAPSHOT.zip zipped file to any desired destination on unzip from there
Make sure u have java installed can check using java --version in your terminal
Inside your unzipped folder double click on openmrs-standalone.jar
And start your openmrs instance

ScreenShots

Showing the folder

Muta-Jonathan · 2025-05-01T19:08:43Z

cc @dkayiwa @wikumChamith

Muta-Jonathan · 2025-05-01T19:14:09Z

src/main/java/org/openmrs/standalone/MariaDbController.java

+
+        // Grant privileges to openmrs user
+        mariaDB.run("GRANT ALL PRIVILEGES ON *.* TO '" + DATABASE_USER_NAME + "'@'localhost';");
+        mariaDB.run("FLUSH PRIVILEGES;");


used direct sql here but these above commands can be compiled in a sql file and sourced using mariaDB.source(fileName) but for now as per this PR's aim this should still be good

dkayiwa · 2025-05-02T12:12:46Z

When i build and run, on choosing the Demonstration mode option, i get these errors in the logs: https://pastebin.com/DF2hiztT

Muta-Jonathan · 2025-05-02T12:26:30Z

I believe these come from the commented out liquidbase changesets @dkayiwa do we have any recent updated liquidbase changesets i can use

liquibase.cieldata
liquibasechangelog (liquibase.databaseChangeLogTableName)

Muta-Jonathan · 2025-05-02T12:34:23Z

When i build and run, on choosing the Demonstration mode option, i get these errors in the logs: https://pastebin.com/DF2hiztT

Also seems there is a database connectivity issue let me look into it 🙂

…rd=true

Muta-Jonathan · 2025-05-04T21:56:35Z

cc @dkayiwa

dkayiwa · 2025-05-04T21:58:12Z

Does this fix the runtime error that i reported?

Muta-Jonathan · 2025-05-04T22:03:01Z

Does this fix the runtime error that i reported?

oh yes now on end get the tables error https://pastebin.com/2Wh4KygA which will need just a bump to the latest dump sql openmrs file

dkayiwa · 2025-05-04T22:05:06Z

Meaning that this is not yet ready for review and testing?

Muta-Jonathan · 2025-05-04T22:11:34Z

sure in the meantime am going to convert it to draft .. however am wondering if we have archives for the latest dump files I can use inorder to fix the sql table errors also updated liquibase files to try out on the latest version

dkayiwa · 2025-05-04T22:12:21Z

And FWIW, now the mvn package stage results into this: https://pastebin.com/SWD8mpSF

Muta-Jonathan · 2025-05-13T13:59:30Z

cc @wikumChamith @dkayiwa

dkayiwa · 2025-05-13T19:41:39Z

Choosing the demo data option takes me to the The OpenMRS 2.8.0 server is currently in maintenance mode. page. It is supposed to do all the setup behind the scenes and just take me to the log in page.

Muta-Jonathan · 2025-05-15T18:42:18Z

On this small adjustment have been able to load the updated demo data which have named 2.0.0 however found out that in liquibase-update-to-latest.xml we have changesets with defined dbms="mysql" such as <changeSet id="20100128-1" author="djazayeri" dbms="mysql"> and this to note comes from the core which is resulting into

resulting into errors like

Running Changeset: liquibase-update-to-latest.xml::20090402-1516-encounter_type::bwolfe
INFO - Slf4JLogger.log(43) |2025-05-15T21:23:20,743| Marking ChangeSet: "liquibase-update-to-latest.xml::20090402-1516-encounter_type::bwolfe" as ran despite precondition failure due to onFail='MARK_RAN': 
          liquibase-update-to-latest.xml : DBMS Precondition failed: expected mysql, got mariadb
          liquibase-update-to-latest.xml : DBMS Precondition failed: expected oracle, got mariadb

find the full errors here https://pastebin.com/LbhM5DRY

cc @dkayiwa @wikumChamith

dkayiwa · 2025-05-15T20:47:34Z

@Muta-Jonathan which version of liquibase is the standalone configured with?

Muta-Jonathan · 2025-05-15T21:31:21Z

liquibase

4.4.1

dkayiwa · 2025-05-15T21:35:24Z

Does it help if you switch it to 4.31.1?

Muta-Jonathan · 2025-05-15T21:36:19Z

Does it help if you switch it to 4.31.1?

let me try that

dkayiwa · 2025-05-15T21:39:17Z

@Muta-Jonathan i have just run your changes but i still get the The OpenMRS 2.8.0 server is currently in maintenance mode. I should instead see the normal login screen instead of the setup login page.

Muta-Jonathan · 2025-05-15T21:45:51Z

@Muta-Jonathan i have just run your changes but i still get the The OpenMRS 2.8.0 server is currently in maintenance mode. I should instead see the normal login screen instead of the setup login page.

Am trying to figure out why the war file is not triggering the setup wizard which will then re-direct to the login in page but currently unable to find the cause .... however seen in core for the installation wizard to load the setup must be missing openmrs-runtime.properties For this i dont know for sure if am oon the right track could there be something am missing?

…ptyDatabase

…iadb directories

…n install

…-data.xml

Muta-Jonathan · 2025-05-14T03:22:20Z

src/main/assembly/zip-demo-data.xml

 		<fileSet>
 			<outputDirectory>/</outputDirectory>
-			<directory>${project.build.directory}/demodatabase/data</directory>
+<!--			<directory>${project.build.directory}/demodatabase/data</directory>-->


Choosing the demo data option takes me to the The OpenMRS 2.8.0 server is currently in maintenance mode. page. It is supposed to do all the setup behind the scenes and just take me to the log in page.

I believe this is happening because of this line in trying to zip the demodatabase it fails here saying the demo is not in the directory

Muta-Jonathan · 2025-05-14T03:28:01Z

pom-step-04.xml

 						<configuration>
 							<tasks>
-\								<unzip src="${liquibase.demodata.archive}" dest="." />
+								<unzip src="${liquibase.demodata.archive}" dest="." />


which executes here named in the pom.xml as
<liquibase.demodata.archive>Demo-1.9.0.sql.zip</liquibase.demodata.archive>
And since the Unzipped db at this stage contains no tables and schema for the Db ... the tables are not created hence the re-direct to Maintainance page

Muta-Jonathan · 2025-05-16T12:50:16Z

src/main/java/org/openmrs/standalone/MariaDbController.java

+        mariaDB.run("FLUSH PRIVILEGES;");
+
+        // Create the OpenMRS database schema if it doesn't exist
+        mariaDB.createDB(DATABASE_NAME, DATABASE_USER_NAME, userPassword);


Just found the issue causing the installation wizard to fail to start ... this is due to creating the Openmrs database schema here which tells the core that already the scheme is available ...

Muta-Jonathan · 2025-05-29T09:34:48Z

.github/workflows/standalone-ci.yml

  build-macos:
    name: on macOS (Java 21)
-    runs-on: macos-latest
+    runs-on: macos-15


Was getting this error on Ci

dyld[2778]: Symbol not found: _vidattr$NCURSES60
Referenced from: mariadb (built for macOS 15.0 which is newer than running OS)
Expected in: /usr/lib/libncurses.5.4.dylib

According to this so far github maps macos-latest to OS Version: macOS 14.7.6 (23H626) https://arc.net/l/quote/tdcfqkkj ...now had to manually specify macos-15

Muta-Jonathan · 2025-05-29T09:38:33Z

.github/workflows/standalone-ci.yml


      - name: Build and Test with Maven
-        run: mvn clean install
+        run: mvn clean install -DskipTests


Generally this i placed it here because the maven liquibase tests start the db which later seems to delay to close for macos and windows conflicting with the normal java tests causing database fails to start

Muta-Jonathan · 2025-05-29T09:40:01Z

.github/workflows/standalone-ci.yml


      - name: Build and Test with Maven
-        run: mvn clean install
+        run: mvn clean install -DskipTests


But the question still stands ...let me try to refactor for both mac and windows

Muta-Jonathan · 2025-06-16T13:21:03Z

.github/workflows/standalone-ci.yml

  build-macos:
    name: on macOS (Java 21)
-    runs-on: macos-latest
+    runs-on: macos-15


yes @dkayiwa explained the error i was getting above and why i used macos-15

Was getting this error on Ci

dyld[2778]: Symbol not found: _vidattr$NCURSES60
Referenced from: mariadb (built for macOS 15.0 which is newer than running OS)
Expected in: /usr/lib/libncurses.5.4.dylib

According to this so far github maps macos-latest to OS Version: macOS 14.7.6 (23H626) https://arc.net/l/quote/tdcfqkkj ...now had to manually specify macos-15

dkayiwa · 2025-06-16T17:25:44Z

packagezip.sh

 cp ../target/*database.zip .
 ln -s ../tomcat tomcat
 cp ../*.png .
-cp ../openmrs-standalone-runtime.properties .


Is there a reason why you renamed this?

yes this was coping ../openmrs-standalone-runtime.properties which doesnot exist basically it is now ../openmrs-runtime.properties

dkayiwa · 2025-06-16T17:26:04Z

packagezip.sh

-cp ../openmrs-standalone-runtime.properties .
+cp ../openmrs-runtime.properties .
 cp ../target/*.jar .
-cp ../target/openmrs-standalone.war tomcat/webapps/openmrs-standalone.war


Is there a reason why you renamed this?

yes this was coping ../target/openmrs-standalone.war yet in target we have ../target/openmrs.war also the in the standalone code we look for openmrs.war in the webapps directory not openmrs-standalone.war

What did we do to change it from the original name?

from this commit here Muta-Jonathan@c3ea3fc i see that Raf removed it and later openmrs.war was introduced Muta-Jonathan@305097c

this commit explains why it has been like that for a while ... this is the last commit before my changes Muta-Jonathan@248ac66 here i see in pom.xml that

<configuration>Add commentMore actions <artifactItems> <artifactItem> <groupId>org.openmrs.web</groupId> <artifactId>openmrs-webapp</artifactId> <version>${openmrs.version}</version> <type>war</type> <outputDirectory>target</outputDirectory> <destFileName>openmrs-standalone.war</destFileName> </artifactItem> </artifactItems> </configuration>

which shows that the packagezip.sh expected the change also done in that commit cp ../target/openmrs-standalone.war tomcat/webapps/openmrs-standalone.war which later even after the update to openmrs.war in the pom.xml this Muta-Jonathan@305097c wasnt addressed

pom-step-02.xml

pom-step-03.xml

dkayiwa · 2025-06-16T17:31:46Z

src/main/java/org/openmrs/standalone/ApplicationController.java

 				StandaloneUtil.resetConnectionPassword();
-				StandaloneUtil.startupDatabaseToCreateDefaultUser();
+				StandaloneUtil.startupDatabaseToCreateDefaultUser(mySqlPort);
+				System.out.println("Database mode using wizard: " + applyDatabaseChange );


Do you still need the System.out.println statements?

Added these to be shown in the logs

Replace the embedded mysql with mariadb4j

33f3ca3

Muta-Jonathan commented May 1, 2025

View reviewed changes

Muta-Jonathan added 2 commits May 5, 2025 00:53

Addressing the connection.password issue when reset_connection_passwo…

dac0275

…rd=true

removed unnecessary variables

db96960

Muta-Jonathan marked this pull request as draft May 4, 2025 22:12

Muta-Jonathan added 3 commits May 5, 2025 01:21

removed liquibase.plugin.version.2.x to use liquibase.plugin.version.3.x

4884456

removed liquibase.plugin.version.4.x to use liquibase.plugin.version.3.x

9443e36

replacing mysql driver with org.mariadb.jdbc.Driver

74b34ad

Muta-Jonathan marked this pull request as ready for review May 13, 2025 13:52

removing import wildcard

7e91f37

Adding import sql logic for demo database

c212a3c

Muta-Jonathan added 18 commits May 30, 2025 10:25

refactoring windows ci setup to use test password

3e42e22

adding user root as an arg

4d2ee1e

adding root password

eaed266

uncommenting the liquibase changes for preloading demoDatabase and em…

6f212e7

…ptyDatabase

Merge remote-tracking branch 'origin/STAND-109' into STAND-109

34043a5

removing import sql and extraction to use the zipped demodatabase mar…

6b3f6b3

…iadb directories

correcting demo sql path

983886f

adding mariadb to dbms and bumping back to 1.9.0 demo sql

b5271ce

solving the heap size

c34aeff

increase java heap size via github actions

697293e

refactoring to import demo sql via plugin

7bedc63

Merge remote-tracking branch 'origin/STAND-109' into STAND-109

fd7da2f

removed MAVEN_OPTS .... in github actions | refactor to clean and the…

5942003

…n install

removing commented dependencies and updated comment in liquibase-demo…

10c708f

…-data.xml

Refactoring the windows CI to give it permissions

b56cc79

reverting and removing the grant PRIVILEGES for runneradmin

440fa0c

using alter instead of set password

cd5b7b1

skipping tests for windows CI

41ecec8

Muta-Jonathan requested a review from dkayiwa June 16, 2025 07:30

Muta-Jonathan commented Jun 16, 2025

View reviewed changes

dkayiwa reviewed Jun 16, 2025

View reviewed changes

pom-step-02.xml Outdated Show resolved Hide resolved

dkayiwa reviewed Jun 16, 2025

View reviewed changes

pom-step-02.xml Show resolved Hide resolved

dkayiwa reviewed Jun 16, 2025

View reviewed changes

pom-step-03.xml Outdated Show resolved Hide resolved

dkayiwa reviewed Jun 16, 2025

View reviewed changes

Muta-Jonathan added 2 commits June 16, 2025 22:21

removing the MXJ connection url specific parameters

2073134

adding back the macos-15 build for ci

61eca04

Muta-Jonathan requested a review from dkayiwa June 16, 2025 19:38

dkayiwa merged commit 2f591f8 into openmrs:master Jun 16, 2025
2 of 3 checks passed

Conversation

Muta-Jonathan commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Use cases

ScreenShots

Showing the folder

Uh oh!

Muta-Jonathan commented May 1, 2025

Uh oh!

Muta-Jonathan May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dkayiwa commented May 2, 2025

Uh oh!

Muta-Jonathan commented May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Muta-Jonathan commented May 2, 2025

Uh oh!

Muta-Jonathan commented May 4, 2025

Uh oh!

dkayiwa commented May 4, 2025

Uh oh!

Muta-Jonathan commented May 4, 2025

Uh oh!

dkayiwa commented May 4, 2025

Uh oh!

Muta-Jonathan commented May 4, 2025

Uh oh!

dkayiwa commented May 4, 2025

Uh oh!

Muta-Jonathan commented May 13, 2025

Uh oh!

dkayiwa commented May 13, 2025

Uh oh!

Muta-Jonathan commented May 15, 2025

Uh oh!

dkayiwa commented May 15, 2025

Uh oh!

Muta-Jonathan commented May 15, 2025

Uh oh!

dkayiwa commented May 15, 2025

Uh oh!

Muta-Jonathan commented May 15, 2025

Uh oh!

dkayiwa commented May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Muta-Jonathan commented May 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Muta-Jonathan commented May 1, 2025 •

edited

Loading

Muta-Jonathan May 1, 2025 •

edited

Loading

Muta-Jonathan commented May 2, 2025 •

edited

Loading

dkayiwa commented May 15, 2025 •

edited

Loading