Microsoft.data.sqlite by Felixmil · Pull Request #3323 · Open-Systems-Pharmacology/PK-Sim

Felixmil · 2025-11-04T10:11:33Z

Sister PRs:

Description

This PR migrates PK-Sim from System.Data.SQLite to Microsoft.Data.Sqlite. This change enables macOS ARM64 support, improves modern .NET compatibility.

Type of change

Please mark relevant options with an x in the brackets.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires documentation changes (link at least one user or developer documentation issue):
Algorithm update - updates algorithm documentation/questions/answers etc.
Other (please describe): Dependency migration.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Integration tests
Unit tests
Manual tests
No tests required

Reviewer checklist

Mark everything that needs to be checked before merging the PR.

Check if the code is well documented
Check if the behavior is what is expected
Check if the code is well tested
Check if the code is readable and well formatted
Additional checks (document below if any)
Check if documentation update issue(s) are created if the option This change requires a documentation update above is selected

Screenshots (if appropriate):

Questions (if appropriate):

- Replace System.Data.SQLite.Core with Microsoft.Data.Sqlite 8.0.0 - Update DAS layer, NHibernate configuration, and database schema migrator - Update all project files to use Microsoft.Data.Sqlite package - Enables macOS ARM64 support and modern .NET compatibility

- Update SessionFactoryProvider to use SQLite20Driver (compatible with Microsoft.Data.Sqlite) - Add proper error handling and logging to SnapshotRunner for silent failures - Move CloseSession call to outer finally block to ensure cleanup in all scenarios - Add comprehensive test coverage for error scenarios - Resolves issue where convertSnapshot function failed silently on macOS

… native libraries Copy native SQLite libraries from runtimes folder to root before creating the R dependencies artifact for Microsoft.Data.Sqlite compatibility.

Copilot

Pull Request Overview

This PR migrates the project from System.Data.SQLite.Core to Microsoft.Data.Sqlite, refactors simulation event handling to centralize it in InteractiveSimulationRunner, and adds support for resetting expression profile parameters. Key improvements include better error handling in the snapshot runner and updated build workflows.

Replaced all SQLite package references from System.Data.SQLite.Core to Microsoft.Data.Sqlite across all project files
Centralized simulation event publishing (SimulationRunStartedEvent/FinishedEvent) in InteractiveSimulationRunner
Added ResetExpressionParameterCommand for handling expression profile parameter resets
Enhanced error handling and logging in SnapshotRunner

Reviewed Changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
Multiple .csproj files	Updated SQLite package from System.Data.SQLite.Core to Microsoft.Data.Sqlite v8.0.0
DatabaseSchemaMigrator.cs	Updated SQLite class references from SQLiteConnection to SqliteConnection
DAS.cs	Updated provider factory initialization and data reading logic for Microsoft.Data.Sqlite
SessionFactoryProvider.cs	Updated SQLite connection string parameters
InteractiveSimulationRunner.cs	Centralized simulation event publishing with proper timing
IndividualSimulationEngine.cs	Removed event publishing (now handled by InteractiveSimulationRunner)
PopulationSimulationEngine.cs	Removed event publishing (now handled by InteractiveSimulationRunner)
StatusBarPresenter.cs	Updated to use InteractiveSimulationRunner for active simulation count
ParameterTask.cs	Added logic to use ResetExpressionParameterCommand for expression profile parameters
ResetParameterCommand.cs	Changed _oldValue visibility to protected for inheritance
ResetExpressionParameterCommand.cs	New command for resetting expression profile parameters
SnapshotRunner.cs	Enhanced error handling with better logging and exception handling
Build workflows	Updated code signing environment variables and R dependencies packaging

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/PKSim.Core/Services/InteractiveSimulationRunner.cs

src/PKSim.Infrastructure/ORM/DAS/DAS.cs

src/PKSim.CLI.Core/Services/SnapshotRunner.cs

.github/workflows/build-nightly_v13.yml

README.md

tests/PKSim.Tests/CLI/SnapshotRunnerSpecs.cs

- Resolved merge conflicts in: - SnapshotRunner.cs: Removed duplicate logging line - PKSim.Infrastructure.csproj: Kept Microsoft.Data.Sqlite, updated to V13 package versions - PKSim.csproj: Kept Microsoft.Data.Sqlite, updated to V13 versions, kept PostBuild target - PKSim.UI.Tests.csproj: Kept Microsoft.Data.Sqlite, updated to V13 versions - Deleted files removed in upstream/V13: - PKSimRDependencyResolution.csproj - PKSim.Matlab.Tests.csproj

…ration These properties (ContentSQLiteInteropFiles, CopySQLiteInteropFiles, etc.) were used by System.Data.SQLite.Core to handle native library deployment. Microsoft.Data.Sqlite handles native libraries through NuGet package assets automatically, so these properties are no longer needed.

- Resolved conflicts keeping Microsoft.Data.Sqlite migration - Removed Foreign Keys connection string parameter (Microsoft.Data.Sqlite has foreign keys enabled by default) - Kept manual data reading implementation in FillDataTable for Microsoft.Data.Sqlite compatibility - Preserved additional error handling test classes

This reverts commit 14b5b43.

.github/workflows/build-nightly_v13.yml

src/PKSim.CLI.Core/Services/SnapshotRunner.cs

- Add Content Include items to copy SQLite native libraries to output directory - Simplify workflow by removing PowerShell copy script, use -xr!runtimes flag

rwmcintosh · 2025-11-05T18:03:36Z

This won't build until we get a core merge, but this now works and only uses Microsoft.Data.SQLite where up to now it was using a hybrid of both.

src/PKSim.Infrastructure/ORM/DAS/DAS.cs

rwmcintosh · 2025-11-07T10:42:37Z

src/PKSim.Infrastructure/Services/SessionFactoryProvider.cs

-         var update = new SchemaUpdate(cfg);
-         update.Execute(useStdOut: false, doUpdate: true);


This won't work with Microsoft.Data.SQLite. Based on their public GitHub discussions, it won't work anytime soon.

We will have to manage migrations manually in the project database. That will also be required in MoBi.

We can write it in the developer docs.

Copilot

Pull Request Overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/PKSim.Infrastructure/Services/SessionFactoryProvider.cs

src/PKSim.Infrastructure/ORM/DAS/DAS.cs

src/PKSim.Infrastructure/Services/SessionFactoryProvider.cs

src/PKSim.Infrastructure/Serialization/DatabaseSchemaMigrator.cs

tests/PKSim.Tests/Infrastructure/SessionFactoryProviderSpecs.cs

tests/PKSim.Tests/IntegrationTests/LazyLoadingIntegrationTests.cs

rwmcintosh · 2025-11-10T21:03:00Z

I checked the file size differences against the V13 branch by loading a large snapshot file and saving that to pksim5 file.

They are very close.

rwmcintosh · 2025-11-10T21:10:53Z

After removing all the simulations as well.

src/PKSim/PKSim.csproj

msevestre

looks good to me with one question regarding weird command in one project file. Approving but please review just in case this is not intended

Felix added 3 commits November 4, 2025 10:39

Update R dependency artifact creation to handle Microsoft.Data.Sqlite…

53df268

… native libraries Copy native SQLite libraries from runtimes folder to root before creating the R dependencies artifact for Microsoft.Data.Sqlite compatibility.

This was referenced Nov 4, 2025

Migrate from System.Data.SQLite to Microsoft.Data.Sqlite 8.0.0 Open-Systems-Pharmacology/OSPSuite.Core#2699

Merged

Microsoft.data.sqlite Open-Systems-Pharmacology/MoBi#2163

Merged

Yuri05 changed the base branch from develop to V13 November 4, 2025 11:42

Yuri05 requested review from Yuri05, benjaperez1983, Copilot, msevestre and rwmcintosh and removed request for rwmcintosh November 4, 2025 11:42

Copilot AI reviewed Nov 4, 2025

View reviewed changes

Felix added 8 commits November 4, 2025 16:20

Disable foreign keys in Microsoft.Data.Sqlite connection string

ed84383

Fix duplicate error logging in SnapshotRunner

8d97412

Use Count property instead of Count() LINQ extension

14b5b43

Merge upstream/V13 into Microsoft.Data.Sqlite branch

cdee9bb

Revert "Use Count property instead of Count() LINQ extension"

8ee2c98

This reverts commit 14b5b43.

rwmcintosh reviewed Nov 5, 2025

View reviewed changes

.github/workflows/build-nightly_v13.yml Outdated Show resolved Hide resolved

rwmcintosh reviewed Nov 5, 2025

View reviewed changes

src/PKSim.CLI.Core/Services/SnapshotRunner.cs Outdated Show resolved Hide resolved

Felix and others added 3 commits November 5, 2025 16:51

Revert SnapshotRunner changes - out of scope for SQLite migration

5451bbf

Aligns with how other native DLLs are handled

bb7e503

- Add Content Include items to copy SQLite native libraries to output directory - Simplify workflow by removing PowerShell copy script, use -xr!runtimes flag

Update to use NHibernate.Extensions.SQLite and new core

b9838de

msevestre requested changes Nov 5, 2025

View reviewed changes

src/PKSim.Infrastructure/ORM/DAS/DAS.cs Show resolved Hide resolved

src/PKSim.Infrastructure/ORM/DAS/DAS.cs Outdated Show resolved Hide resolved

rwmcintosh reviewed Nov 5, 2025

View reviewed changes

src/PKSim.Infrastructure/ORM/DAS/DAS.cs Outdated Show resolved Hide resolved

do not automatically update schema from nhibernate

5dfc52c

rwmcintosh reviewed Nov 7, 2025

View reviewed changes

remove constructor init of factory

29be0d9

Yuri05 requested a review from Copilot November 8, 2025 20:07

Copilot AI reviewed Nov 8, 2025

View reviewed changes

rwmcintosh added 6 commits November 9, 2025 12:11

foreign keys enabled

e07b263

enable foreign keys

e03ced1

update core

6d3e50d

pr feedback

3a2dc3a

undo

e787068

the default connection did not enforce foreign keys

8e87fc9

rwmcintosh marked this pull request as ready for review November 10, 2025 16:38

rwmcintosh requested review from msevestre and rwmcintosh November 10, 2025 16:38

rwmcintosh marked this pull request as draft November 10, 2025 17:17

rwmcintosh linked an issue Nov 10, 2025 that may be closed by this pull request

Migrate from System.Data.SQLite to Microsoft.Data.Sqlite #3325

Closed

rwmcintosh added this to v13 Nov 10, 2025

update core

545c2b3

rwmcintosh marked this pull request as ready for review November 10, 2025 19:24

rwmcintosh requested review from rwmcintosh and removed request for rwmcintosh November 10, 2025 19:36

msevestre reviewed Nov 10, 2025

View reviewed changes

src/PKSim/PKSim.csproj Outdated Show resolved Hide resolved

msevestre approved these changes Nov 10, 2025

View reviewed changes

remove postbuild and unused references

1955361

Yuri05 approved these changes Nov 11, 2025

View reviewed changes

rwmcintosh merged commit 4479e14 into Open-Systems-Pharmacology:V13 Nov 11, 2025
1 check passed

github-project-automation bot moved this to Done in v13 Nov 11, 2025

Yuri05 moved this from Done to Verified in v13 Mar 10, 2026

		var update = new SchemaUpdate(cfg);
		update.Execute(useStdOut: false, doUpdate: true);

Conversation

Felixmil commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

How Has This Been Tested?

Reviewer checklist

Screenshots (if appropriate):

Questions (if appropriate):

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rwmcintosh commented Nov 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rwmcintosh Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

rwmcintosh Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rwmcintosh commented Nov 10, 2025

Uh oh!

rwmcintosh commented Nov 10, 2025

Uh oh!

Uh oh!

msevestre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Felixmil commented Nov 4, 2025 •

edited

Loading