|
1 | 1 | --- |
2 | 2 | title: RDBMS Background Jobs |
3 | | -summary: Run Background Jobs and Scheduled Tasks in PostgreSQL, SQL Server or MySQL |
| 3 | +summary: Run Background Jobs and Scheduled Tasks in PostgreSQL, SQL Server or MySQL |
4 | 4 | tags: [db,ormlite,jobs] |
5 | 5 | author: Demis Bellot |
6 | 6 | image: ./img/posts/background-jobs/bg.webp |
7 | | -draft: true |
8 | 7 | --- |
9 | 8 |
|
10 | 9 | We're excited to announce that we've ported our much loved [Background Jobs](https://docs.servicestack.net/background-jobs) |
11 | | -feature for SQLite to the popular **PostgreSQL**, **SQL Server** and **MySQL** RDBMS's. |
| 10 | +feature to the popular **PostgreSQL**, **SQL Server** and **MySQL** RDBMS's! |
12 | 11 |
|
13 | | -Whilst we love [SQLite + Litestream](https://docs.servicestack.net/ormlite/litestream) for its low dev ops maintenance |
14 | | -allowing us to break free from |
15 | | -[expensive cloud hosting hosts](https://docs.servicestack.net/ormlite/litestream#the-right-time-for-server-side-sqlite) |
16 | | -for managed RDBMS's, it's clear many of our Customers need the features of an industrial strength RDBMS. |
| 12 | +Since launching [Background Jobs](https://servicestack.net/posts/background-jobs) in September 2024, it's become |
| 13 | +one of our most popular features - providing a simple, infrastructure-free solution for managing background jobs |
| 14 | +and scheduled tasks in .NET 8+ Apps. The original implementation used SQLite for its durability, which worked |
| 15 | +beautifully for many use cases thanks to SQLite's low latency, fast disk persistence, and zero infrastructure requirements. |
17 | 16 |
|
18 | | -In future we'll also be looking at providing a great self-hosted manged solution for Customers that can be run free of |
19 | | -expensive cloud hosting costs (starting with PostgreSQL). Before we can focus on this we needed to rewrite all our |
20 | | -SQLite-only features to work with OrmLite's other premier supported RDBMS's. |
| 17 | +However, we recognize that many of our customers need the features and scalability of industrial-strength RDBMS systems. |
| 18 | +Whether it's for leveraging existing database infrastructure, meeting enterprise requirements, or utilizing advanced |
| 19 | +database features like native table partitioning - we wanted to ensure Background Jobs could work seamlessly with |
| 20 | +your preferred database platform. |
21 | 21 |
|
22 | | -The new **DatabaseJobFeature** is a new implementation purpose built for PostgreSQL, SQL Server and MySQL backends that's |
23 | | -a drop-in replacement for SQLite's **BackgroundsJobFeature** which can be applied to an existing .NET 8+ project by |
24 | | -[mixing in](https://docs.servicestack.net/mix-tool) the **db-identity** or **db-jobs** gist files to your host project. |
| 22 | +## Introducing DatabaseJobsFeature |
25 | 23 |
|
26 | | -## Install |
| 24 | +The new **DatabaseJobsFeature** is a purpose-built implementation for PostgreSQL, SQL Server, and MySQL that's |
| 25 | +a drop-in replacement for SQLite's **BackgroundsJobFeature**. It maintains the same simple API, data models, |
| 26 | +and service contracts - making migration from SQLite straightforward while unlocking the power of enterprise RDBMS platforms. |
27 | 27 |
|
28 | | -For [ServiceStack ASP.NET Identity Auth](https://servicestack.net/start) Projects: |
| 28 | +Best of all, it can be added to an existing .NET 8+ project with a single command using our |
| 29 | +[mix tool](https://docs.servicestack.net/mix-tool): |
| 30 | + |
| 31 | +## Quick Start |
| 32 | + |
| 33 | +### For Identity Auth Projects |
| 34 | + |
| 35 | +If you're using [ServiceStack ASP.NET Identity Auth](https://servicestack.net/start) templates, simply run: |
29 | 36 |
|
30 | 37 | :::sh |
31 | 38 | x mix db-identity |
32 | 39 | ::: |
33 | 40 |
|
34 | | -Which replaces `Configure.BackgroundJobs.cs` and `Configure.RequestLogs.cs` with an equivalent |
35 | | -version that uses the new `DatabaseJobFeature` for sending Application Emails and `DbRequestLogger` |
36 | | -for API Request Logging. |
| 41 | +This replaces both `Configure.BackgroundJobs.cs` and `Configure.RequestLogs.cs` with RDBMS-compatible versions |
| 42 | +that use `DatabaseJobsFeature` for background jobs and `DbRequestLogger` for API request logging. |
| 43 | + |
| 44 | +### For Other .NET 8+ Apps |
37 | 45 |
|
38 | | -All other .NET 8+ ServiceStack Apps should instead use: |
| 46 | +For all other ServiceStack applications, use: |
39 | 47 |
|
40 | 48 | :::sh |
41 | 49 | x mix db-jobs |
42 | 50 | ::: |
43 | 51 |
|
44 | | -Which replaces `Configure.BackgroundJobs.cs` to use `DatabaseJobFeature`: |
| 52 | +This replaces `Configure.BackgroundJobs.cs` to use the new `DatabaseJobsFeature`: |
45 | 53 |
|
46 | 54 | ```csharp |
47 | 55 | public class ConfigureBackgroundJobs : IHostingStartup |
48 | 56 | { |
49 | 57 | public void Configure(IWebHostBuilder builder) => builder |
50 | 58 | .ConfigureServices(services => { |
51 | 59 | services.AddPlugin(new CommandsFeature()); |
52 | | - services.AddPlugin(new DatabaseJobFeature { |
53 | | - // NamedConnection = "<alternative db>" |
| 60 | + services.AddPlugin(new DatabaseJobsFeature { |
| 61 | + // Optional: Use a separate named connection |
| 62 | + // NamedConnection = "jobs" |
54 | 63 | }); |
55 | 64 | services.AddHostedService<JobsHostedService>(); |
56 | 65 | }).ConfigureAppHost(afterAppHostInit: appHost => { |
57 | 66 | var services = appHost.GetApplicationServices(); |
58 | 67 | var jobs = services.GetRequiredService<IBackgroundJobs>(); |
59 | | - // Example of registering a Recurring Job to run Every Hour |
60 | | - //jobs.RecurringCommand<MyCommand>(Schedule.Hourly); |
| 68 | + // Example: Register recurring jobs to run on a schedule |
| 69 | + // jobs.RecurringCommand<MyCommand>(Schedule.Hourly); |
61 | 70 | }); |
62 | 71 | } |
63 | 72 |
|
64 | | -public class JobsHostedService(ILogger<JobsHostedService> log, IBackgroundJobs jobs) : BackgroundService |
| 73 | +public class JobsHostedService(ILogger<JobsHostedService> log, IBackgroundJobs jobs) |
| 74 | + : BackgroundService |
65 | 75 | { |
66 | 76 | protected override async Task ExecuteAsync(CancellationToken stoppingToken) |
67 | 77 | { |
68 | 78 | await jobs.StartAsync(stoppingToken); |
69 | | - |
| 79 | + |
70 | 80 | using var timer = new PeriodicTimer(TimeSpan.FromSeconds(3)); |
71 | | - while (!stoppingToken.IsCancellationRequested && await timer.WaitForNextTickAsync(stoppingToken)) |
| 81 | + while (!stoppingToken.IsCancellationRequested && |
| 82 | + await timer.WaitForNextTickAsync(stoppingToken)) |
72 | 83 | { |
73 | 84 | await jobs.TickAsync(); |
74 | 85 | } |
75 | 86 | } |
76 | 87 | } |
77 | 88 | ``` |
78 | 89 |
|
79 | | -Fortunately we were able reuse the same `IBackgroundJobs` interface, Data Models, and API Service Contracts |
80 | | -which greatly simplifies any migration efforts from SQLite's **ServiceStack.Jobs** implementation. |
| 90 | +## Seamless Migration from SQLite |
| 91 | + |
| 92 | +We've maintained the same `IBackgroundJobs` interface, data models, and API service contracts, which means: |
| 93 | + |
| 94 | +- **Zero code changes** to your existing job enqueueing logic |
| 95 | +- **Same Admin UI** for monitoring and managing jobs |
| 96 | +- **Compatible APIs** - all your existing commands and job configurations work as-is |
| 97 | + |
| 98 | +The only change needed is swapping `BackgroundsJobFeature` for `DatabaseJobsFeature` in your configuration! |
81 | 99 |
|
82 | | -By implementing the same API Service Contracts (i.e. Request/Response DTOs) we're also able to reuse the same |
83 | | -[built-in](/auto-ui) Management UI to provide real-time monitoring, inspection and management of background jobs: |
| 100 | +Watch our video introduction to Background Jobs to see it in action: |
84 | 101 |
|
85 | 102 | :::youtube 2Cza_a_rrjA |
86 | 103 | Durable C# Background Jobs and Scheduled Tasks for .NET |
87 | 104 | ::: |
88 | 105 |
|
89 | | -## RDBMS Optimizations |
| 106 | +## Smart RDBMS Optimizations |
90 | 107 |
|
91 | | -A key benefit of using SQLite for Background Jobs was the ability to easily maintain completed and failed job history in |
92 | | -separate **monthly databases**. This approach prevented the main application database from growing unbounded by archiving |
93 | | -historical job data into isolated monthly SQLite database files (e.g., `jobs_2025-01.db`, `jobs_2025-02.db`). |
94 | | -These monthly databases could be easily backed up, archived to cold storage, or deleted after a retention period, |
95 | | -providing a simple yet effective data lifecycle management strategy. |
| 108 | +One of the key benefits of SQLite Background Jobs was the ability to maintain completed and failed job history in |
| 109 | +separate **monthly databases** (e.g., `jobs_2025-01.db`, `jobs_2025-02.db`). This prevented unbounded database growth |
| 110 | +and made it easy to archive or delete old job history. |
96 | 111 |
|
97 | | -For the new **DatabaseJobFeature** supporting PostgreSQL, SQL Server, and MySQL, we've replicated this monthly |
98 | | -partitioning strategy using **monthly partitioned SQL tables** for the `CompletedJob` and `FailedJob` archive tables. |
| 112 | +For `DatabaseJobsFeature`, we've replicated this monthly partitioning strategy using **monthly partitioned tables** |
| 113 | +for the `CompletedJob` and `FailedJob` archive tables - but the implementation varies by database platform to leverage |
| 114 | +each RDBMS's strengths. |
99 | 115 |
|
100 | 116 | ### PostgreSQL - Native Table Partitioning |
101 | 117 |
|
|
0 commit comments