Skip to content

#3171 Solved the problem of systematic data archiving #3173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Patrykb0802 wants to merge 7 commits into
base: release/2.8.0
Choose a base branch
from
Open

#3171 Solved the problem of systematic data archiving #3173

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
6b40ab5
#3171 Solved the problem of systematic data archiving
Patrykb0802 Jul 18, 2025
ed6caea
#3171 Solved the problem of systematic data archiving
Patrykb0802 Jul 30, 2025
dc37796
#3171 Solved the problem of systematic data archiving
Patrykb0802 Jul 31, 2025
28dbf80
#3171 Solved the problem of systematic data archiving
Patrykb0802 Aug 4, 2025
520524d
#3171 Solved the problem of systematic data archiving
Patrykb0802 Aug 25, 2025
9d35f3d
#3171 Solved the problem of systematic data archiving
Patrykb0802 Sep 1, 2025
997c788
#3171 Solved the problem of systematic data archiving
Patrykb0802 Sep 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions WebContent/WEB-INF/applicationContext.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
<bean id="watchListDAO" class="org.scada_lts.dao.watchlist.WatchListDAO"/>
<bean id="dataSourceDAO" class="org.scada_lts.dao.DataSourceDAO"/>
<bean id="userDAO" class="org.scada_lts.dao.UserDAO"/>
<bean id="systemSettingsDAO" class="org.scada_lts.dao.SystemSettingsDAO"/>

<bean id="watchListGetShareUsers" class="org.scada_lts.permissions.service.WatchListGetShareUsers">
<constructor-arg ref="watchListDAO"/>
Expand Down Expand Up @@ -358,6 +359,38 @@
<constructor-arg ref="updatePendingEventsJobDetail"/>
</bean>

<!-- Quartz Archiver -->

<bean id="archiveTrigger" class="org.quartz.CronTrigger">
<constructor-arg value="Quartz - trigger-Archiving"/>
<constructor-arg value="DEFAULT"/>
</bean>

<bean id="archiveJobDetail" class="org.quartz.JobDetail">
<constructor-arg value="Quartz - job-Archiving"/>
<constructor-arg value="DEFAULT"/>
<constructor-arg value="org.scada_lts.archive.ArchiveJob" type="java.lang.Class"/>
</bean>

<bean id="archiveScheduler" class="org.scada_lts.quartz.CronTriggerScheduler" destroy-method="stop">
<constructor-arg ref="scadaScheduler"/>
<constructor-arg ref="archiveTrigger"/>
<constructor-arg ref="archiveJobDetail"/>
</bean>

<bean id="archiveQueryProviderFactory"
class="org.scada_lts.archive.ArchiveQueryProviderFactory">
<constructor-arg ref="databaseAccess"/>
</bean>

<bean id="archiveQueryProvider"
factory-bean="archiveQueryProviderFactory"
factory-method="get"/>

<bean id="archiveService" class="org.scada_lts.archive.ArchiveService">
<constructor-arg ref="archiveQueryProvider"/>
</bean>
Comment on lines +381 to +392
Copy link

@coderabbitai coderabbitai bot Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Spring wiring likely injects a provider that ArchiveService doesn’t use.

archiveService is constructed with archiveQueryProvider, but ArchiveService.runArchive() currently creates a provider via ArchiveQueryProviderFactory.forTarget(...) instead. Either:

  • inject a factory (or datasource manager) into the service and use that, or
  • remove the constructor arg + bean wiring to avoid confusion.
🤖 Prompt for AI Agents 
In @WebContent/WEB-INF/applicationContext.xml around lines 381 - 392, The bean
wiring injects archiveQueryProvider into ArchiveService but
ArchiveService.runArchive() ignores it and calls
ArchiveQueryProviderFactory.forTarget(...); update the service to use the
injected provider (remove the internal call to
ArchiveQueryProviderFactory.forTarget in ArchiveService.runArchive and use the
constructor-injected ArchiveQueryProvider instance) or alternatively change the
wiring to inject the factory instead (replace the constructor-arg on the
archiveService bean with a reference to archiveQueryProviderFactory and update
ArchiveService constructor/signature to accept and use
ArchiveQueryProviderFactory.forTarget), and remove the unused
archiveQueryProvider bean if you choose the factory injection route to avoid
conflicting/unused beans.


<bean id="opcUaTrustListManager" class="org.scada_lts.ds.polling.protocol.opcua.client.impl.OpcUaTrustListManager"
destroy-method="close" lazy-init="true"/>
</beans>
67 changes: 67 additions & 0 deletions WebContent/WEB-INF/dox/de/dataArchiveSettings.htm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<!--
Scada-LTS
Data Archiving Help
@author PatrykB
-->
<h1>Overview</h1>
<p>
The Data Archiving module allows you to automatically copy and/or delete old data from selected database tables
to an external archival database. This helps reduce the size of the primary database while keeping historical data accessible.
</p>

<p>
Archiving is configured using flexible rules which define how old the data must be to be archived or deleted, which tables to process,
and what action to take (copy or delete). You can also define the destination archive database and how frequently the archiving process runs.
</p>

<p>
Option <b>Default values limit for purge</b> is related to property that can be set inside Point properties in Logging properties.
It indicates how many values will be left in database after purging data when purge strategy with <b>Values limit</b> is selected for a given datapoint.
This default value is used during datapoint creation.
</p>

<h2>Settings</h2>
<p>The configuration consists of the following options:</p>
<ul>
<li><b>Enable data archiving:</b> Activates or deactivates the archiving mechanism.</li>
<li><b>Archive database URL:</b> JDBC URL of the target archive database (e.g., <code>jdbc:mysql://localhost:3308/scadalts_archive</code>).</li>
<li><b>Archive database username:</b> Username for the archive database.</li>
<li><b>Archive database password:</b> Password for the archive database user.</li>
<li><b>Batch size:</b> Number of records processed in each step during archiving. Helps manage memory and performance.</li>
<li><b>Archive frequency (cron):</b> Cron expression that determines how often the archiving task is run (e.g., <code>0 0 1 * * ?</code> for daily at 1:00 AM).</li>
<li><b>Archiving Rules:</b> List of individual tasks that define what data should be archived or deleted.</li>
</ul>

<h2>Archiving Rules</h2>
<p>Each archiving rule specifies:</p>
<ul>
<li><b>Older than:</b> How old the data must be to qualify (e.g., 30 days).</li>
<li><b>Action:</b> Whether to <b>Archive</b> (copy to archive DB) or <b>Delete</b> (from main DB if already archived).</li>
<li><b>Table:</b> Which data table to apply the rule to (e.g., <code>pointValues</code>, <code>events</code>).</li>
</ul>

<h2>Usage Example</h2>
<p>Archive historical point values older than 30 days to a secondary MySQL database.</p>
<ol>
<li>Enable the "Enable data archiving" checkbox.</li>
<li>Set the Archive database URL to: <code>jdbc:mysql://localhost:3308/scadalts_archive</code></li>
<li>Set username and password for the archive database.</li>
<li>Set Batch size to <code>1000</code> (default).</li>
<li>Set Archive frequency to <code>0 0 1 * * ?</code> (every day at 1:00 AM).</li>
<li>In Archiving Rules, add a new rule:</li>
<ul>
<li><b>Older than:</b> 30 day(s)</li>
<li><b>Action:</b> Archive</li>
<li><b>Table:</b> pointValues</li>
</ul>
<li>Save configuration.</li>
<li>At 1:00 AM each day, Scada-LTS will move records from <code>pointValues</code> older than 30 days to the archive database.</li>
</ol>

<h2>Additional Notes</h2>
<ul>
<li>Ensure that the archive database has the same schema structure as the source tables.</li>
<li>The archiving engine will automatically copy table structure if the archive table doesn't exist.</li>
<li>Deleted data cannot be recovered unless previously archived.</li>
<li>It is recommended to regularly back up both your main and archive databases.</li>
</ul>
67 changes: 67 additions & 0 deletions WebContent/WEB-INF/dox/en/dataArchiveSettings.htm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<!--
Scada-LTS
Data Archiving Help
@author PatrykB
-->
<h1>Overview</h1>
<p>
The Data Archiving module allows you to automatically copy and/or delete old data from selected database tables
to an external archival database. This helps reduce the size of the primary database while keeping historical data accessible.
</p>

<p>
Archiving is configured using flexible rules which define how old the data must be to be archived or deleted, which tables to process,
and what action to take (copy or delete). You can also define the destination archive database and how frequently the archiving process runs.
</p>

<p>
Option <b>Default values limit for purge</b> is related to property that can be set inside Point properties in Logging properties.
It indicates how many values will be left in database after purging data when purge strategy with <b>Values limit</b> is selected for a given datapoint.
This default value is used during datapoint creation.
</p>

<h2>Settings</h2>
<p>The configuration consists of the following options:</p>
<ul>
<li><b>Enable data archiving:</b> Activates or deactivates the archiving mechanism.</li>
<li><b>Archive database URL:</b> JDBC URL of the target archive database (e.g., <code>jdbc:mysql://localhost:3308/scadalts_archive</code>).</li>
<li><b>Archive database username:</b> Username for the archive database.</li>
<li><b>Archive database password:</b> Password for the archive database user.</li>
<li><b>Batch size:</b> Number of records processed in each step during archiving. Helps manage memory and performance.</li>
<li><b>Archive frequency (cron):</b> Cron expression that determines how often the archiving task is run (e.g., <code>0 0 1 * * ?</code> for daily at 1:00 AM).</li>
<li><b>Archiving Rules:</b> List of individual tasks that define what data should be archived or deleted.</li>
</ul>

<h2>Archiving Rules</h2>
<p>Each archiving rule specifies:</p>
<ul>
<li><b>Older than:</b> How old the data must be to qualify (e.g., 30 days).</li>
<li><b>Action:</b> Whether to <b>Archive</b> (copy to archive DB) or <b>Delete</b> (from main DB if already archived).</li>
<li><b>Table:</b> Which data table to apply the rule to (e.g., <code>pointValues</code>, <code>events</code>).</li>
</ul>

<h2>Usage Example</h2>
<p>Archive historical point values older than 30 days to a secondary MySQL database.</p>
<ol>
<li>Enable the "Enable data archiving" checkbox.</li>
<li>Set the Archive database URL to: <code>jdbc:mysql://localhost:3308/scadalts_archive</code></li>
<li>Set username and password for the archive database.</li>
<li>Set Batch size to <code>1000</code> (default).</li>
<li>Set Archive frequency to <code>0 0 1 * * ?</code> (every day at 1:00 AM).</li>
<li>In Archiving Rules, add a new rule:</li>
<ul>
<li><b>Older than:</b> 30 day(s)</li>
<li><b>Action:</b> Archive</li>
<li><b>Table:</b> pointValues</li>
</ul>
<li>Save configuration.</li>
<li>At 1:00 AM each day, Scada-LTS will move records from <code>pointValues</code> older than 30 days to the archive database.</li>
</ol>

<h2>Additional Notes</h2>
<ul>
<li>Ensure that the archive database has the same schema structure as the source tables.</li>
<li>The archiving engine will automatically copy table structure if the archive table doesn't exist.</li>
<li>Deleted data cannot be recovered unless previously archived.</li>
<li>It is recommended to regularly back up both your main and archive databases.</li>
</ul>
67 changes: 67 additions & 0 deletions WebContent/WEB-INF/dox/es/dataArchiveSettings.htm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<!--
Scada-LTS
Data Archiving Help
@author PatrykB
-->
<h1>Overview</h1>
<p>
The Data Archiving module allows you to automatically copy and/or delete old data from selected database tables
to an external archival database. This helps reduce the size of the primary database while keeping historical data accessible.
</p>

<p>
Archiving is configured using flexible rules which define how old the data must be to be archived or deleted, which tables to process,
and what action to take (copy or delete). You can also define the destination archive database and how frequently the archiving process runs.
</p>

<p>
Option <b>Default values limit for purge</b> is related to property that can be set inside Point properties in Logging properties.
It indicates how many values will be left in database after purging data when purge strategy with <b>Values limit</b> is selected for a given datapoint.
This default value is used during datapoint creation.
</p>

<h2>Settings</h2>
<p>The configuration consists of the following options:</p>
<ul>
<li><b>Enable data archiving:</b> Activates or deactivates the archiving mechanism.</li>
<li><b>Archive database URL:</b> JDBC URL of the target archive database (e.g., <code>jdbc:mysql://localhost:3308/scadalts_archive</code>).</li>
<li><b>Archive database username:</b> Username for the archive database.</li>
<li><b>Archive database password:</b> Password for the archive database user.</li>
<li><b>Batch size:</b> Number of records processed in each step during archiving. Helps manage memory and performance.</li>
<li><b>Archive frequency (cron):</b> Cron expression that determines how often the archiving task is run (e.g., <code>0 0 1 * * ?</code> for daily at 1:00 AM).</li>
<li><b>Archiving Rules:</b> List of individual tasks that define what data should be archived or deleted.</li>
</ul>

<h2>Archiving Rules</h2>
<p>Each archiving rule specifies:</p>
<ul>
<li><b>Older than:</b> How old the data must be to qualify (e.g., 30 days).</li>
<li><b>Action:</b> Whether to <b>Archive</b> (copy to archive DB) or <b>Delete</b> (from main DB if already archived).</li>
<li><b>Table:</b> Which data table to apply the rule to (e.g., <code>pointValues</code>, <code>events</code>).</li>
</ul>

<h2>Usage Example</h2>
<p>Archive historical point values older than 30 days to a secondary MySQL database.</p>
<ol>
<li>Enable the "Enable data archiving" checkbox.</li>
<li>Set the Archive database URL to: <code>jdbc:mysql://localhost:3308/scadalts_archive</code></li>
<li>Set username and password for the archive database.</li>
<li>Set Batch size to <code>1000</code> (default).</li>
<li>Set Archive frequency to <code>0 0 1 * * ?</code> (every day at 1:00 AM).</li>
<li>In Archiving Rules, add a new rule:</li>
<ul>
<li><b>Older than:</b> 30 day(s)</li>
<li><b>Action:</b> Archive</li>
<li><b>Table:</b> pointValues</li>
</ul>
<li>Save configuration.</li>
<li>At 1:00 AM each day, Scada-LTS will move records from <code>pointValues</code> older than 30 days to the archive database.</li>
</ol>

<h2>Additional Notes</h2>
<ul>
<li>Ensure that the archive database has the same schema structure as the source tables.</li>
<li>The archiving engine will automatically copy table structure if the archive table doesn't exist.</li>
<li>Deleted data cannot be recovered unless previously archived.</li>
<li>It is recommended to regularly back up both your main and archive databases.</li>
</ul>
67 changes: 67 additions & 0 deletions WebContent/WEB-INF/dox/fr/dataArchiveSettings.htm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<!--
Scada-LTS
Data Archiving Help
@author PatrykB
-->
<h1>Overview</h1>
<p>
The Data Archiving module allows you to automatically copy and/or delete old data from selected database tables
to an external archival database. This helps reduce the size of the primary database while keeping historical data accessible.
</p>

<p>
Archiving is configured using flexible rules which define how old the data must be to be archived or deleted, which tables to process,
and what action to take (copy or delete). You can also define the destination archive database and how frequently the archiving process runs.
</p>

<p>
Option <b>Default values limit for purge</b> is related to property that can be set inside Point properties in Logging properties.
It indicates how many values will be left in database after purging data when purge strategy with <b>Values limit</b> is selected for a given datapoint.
This default value is used during datapoint creation.
</p>

<h2>Settings</h2>
<p>The configuration consists of the following options:</p>
<ul>
<li><b>Enable data archiving:</b> Activates or deactivates the archiving mechanism.</li>
<li><b>Archive database URL:</b> JDBC URL of the target archive database (e.g., <code>jdbc:mysql://localhost:3308/scadalts_archive</code>).</li>
<li><b>Archive database username:</b> Username for the archive database.</li>
<li><b>Archive database password:</b> Password for the archive database user.</li>
<li><b>Batch size:</b> Number of records processed in each step during archiving. Helps manage memory and performance.</li>
<li><b>Archive frequency (cron):</b> Cron expression that determines how often the archiving task is run (e.g., <code>0 0 1 * * ?</code> for daily at 1:00 AM).</li>
<li><b>Archiving Rules:</b> List of individual tasks that define what data should be archived or deleted.</li>
</ul>

<h2>Archiving Rules</h2>
<p>Each archiving rule specifies:</p>
<ul>
<li><b>Older than:</b> How old the data must be to qualify (e.g., 30 days).</li>
<li><b>Action:</b> Whether to <b>Archive</b> (copy to archive DB) or <b>Delete</b> (from main DB if already archived).</li>
<li><b>Table:</b> Which data table to apply the rule to (e.g., <code>pointValues</code>, <code>events</code>).</li>
</ul>

<h2>Usage Example</h2>
<p>Archive historical point values older than 30 days to a secondary MySQL database.</p>
<ol>
<li>Enable the "Enable data archiving" checkbox.</li>
<li>Set the Archive database URL to: <code>jdbc:mysql://localhost:3308/scadalts_archive</code></li>
<li>Set username and password for the archive database.</li>
<li>Set Batch size to <code>1000</code> (default).</li>
<li>Set Archive frequency to <code>0 0 1 * * ?</code> (every day at 1:00 AM).</li>
<li>In Archiving Rules, add a new rule:</li>
<ul>
<li><b>Older than:</b> 30 day(s)</li>
<li><b>Action:</b> Archive</li>
<li><b>Table:</b> pointValues</li>
</ul>
<li>Save configuration.</li>
<li>At 1:00 AM each day, Scada-LTS will move records from <code>pointValues</code> older than 30 days to the archive database.</li>
</ol>

<h2>Additional Notes</h2>
<ul>
<li>Ensure that the archive database has the same schema structure as the source tables.</li>
<li>The archiving engine will automatically copy table structure if the archive table doesn't exist.</li>
<li>Deleted data cannot be recovered unless previously archived.</li>
<li>It is recommended to regularly back up both your main and archive databases.</li>
</ul>
4 changes: 3 additions & 1 deletion WebContent/WEB-INF/dox/manifest.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@
</item>

<item id="emailSettings"/>

<item id="galilDS">
<relation id="galilPP"/>
</item>
Expand Down Expand Up @@ -439,4 +439,6 @@
<item id="internalDS">
<relation id="internalPP"/>
</item>

<item id ="dataArchiveSettings"/>
</doxManifest>
Loading
Loading