You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: 13/umbraco-ums/developers/introduction/performance.md
+19-16Lines changed: 19 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,19 +10,19 @@ Within the Umbraco uMS we have implemented several performance optimizations and
10
10
11
11
## Separate processes for storing, parsing, and reporting
12
12
13
-
As documented in the [dataflow process](../../../../the-umarketingsuite-broad-overview/dataflow-pipeline/) there are different steps for [collecting](../../../../the-umarketingsuite-broad-overview/dataflow-pipeline/data-collection/), [storing](../../../../the-umarketingsuite-broad-overview/dataflow-pipeline/data-storage/), [parsing](../../../../the-umarketingsuite-broad-overview/dataflow-pipeline/data-parsing/), and [reporting](../../../../the-umarketingsuite-broad-overview/dataflow-pipeline/reporting/) the data. This is primarily done for performance reasons.
13
+
As documented in the [dataflow process](dataflow-pipeline/) there are different steps for [collecting](dataflow-pipeline/data-collection.md), [storing](dataflow-pipeline/data-storage.md), [parsing](dataflow-pipeline/data-parsing.md), and [reporting](dataflow-pipeline/reporting.md) the data. This is primarily done for performance reasons.
14
14
15
15
The collection is done in memory of the web server (or webservers if you have multiple web servers in a load-balanced
16
16
17
-
Storing causes the data to flow from the memory to the database. The memory is free again and can be used for other data. The data is stored at that moment in the raw data tables.
17
+
Storing causes the data to flow from the memory to the database. The memory is free again and can be used for other data. The data is stored in the raw data tables at that moment.
18
18
19
-
In our production websites we see that the average data size per record in the table `uMarketingSuiteAnalyticsRawPageView` is 0,9 kb. This means that every visitor stores 0,9 kb of data.
19
+
We see the average data size per record in the table `uMarketingSuiteAnalyticsRawPageView` is 0,9 kb in our production websites,. This means that every visitor stores 0,9 kb of data.
20
20
21
-
For the `uMarketingSuiteRawClientSideData` it depends a bit on the implementation of the clientside events. If you track a lot of [custom events](../../../../the-umarketingsuite-broad-overview/dataflow-pipeline/data-collection/) this table will probably be bigger. On average we see that this is around 0,4 kb per visitor that can execute JavaScript (all bots are excluded because of this).
21
+
For the `uMarketingSuiteRawClientSideData` it depends a bit on the implementation of the clientside events. If you track many [custom events](dataflow-pipeline/data-collection.md) this table will probably be bigger. On average we see that this is around 0,4 kb per visitor that can execute JavaScript (all bots are excluded because of this).
22
22
23
23
This data can be found for your database with the following SQL Statement:
24
24
25
-
```
25
+
```sql
26
26
/***
27
27
Copied and applied for the raw data tables from https://support.managed.com/kb/a227/how-to-find-large-tables-in-sql-database.aspx
28
28
* Find the number of rows and the size of tables
@@ -37,15 +37,16 @@ DROP TABLE #temp
37
37
38
38
### Data parsing
39
39
40
-
In the data parsing the data is fetched from the raw data tables and stored in normalized tables. At that moment the raw data could be deleted and only the normalized data tables are needed. The data parsing is the heaviest step of the total process so this is where we put most of our performance TLC in.
40
+
The data is fetched from the raw data tables and stored in normalized tables in the data parsing. The raw data could be deleted and only the normalized data tables kept. The data parsing is the heaviest step of the total process so this is where we put most of our performance TLC in.
41
41
42
-
The data parsing process runs in a background job on the webserver. Within [the configuration file](../../../../installing-umarketingsuite/configuration-options-1-x/) you can specify how many records are fetched to parse and how often the process needs to run. It's also possible to specify which server(s) need to be responsible for the parsing process.
42
+
The data parsing process runs in a background job on the webserver. Within [the configuration file](../../../../installing-umarketingsuite/configuration-options-1-x/), you can specify how many records are fetched to parse and how often the process needs to run. It's also possible to specify which server(s) need to be responsible for the parsing process.
43
43
44
-
On average we see that the amount of stored data is only 0,1 kb per visit. This is only 10% of the original amount in the raw data tables.
44
+
We see on average that the amount of stored data is only 0,1 kb per visit. This is only 10% of the original amount in the raw data tables.
45
45
46
-
The SQL Script for determining that is:
46
+
The `SQL` Script for determining that is:
47
47
48
-
```
48
+
{% code overflow="wrap" %}
49
+
```sql
49
50
/***
50
51
* Copied and applied for the Umbraco uMS data tables from https://support.managed.com/kb/a227/how-to-find-large-tables-in-sql-database.aspx
51
52
* Find the number of rows and the size of tables
@@ -59,12 +60,13 @@ SELECT a.table_name,a.row_count,COUNT(*) AS col_count,a.data_sizeFROM #temp aINN
59
60
60
61
DROPTABLE#temp
61
62
```
63
+
{% endcode %}
62
64
63
65
## Show me more numbers
64
66
65
-
With the two SQL scripts above you can find out how much data is used by the Umbraco uMS. If you want to see how long the processing step takes you can run this script to see the processing speed for the raw pageviews:
67
+
With the two SQL scripts above you can find out how much data the Umbraco uMS uses. If you want to see how long the processing step takes you can run this script to see the processing speed for the raw pageviews:
@@ -94,5 +96,6 @@ We try to keep the average parsing **speed under 100 ms**. Please let us know if
94
96
95
97
You can also optimize your server infrastructure to tweak the performance. There are a few options that you could apply:
96
98
97
-
* You could set up more webservers in [a load-balanced setup](https://our.umbraco.com/Documentation/Getting-Started/Setup/Server-Setup/Load-Balancing/). Each of these web servers will collect data from the visitor, but you can specify in the configuration which web server (s) is responsible for the parsing of the data. You could also set up one specific server only for parsing the data. In that case, the other web servers will have almost no impact on their performance. To set this up you need to set the parameter '`IsProcessingServer`' to '`false`' in [your configuration file](../../../../installing-umarketingsuite/settings-section/the-configuration-file/) for all servers that do not need to process the data and set it to '`true`' on the server(s) that is responsible for parsing. If there is no server with this setting set to '`true`' the raw data of the Umbraco uMS will take place, but the data will never be processed.
98
-
* By default the uMarketingSuite stores its data in the same database as Umbraco. It uses the default connection string of Umbraco (named '`umbracoDbDSN`'). It is possible to specify a separate database for all uMarketingSuite data. This could be another database on the same server but also another database server. To do this you need to specify a new connection string in your application and give that connection string a name. In [the configuration file](../../../../installing-umarketingsuite/configuration-options-1-x/), you can now specify this name in the field '`DatabaseConnectionStringName`'.
99
+
* You could set up more web servers in [a load-balanced setup](https://docs.umbraco.com/umbraco-cms/13.latest-lts/fundamentals/setup/server-setup/load-balancing). Each web server will collect data from the visitor, but you can specify which web server is responsible for parsing the data in the configuration. 
100
+
* You can also set up one specific server only to parse the data. In that case, the other web servers will have almost no impact on their performance. To set this up you need to set the parameter '`IsProcessingServer`' to '`false`' in [your configuration file](../../../../installing-umarketingsuite/settings-section/the-configuration-file/) for all servers that do not need to process the data and set it to '`true`' on the server(s) that is responsible for parsing. If there is no server with this setting set to '`true`' the raw data of the Umbraco uMS will take place, but the data will never be processed.
101
+
* By default, the Umbraco uMS stores its data in the same database as Umbraco. It uses the default connection string of Umbraco (named '`umbracoDbDSN`'). It is possible to specify a separate database for all Umbraco uMS data. This could be another database on the same server or also another database server. To do this you need to specify a new connection string in your application and give that connection string a name. In [the configuration file](../../getting-started/for-developers/configuration-options-2-x.md), you can now specify this name in the field '`DatabaseConnectionStringName`'.
0 commit comments