-
Notifications
You must be signed in to change notification settings - Fork 9
Logs Definition
The logs definition input file defines all the logs that the data generator should generate for the specified resources. Each log type to be generated is defined in this file along with the following field which is applicable for all the logs:
- globalPayloadFrequencySeconds - Number of seconds after which the logs payloads are posted. You can override this at each log level.
The definition for each log has the following fields:
| Field | Is Mandatory? | Description |
|---|---|---|
| name | No | Name of the log. The same value is sent for the name field in LogRecord. The names of all logs should be distinct. If not provided, the tool will automatically generate a name and use it. |
| reportingResourceCounts | No | Map of resource types (which should be pre-defined in resource model definition) and the number of resources of that type reporting this log. For Example, Pod 10 implies that this log definition is posted for 10 pods randomly chosen out of total pods (as defined in Resource Model Definition). If the total payload is 10 then, for each of the 10 payloads, the log is sent for random 10 pods and not all the pods. Valid value must be provided for reportingResources or filteredReportingResources field. |
| filteredReportingResources | No | A map of resource names and a set of corresponding attribute value filters resulting in a list of resources that should report this log. See this for example. If a resource name is mentioned in both reportingResourcesCounts and filteredReportingResources field, an error will be thrown. Multiple filters, if specified for a resource, are treated as a combined filter, meaning only the resources matching all the filter conditions will be selected. Any invalid filters (not having exactly one '=') will simply be ignored and in case a resource type is specified with no valid filters, or, effectively an empty set of filters, all resources of that type will be selected. |
| payloadCount | Yes | The number of times this log should be posted. You can configure different payloadCount for different log definitions. |
| severityOrderFunction | Yes | The expression that should be evaluated to obtain the severity type value of the log. See Severity Order Functions. |
| payloadFrequencySeconds | No | Number of seconds after which the log payloads are to be posted. This is an optional override; you can use globalPayloadFrequencySeconds instead. A random value is selected if both are unavailable for a log. See Payload Frequency. |
| attributes | No | List of key-value pairs in the format <ATTRIBUTE_NAME, VALUE_EXPRESSION> where each pair defines an attribute to be sent for this log and an expression that is evaluated to generate the value for that attribute. See Attribute Value Expressions. |
| copyResourceAttributes | No | A set of attribute names defined in the reporting resource(s) which should be copied to the log attributes. If the attribute is not available in the reporting resource, it will be sent with an empty string value. |
You can define the optional field globalPayloadFrequencySeconds in the log definition file. It is the payload frequency to be used for all the logs that do not have payloadFrequencySeconds defined. The following table shows how the payload frequency is selected in different cases.
| payloadFrequencySeconds defined at the log level? | globalPayloadFrequencySeconds is defined? | Value selected |
|---|---|---|
| Yes | No | payloadFrequencySeconds defined at the log level |
| No | Yes | globalPayloadFrequencySeconds |
| No | No | One of (15, 30, 45, 60, 75, 90) |
| Yes | Yes | payloadFrequencySeconds defined at the log level |
After payloadFrequencySeconds is determined for every log, it is revised to an appropriate value to rationalize the data generation as per the following formula:
totalPacketCount = Number_of_Reporting_resources * copyCount
IF (totalPacketCount >=2000 && totalPacketCount <= 25000) THEN payloadFrequencySeconds = 30
IF (totalPacketCount >=25000 && totalPacketCount <= 80000) THEN payloadFrequencySeconds = 60
IF (totalPacketCount >=80000) THEN payloadFrequencySeconds = 90
Implementation details are available here.
This is done so that threads do not crash due to overload.
The severityOrderFunction field specified for each log evaluates the severity type of that log. The value function expression specified for each log packet must be one of the defined custom expression methods that should return a severity (type String) value when evaluated using the Jakarta Expression Language (JEL) processor.
All the resources that report the log to get the same severity value in the same payload. The log generator thread for each log is invoked every X (payloadFrequencySeconds) seconds and that log is reported in each invocation, for all of the resources specified in the definition. Before this happens, the severityOrderfunction is evaluated for the log and the same value is provided to all the resources. Next time the thread is invoked, the value is updated based on the expression provided in the severityOrderfunction. This is done to optimise functional testing scenarios and the tool's performance.
Use the following methods to evaluate the severity type for the log:
| Method | Description |
|---|---|
| severityDistributionCount(List severity, List frequency) | Returns one of the severity values from the severity list (in the specified order) for the specified frequency number of times, before moving on to the next index. If the Payload count is less than the sum of frequency values, the remainder of severity values are not created. For example: severityDistributionCount(["Normal", "Severe", "Warning"], [25, 5, 10]) In this case, the first 25 logs generated are of Normal severity, followed by 5 Severe logs and finally 10 Warning logs. The cycle repeats itself in the same order. |
| severityDistributionPercentage(List severity, List percentageValue) | Returns one of the severity values from the severity list (in the specified order). The actual count of each severity is determined by the total payload count as explained in the example. In the case of an irreducible fraction, floor value will be considered. If a non-zero percentage returns 0 as the floor value, 1 packet will still be created. For example: severityDistributionPercentage(["Normal", "Severe", "Warning"], [20, 70, 10]) In this case, if the payload count equals 10 then, 2 logs are of type Normal, 7 are of type Severe and 1 is of type Warning. The sum of frequency percentages in percentageValue should always be equal to 100. |
It is possible to implement and provide your own expression methods for generating severity levels. See User Defined Expressions.
As per the OpenTelemetry specification, adding event.name and event.domain to the log attributes is the way to specify a log as an event and should be processed as such by the consuming platform. The generator already allows specifying attributes and so this is easily accomplished by adding these attributes to a log definition. See log definition files available in example definitions for a Kubernetes event that has been defined as an example.
Read more about the OpenTelemetry specification for events here.