Skip to content

Commit f922a79

Browse files
committed
feat(error-detection): add multiple pattern support
Add support for configuring multiple error detection regex patterns at both global (ConfigMap) and repository (CR) levels. This allows users to match different error formats from various linters and tools in a single pipeline. Global Configuration (ConfigMap): - Changed error-detection-simple-regexp to support arrays - Supports 3 formats: single pattern (backward compatible), multi-line YAML, and JSON array Repository CR: - Added ErrorDetectionSettings with patterns array and max_number_of_lines - Patterns are additive with global patterns - Per-repository max_number_of_lines override Jira: https://issues.redhat.com/browse/SRVKP-7237 Signed-off-by: Akshay Pant <[email protected]> Assisted-by: Cursor <[email protected]>
1 parent d0425d9 commit f922a79

File tree

10 files changed

+449
-56
lines changed

10 files changed

+449
-56
lines changed

config/300-repositories.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -446,6 +446,34 @@ spec:
446446
- roles
447447
- secret_ref
448448
type: object
449+
error_detection:
450+
description: |-
451+
ErrorDetection configures error detection for this repository. Error detection scans
452+
container logs for error patterns and creates annotations (currently GitHub-only).
453+
The global error-detection-from-container-logs setting must be enabled for this to work.
454+
If not specified, falls back to global error detection pattern.
455+
properties:
456+
max_number_of_lines:
457+
description: |-
458+
MaxNumberOfLines specifies how many lines to scan from the end of container logs
459+
when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
460+
Higher values may increase memory usage. Use -1 for unlimited.
461+
If not specified, uses the global setting (default: 50).
462+
type: integer
463+
patterns:
464+
description: |-
465+
Patterns is an array of regular expressions used to detect errors in container logs.
466+
Each pattern must use named groups to capture: filename, line, and error.
467+
The column group is optional. Example pattern:
468+
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
469+
470+
Multiple patterns can be specified to match different error formats.
471+
If empty array is provided, error detection is effectively disabled for this repository.
472+
If not specified, falls back to the global error-detection-simple-regexp setting.
473+
items:
474+
type: string
475+
type: array
476+
type: object
449477
github:
450478
properties:
451479
comment_strategy:

config/302-pac-configmap.yaml

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,22 @@ data:
101101
# memory usage. Use -1 for unlimited lines.
102102
error-detection-max-number-of-lines: "50"
103103

104-
# The default regexp used when we use the simple error detection
104+
# The default regexp(s) used for simple error detection.
105+
# Supports multiple formats for backward compatibility:
106+
#
107+
# 1. Single pattern (backward compatible):
108+
# error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
109+
#
110+
# 2. YAML list format (multiple patterns, one per line):
111+
# error-detection-simple-regexp: |-
112+
# ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
113+
# ^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
114+
# ^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
115+
#
116+
# 3. JSON array format:
117+
# error-detection-simple-regexp: '["^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)", "^ERROR:.*"]'
118+
#
119+
# Each pattern must include named groups: filename, line, and error (column is optional)
105120
error-detection-simple-regexp: |-
106121
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
107122

docs/content/docs/guide/repositorycrd.md

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,106 @@ spec:
185185
comment_strategy: "disable_all"
186186
```
187187

188+
## Error Detection
189+
190+
{{< tech_preview "Repository-level error detection patterns" >}}
191+
192+
Error detection scans container logs for error patterns and creates inline
193+
annotations on Pull Requests. This feature is currently **only supported for
194+
GitHub Apps**.
195+
196+
By default, error detection uses the global pattern configured in the
197+
`pipelines-as-code` ConfigMap via the `error-detection-simple-regexp` setting.
198+
However, you can customize error detection patterns on a per-repository basis
199+
using the Repository CR.
200+
201+
### Configuring Error Detection Patterns
202+
203+
You can specify multiple regex patterns to detect different error formats in
204+
your repository:
205+
206+
```yaml
207+
apiVersion: "pipelinesascode.tekton.dev/v1alpha1"
208+
kind: Repository
209+
metadata:
210+
name: my-repo
211+
spec:
212+
url: "https://github.com/owner/repo"
213+
settings:
214+
error_detection:
215+
patterns:
216+
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
217+
- "^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)"
218+
max_number_of_lines: 100
219+
```
220+
221+
**Pattern Requirements:**
222+
223+
Each pattern must use [named groups](https://www.regular-expressions.info/named.html) to capture:
224+
225+
- `filename`: The file path where the error occurred
226+
- `line`: The line number
227+
- `error`: The error message
228+
- `column`: (optional) The column number
229+
230+
**Configuration Options:**
231+
232+
- `patterns`: Array of regex patterns. Repository-specific patterns are tried
233+
first, followed by global patterns. If not specified or empty, only the
234+
global patterns are used. **Note:** Providing an empty array does not disable
235+
error detection; it falls back to using only the global patterns defined in
236+
the `pipelines-as-code` ConfigMap.
237+
- `max_number_of_lines`: Number of log lines to scan (overrides global
238+
setting). Default is 50. Use -1 for unlimited.
239+
240+
{{< hint info >}}
241+
**Global Override:** The global `error-detection-from-container-logs` setting
242+
must be enabled (default: `true`) for error detection to work. If disabled
243+
globally, repository-level settings cannot override it.
244+
{{< /hint >}}
245+
246+
### Examples
247+
248+
**Multiple error formats:**
249+
250+
```yaml
251+
spec:
252+
settings:
253+
error_detection:
254+
patterns:
255+
# Standard format (make, gcc, eslint, etc.)
256+
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
257+
# Python traceback format
258+
- 'File "(?P<filename>[^"]+)", line (?P<line>[0-9]+).*\n.*(?P<error>.*)'
259+
# Custom CI format
260+
- "^\\[(?P<filename>[^\\]]+)\\]:(?P<line>[0-9]+) - (?P<error>.*)"
261+
max_number_of_lines: 200
262+
```
263+
264+
**Using only global patterns:**
265+
266+
If you want to use only the global patterns defined in the `pipelines-as-code`
267+
ConfigMap, simply omit the `error_detection` field or provide an empty
268+
`patterns` array:
269+
270+
```yaml
271+
spec:
272+
settings:
273+
error_detection:
274+
patterns: [] # Uses only global patterns
275+
max_number_of_lines: 100 # Can still override line count
276+
```
277+
278+
{{< hint info >}}
279+
**Pattern Priority:** When repository-specific patterns are defined, they are
280+
tried first for each log line. If no repository pattern matches, the global
281+
patterns are then tried. This allows you to add repository-specific patterns
282+
while still benefiting from the global patterns as a fallback.
283+
{{< /hint >}}
284+
285+
For more information about error detection and log snippets, see the
286+
[Status documentation]({{< relref "/docs/guide/statuses.md" >}}).
287+
188288
## Concurrency
189289

190290
`concurrency_limit` allows you to define the maximum number of PipelineRuns running at any time for a Repository.

docs/content/docs/guide/statuses.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,14 @@ system will search through all available lines for errors. Keep in mind that
9494
increasing this maximum number of lines may increase the memory usage of the
9595
watcher.
9696

97+
{{< hint info >}}
98+
**Repository-level configuration:** You can also configure error detection
99+
patterns on a per-repository basis using the Repository CR. This allows you to
100+
define multiple patterns and customize settings for individual repositories.
101+
See the [Repository CR documentation]({{< relref "/docs/guide/repositorycrd.md#error-detection" >}})
102+
for more details.
103+
{{< /hint >}}
104+
97105
![annotations](/images/github-annotation-error-failure-detection.png)
98106

99107
## Namespace Event stream

docs/content/docs/install/operator_installation.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ spec:
4545
hub-url: 'https://artifacthub.io'
4646
hub-catalog-type: 'artifacthub'
4747
error-detection-max-number-of-lines: '50'
48+
# Single pattern example. For multiple patterns, use multi-line format (see settings docs)
4849
error-detection-simple-regexp: >-
4950
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+):([
5051
]*)?(?P<error>.*)

docs/content/docs/install/settings.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -291,6 +291,39 @@ A few settings are available to configure this feature:
291291
By default the error detection only support a simple output, the way GCC or
292292
Make will output error, which is supported by most linters and command line tools.
293293

294+
**Multiple Patterns Support:** You can now specify multiple regex patterns to
295+
match different error formats. The setting supports three formats:
296+
297+
1. **Single pattern** (backward compatible):
298+
299+
```yaml
300+
error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
301+
```
302+
303+
2. **Multi-line YAML** (recommended for multiple patterns):
304+
305+
```yaml
306+
error-detection-simple-regexp: |-
307+
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
308+
^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
309+
^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
310+
```
311+
312+
3. **JSON array format**:
313+
314+
```yaml
315+
error-detection-simple-regexp: '["^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)", "^ERROR:.*"]'
316+
```
317+
318+
Each pattern will be tried in order until one matches. This allows you to detect
319+
errors from multiple tools with different output formats.
320+
321+
**Pattern Requirements:** Each pattern must use regexp named groups to capture:
322+
* `(?P<filename>...)` - The file path where the error occurred
323+
* `(?P<line>...)` - The line number
324+
* `(?P<error>...)` - The error message
325+
* `(?P<column>...)` - Column number (optional)
326+
294327
An example of an error that is supported is :
295328

296329
```console

pkg/apis/pipelinesascode/v1alpha1/types.go

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,13 @@ type Settings struct {
164164
// AIAnalysis contains AI/LLM analysis configuration for automated CI/CD pipeline analysis.
165165
// +optional
166166
AIAnalysis *AIAnalysisConfig `json:"ai,omitempty"`
167+
168+
// ErrorDetection configures error detection for this repository. Error detection scans
169+
// container logs for error patterns and creates annotations (currently GitHub-only).
170+
// The global error-detection-from-container-logs setting must be enabled for this to work.
171+
// If not specified, uses only global error detection pattern.
172+
// +optional
173+
ErrorDetection *ErrorDetectionSettings `json:"error_detection,omitempty"`
167174
}
168175

169176
type GitlabSettings struct {
@@ -184,6 +191,28 @@ type GithubSettings struct {
184191
CommentStrategy string `json:"comment_strategy,omitempty"`
185192
}
186193

194+
// ErrorDetectionSettings configures how errors are detected from container logs and
195+
// exposed as annotations on Pull Requests. Currently only supported for GitHub Apps.
196+
type ErrorDetectionSettings struct {
197+
// Patterns is an array of regular expressions used to detect errors in container logs.
198+
// Each pattern must use named groups to capture: filename, line, and error.
199+
// The column group is optional. Example pattern:
200+
// ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
201+
//
202+
// Multiple patterns can be specified to match different error formats.
203+
// Repository-specific patterns are tried first, followed by global patterns.
204+
// If not specified or empty, only the global error-detection-simple-regexp patterns are used.
205+
// +optional
206+
Patterns []string `json:"patterns,omitempty"`
207+
208+
// MaxNumberOfLines specifies how many lines to scan from the end of container logs
209+
// when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
210+
// Higher values may increase memory usage. Use -1 for unlimited.
211+
// If not specified, uses the global setting (default: 50).
212+
// +optional
213+
MaxNumberOfLines *int `json:"max_number_of_lines,omitempty"`
214+
}
215+
187216
func (s *Settings) Merge(newSettings *Settings) {
188217
if newSettings.PipelineRunProvenance != "" && s.PipelineRunProvenance == "" {
189218
s.PipelineRunProvenance = newSettings.PipelineRunProvenance
@@ -197,6 +226,9 @@ func (s *Settings) Merge(newSettings *Settings) {
197226
if newSettings.AIAnalysis != nil && s.AIAnalysis == nil {
198227
s.AIAnalysis = newSettings.AIAnalysis
199228
}
229+
if newSettings.ErrorDetection != nil && s.ErrorDetection == nil {
230+
s.ErrorDetection = newSettings.ErrorDetection
231+
}
200232
}
201233

202234
type Policy struct {

pkg/params/settings/config.go

Lines changed: 75 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package settings
22

33
import (
4+
"encoding/json"
45
"fmt"
56
"net/url"
67
"regexp"
@@ -64,11 +65,11 @@ type Settings struct {
6465
SecretGHAppRepoScoped bool `default:"true" json:"secret-github-app-token-scoped"`
6566
SecretGhAppTokenScopedExtraRepos string `json:"secret-github-app-scope-extra-repos"`
6667

67-
ErrorLogSnippet bool `default:"true" json:"error-log-snippet"`
68-
ErrorLogSnippetNumberOfLines int `default:"3" json:"error-log-snippet-number-of-lines"`
69-
ErrorDetection bool `default:"true" json:"error-detection-from-container-logs"`
70-
ErrorDetectionNumberOfLines int `default:"50" json:"error-detection-max-number-of-lines"`
71-
ErrorDetectionSimpleRegexp string `default:"^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)" json:"error-detection-simple-regexp"`
68+
ErrorLogSnippet bool `default:"true" json:"error-log-snippet"`
69+
ErrorLogSnippetNumberOfLines int `default:"3" json:"error-log-snippet-number-of-lines"`
70+
ErrorDetection bool `default:"true" json:"error-detection-from-container-logs"`
71+
ErrorDetectionNumberOfLines int `default:"50" json:"error-detection-max-number-of-lines"`
72+
ErrorDetectionSimpleRegexp []string `default:"^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"` // Note: no json tag, handled specially
7273

7374
EnableCancelInProgressOnPullRequests bool `json:"enable-cancel-in-progress-on-pull-requests"`
7475
EnableCancelInProgressOnPush bool `json:"enable-cancel-in-progress-on-push"`
@@ -105,11 +106,10 @@ func DefaultSettings() Settings {
105106

106107
func DefaultValidators() map[string]func(string) error {
107108
return map[string]func(string) error{
108-
"ErrorDetectionSimpleRegexp": isValidRegex,
109-
"TektonDashboardURL": isValidURL,
110-
"CustomConsoleURL": isValidURL,
111-
"CustomConsolePRTaskLog": startWithHTTPorHTTPS,
112-
"CustomConsolePRDetail": startWithHTTPorHTTPS,
109+
"TektonDashboardURL": isValidURL,
110+
"CustomConsoleURL": isValidURL,
111+
"CustomConsolePRTaskLog": startWithHTTPorHTTPS,
112+
"CustomConsolePRDetail": startWithHTTPorHTTPS,
113113
}
114114
}
115115

@@ -121,6 +121,13 @@ func SyncConfig(logger *zap.SugaredLogger, setting *Settings, config map[string]
121121
return fmt.Errorf("failed to validate and assign values: %w", err)
122122
}
123123

124+
// Parse error detection patterns (supports backward compatibility)
125+
// Handling error detection specially as we want to keep backward compatibility while
126+
// allowing []string as a valid input value.
127+
if err := parseErrorDetectionPatterns(setting, config); err != nil {
128+
return fmt.Errorf("failed to parse error detection patterns: %w", err)
129+
}
130+
124131
value, _ := setting.HubCatalogs.Load("default")
125132
catalogDefault, ok := value.(HubCatalog)
126133
if ok {
@@ -159,3 +166,61 @@ func startWithHTTPorHTTPS(url string) error {
159166
}
160167
return nil
161168
}
169+
170+
// parseErrorDetectionPatterns parses the error-detection-simple-regexp from ConfigMap
171+
// and populates ErrorDetectionSimpleRegexp as []string. It supports:
172+
// 1. Single string pattern (backward compatible)
173+
// 2. JSON array format: ["pattern1", "pattern2"]
174+
// 3. Newline-separated patterns (YAML multi-line).
175+
func parseErrorDetectionPatterns(setting *Settings, config map[string]string) error {
176+
regexpValue := config["error-detection-simple-regexp"]
177+
178+
// If not in config, use default
179+
if regexpValue == "" {
180+
regexpValue = "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
181+
}
182+
183+
regexpValue = strings.TrimSpace(regexpValue)
184+
185+
// Try to parse as JSON array first
186+
if strings.HasPrefix(regexpValue, "[") && strings.HasSuffix(regexpValue, "]") {
187+
var patterns []string
188+
if err := json.Unmarshal([]byte(regexpValue), &patterns); err == nil {
189+
// Validate each pattern
190+
for _, pattern := range patterns {
191+
if err := isValidRegex(pattern); err != nil {
192+
return fmt.Errorf("invalid regex in array: %w", err)
193+
}
194+
}
195+
setting.ErrorDetectionSimpleRegexp = patterns
196+
return nil
197+
}
198+
}
199+
200+
// Check if it contains newlines (multi-line YAML format)
201+
if strings.Contains(regexpValue, "\n") {
202+
lines := strings.Split(regexpValue, "\n")
203+
var patterns []string
204+
for _, line := range lines {
205+
line = strings.TrimSpace(line)
206+
if line == "" {
207+
continue
208+
}
209+
if err := isValidRegex(line); err != nil {
210+
return fmt.Errorf("invalid regex in multi-line format: %w", err)
211+
}
212+
patterns = append(patterns, line)
213+
}
214+
if len(patterns) > 0 {
215+
setting.ErrorDetectionSimpleRegexp = patterns
216+
return nil
217+
}
218+
}
219+
220+
// Single pattern (backward compatible)
221+
if err := isValidRegex(regexpValue); err != nil {
222+
return err
223+
}
224+
setting.ErrorDetectionSimpleRegexp = []string{regexpValue}
225+
return nil
226+
}

0 commit comments

Comments
 (0)