Skip to content

Commit feefb68

Browse files
committed
feat(error-detection): add multiple pattern support
Add support for configuring multiple error detection regex patterns at both global (ConfigMap) and repository (CR) levels. This allows users to match different error formats from various linters and tools in a single pipeline. Global Configuration (ConfigMap): - Changed error-detection-simple-regexp to support arrays - Supports 3 formats: single pattern (backward compatible), multi-line YAML, and JSON array Repository CR: - Added ErrorDetectionSettings with patterns array and max_number_of_lines - Patterns are additive with global patterns - Per-repository max_number_of_lines override Jira: https://issues.redhat.com/browse/SRVKP-7237 Signed-off-by: Akshay Pant <[email protected]>
1 parent a8212d3 commit feefb68

File tree

10 files changed

+424
-56
lines changed

10 files changed

+424
-56
lines changed

config/300-repositories.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -446,6 +446,34 @@ spec:
446446
- roles
447447
- secret_ref
448448
type: object
449+
error_detection:
450+
description: |-
451+
ErrorDetection configures error detection for this repository. Error detection scans
452+
container logs for error patterns and creates annotations (currently GitHub-only).
453+
The global error-detection-from-container-logs setting must be enabled for this to work.
454+
If not specified, falls back to global error detection pattern.
455+
properties:
456+
max_number_of_lines:
457+
description: |-
458+
MaxNumberOfLines specifies how many lines to scan from the end of container logs
459+
when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
460+
Higher values may increase memory usage. Use -1 for unlimited.
461+
If not specified, uses the global setting (default: 50).
462+
type: integer
463+
patterns:
464+
description: |-
465+
Patterns is an array of regular expressions used to detect errors in container logs.
466+
Each pattern must use named groups to capture: filename, line, and error.
467+
The column group is optional. Example pattern:
468+
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
469+
470+
Multiple patterns can be specified to match different error formats.
471+
If empty array is provided, error detection is effectively disabled for this repository.
472+
If not specified, falls back to the global error-detection-simple-regexp setting.
473+
items:
474+
type: string
475+
type: array
476+
type: object
449477
github:
450478
properties:
451479
comment_strategy:

config/302-pac-configmap.yaml

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,22 @@ data:
101101
# memory usage. Use -1 for unlimited lines.
102102
error-detection-max-number-of-lines: "50"
103103

104-
# The default regexp used when we use the simple error detection
104+
# The default regexp(s) used for simple error detection.
105+
# Supports multiple formats for backward compatibility:
106+
#
107+
# 1. Single pattern (backward compatible):
108+
# error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
109+
#
110+
# 2. YAML list format (multiple patterns, one per line):
111+
# error-detection-simple-regexp: |-
112+
# ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
113+
# ^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
114+
# ^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
115+
#
116+
# 3. JSON array format:
117+
# error-detection-simple-regexp: '["^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)", "^ERROR:.*"]'
118+
#
119+
# Each pattern must include named groups: filename, line, and error (column is optional)
105120
error-detection-simple-regexp: |-
106121
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
107122

docs/content/docs/guide/repositorycrd.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,82 @@ spec:
185185
comment_strategy: "disable_all"
186186
```
187187

188+
## Error Detection
189+
190+
{{< tech_preview "Repository-level error detection patterns" >}}
191+
192+
Error detection scans container logs for error patterns and creates inline
193+
annotations on Pull Requests. This feature is currently **only supported for
194+
GitHub Apps**.
195+
196+
By default, error detection uses the global pattern configured in the
197+
`pipelines-as-code` ConfigMap via the `error-detection-simple-regexp` setting.
198+
However, you can customize error detection patterns on a per-repository basis
199+
using the Repository CR.
200+
201+
### Configuring Error Detection Patterns
202+
203+
You can specify multiple regex patterns to detect different error formats in
204+
your repository:
205+
206+
```yaml
207+
apiVersion: "pipelinesascode.tekton.dev/v1alpha1"
208+
kind: Repository
209+
metadata:
210+
name: my-repo
211+
spec:
212+
url: "https://github.com/owner/repo"
213+
settings:
214+
error_detection:
215+
patterns:
216+
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
217+
- "^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)"
218+
max_number_of_lines: 100
219+
```
220+
221+
**Pattern Requirements:**
222+
223+
Each pattern must use [named groups](https://www.regular-expressions.info/named.html) to capture:
224+
225+
- `filename`: The file path where the error occurred
226+
- `line`: The line number
227+
- `error`: The error message
228+
- `column`: (optional) The column number
229+
230+
**Configuration Options:**
231+
232+
- `patterns`: Array of regex patterns. If not specified, falls back to the
233+
global pattern.
234+
- `max_number_of_lines`: Number of log lines to scan (overrides global
235+
setting). Default is 50. Use -1 for unlimited.
236+
237+
{{< hint info >}}
238+
**Global Override:** The global `error-detection-from-container-logs` setting
239+
must be enabled (default: `true`) for error detection to work. If disabled
240+
globally, repository-level settings cannot override it.
241+
{{< /hint >}}
242+
243+
### Examples
244+
245+
**Multiple error formats:**
246+
247+
```yaml
248+
spec:
249+
settings:
250+
error_detection:
251+
patterns:
252+
# Standard format (make, gcc, eslint, etc.)
253+
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
254+
# Python traceback format
255+
- 'File "(?P<filename>[^"]+)", line (?P<line>[0-9]+).*\n.*(?P<error>.*)'
256+
# Custom CI format
257+
- "^\\[(?P<filename>[^\\]]+)\\]:(?P<line>[0-9]+) - (?P<error>.*)"
258+
max_number_of_lines: 200
259+
```
260+
261+
For more information about error detection and log snippets, see the
262+
[Status documentation]({{< relref "/docs/guide/statuses.md" >}}).
263+
188264
## Concurrency
189265

190266
`concurrency_limit` allows you to define the maximum number of PipelineRuns running at any time for a Repository.

docs/content/docs/guide/statuses.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,14 @@ system will search through all available lines for errors. Keep in mind that
9494
increasing this maximum number of lines may increase the memory usage of the
9595
watcher.
9696

97+
{{< hint info >}}
98+
**Repository-level configuration:** You can also configure error detection
99+
patterns on a per-repository basis using the Repository CR. This allows you to
100+
define multiple patterns and customize settings for individual repositories.
101+
See the [Repository CR documentation]({{< relref "/docs/guide/repositorycrd.md#error-detection" >}})
102+
for more details.
103+
{{< /hint >}}
104+
97105
![annotations](/images/github-annotation-error-failure-detection.png)
98106

99107
## Namespace Event stream

docs/content/docs/install/operator_installation.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ spec:
4545
hub-url: 'https://artifacthub.io'
4646
hub-catalog-type: 'artifacthub'
4747
error-detection-max-number-of-lines: '50'
48+
# Single pattern example. For multiple patterns, use multi-line format (see settings docs)
4849
error-detection-simple-regexp: >-
4950
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+):([
5051
]*)?(?P<error>.*)

docs/content/docs/install/settings.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -291,6 +291,39 @@ A few settings are available to configure this feature:
291291
By default the error detection only support a simple output, the way GCC or
292292
Make will output error, which is supported by most linters and command line tools.
293293

294+
**Multiple Patterns Support:** You can now specify multiple regex patterns to
295+
match different error formats. The setting supports three formats:
296+
297+
1. **Single pattern** (backward compatible):
298+
299+
```yaml
300+
error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
301+
```
302+
303+
2. **Multi-line YAML** (recommended for multiple patterns):
304+
305+
```yaml
306+
error-detection-simple-regexp: |-
307+
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
308+
^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
309+
^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
310+
```
311+
312+
3. **JSON array format**:
313+
314+
```yaml
315+
error-detection-simple-regexp: '["^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)", "^ERROR:.*"]'
316+
```
317+
318+
Each pattern will be tried in order until one matches. This allows you to detect
319+
errors from multiple tools with different output formats.
320+
321+
**Pattern Requirements:** Each pattern must use regexp named groups to capture:
322+
* `(?P<filename>...)` - The file path where the error occurred
323+
* `(?P<line>...)` - The line number
324+
* `(?P<error>...)` - The error message
325+
* `(?P<column>...)` - Column number (optional)
326+
294327
An example of an error that is supported is :
295328

296329
```console

pkg/apis/pipelinesascode/v1alpha1/types.go

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,13 @@ type Settings struct {
164164
// AIAnalysis contains AI/LLM analysis configuration for automated CI/CD pipeline analysis.
165165
// +optional
166166
AIAnalysis *AIAnalysisConfig `json:"ai,omitempty"`
167+
168+
// ErrorDetection configures error detection for this repository. Error detection scans
169+
// container logs for error patterns and creates annotations (currently GitHub-only).
170+
// The global error-detection-from-container-logs setting must be enabled for this to work.
171+
// If not specified, uses only global error detection pattern.
172+
// +optional
173+
ErrorDetection *ErrorDetectionSettings `json:"error_detection,omitempty"`
167174
}
168175

169176
type GitlabSettings struct {
@@ -184,6 +191,27 @@ type GithubSettings struct {
184191
CommentStrategy string `json:"comment_strategy,omitempty"`
185192
}
186193

194+
// ErrorDetectionSettings configures how errors are detected from container logs and
195+
// exposed as annotations on Pull Requests. Currently only supported for GitHub Apps.
196+
type ErrorDetectionSettings struct {
197+
// Patterns is an array of regular expressions used to detect errors in container logs.
198+
// Each pattern must use named groups to capture: filename, line, and error.
199+
// The column group is optional. Example pattern:
200+
// ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
201+
//
202+
// Multiple patterns can be specified to match different error formats.
203+
// If not specified, uses only the global error-detection-simple-regexp setting.
204+
// +optional
205+
Patterns []string `json:"patterns,omitempty"`
206+
207+
// MaxNumberOfLines specifies how many lines to scan from the end of container logs
208+
// when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
209+
// Higher values may increase memory usage. Use -1 for unlimited.
210+
// If not specified, uses the global setting (default: 50).
211+
// +optional
212+
MaxNumberOfLines *int `json:"max_number_of_lines,omitempty"`
213+
}
214+
187215
func (s *Settings) Merge(newSettings *Settings) {
188216
if newSettings.PipelineRunProvenance != "" && s.PipelineRunProvenance == "" {
189217
s.PipelineRunProvenance = newSettings.PipelineRunProvenance
@@ -197,6 +225,9 @@ func (s *Settings) Merge(newSettings *Settings) {
197225
if newSettings.AIAnalysis != nil && s.AIAnalysis == nil {
198226
s.AIAnalysis = newSettings.AIAnalysis
199227
}
228+
if newSettings.ErrorDetection != nil && s.ErrorDetection == nil {
229+
s.ErrorDetection = newSettings.ErrorDetection
230+
}
200231
}
201232

202233
type Policy struct {

pkg/params/settings/config.go

Lines changed: 75 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package settings
22

33
import (
4+
"encoding/json"
45
"fmt"
56
"net/url"
67
"regexp"
@@ -64,11 +65,11 @@ type Settings struct {
6465
SecretGHAppRepoScoped bool `default:"true" json:"secret-github-app-token-scoped"`
6566
SecretGhAppTokenScopedExtraRepos string `json:"secret-github-app-scope-extra-repos"`
6667

67-
ErrorLogSnippet bool `default:"true" json:"error-log-snippet"`
68-
ErrorLogSnippetNumberOfLines int `default:"3" json:"error-log-snippet-number-of-lines"`
69-
ErrorDetection bool `default:"true" json:"error-detection-from-container-logs"`
70-
ErrorDetectionNumberOfLines int `default:"50" json:"error-detection-max-number-of-lines"`
71-
ErrorDetectionSimpleRegexp string `default:"^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)" json:"error-detection-simple-regexp"`
68+
ErrorLogSnippet bool `default:"true" json:"error-log-snippet"`
69+
ErrorLogSnippetNumberOfLines int `default:"3" json:"error-log-snippet-number-of-lines"`
70+
ErrorDetection bool `default:"true" json:"error-detection-from-container-logs"`
71+
ErrorDetectionNumberOfLines int `default:"50" json:"error-detection-max-number-of-lines"`
72+
ErrorDetectionSimpleRegexp []string `default:"^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"` // Note: no json tag, handled specially
7273

7374
EnableCancelInProgressOnPullRequests bool `json:"enable-cancel-in-progress-on-pull-requests"`
7475
EnableCancelInProgressOnPush bool `json:"enable-cancel-in-progress-on-push"`
@@ -105,11 +106,10 @@ func DefaultSettings() Settings {
105106

106107
func DefaultValidators() map[string]func(string) error {
107108
return map[string]func(string) error{
108-
"ErrorDetectionSimpleRegexp": isValidRegex,
109-
"TektonDashboardURL": isValidURL,
110-
"CustomConsoleURL": isValidURL,
111-
"CustomConsolePRTaskLog": startWithHTTPorHTTPS,
112-
"CustomConsolePRDetail": startWithHTTPorHTTPS,
109+
"TektonDashboardURL": isValidURL,
110+
"CustomConsoleURL": isValidURL,
111+
"CustomConsolePRTaskLog": startWithHTTPorHTTPS,
112+
"CustomConsolePRDetail": startWithHTTPorHTTPS,
113113
}
114114
}
115115

@@ -121,6 +121,13 @@ func SyncConfig(logger *zap.SugaredLogger, setting *Settings, config map[string]
121121
return fmt.Errorf("failed to validate and assign values: %w", err)
122122
}
123123

124+
// Parse error detection patterns (supports backward compatibility)
125+
// Handling error detection specially as we want to keep backward compatibility while
126+
// allowing []string as a valid input value.
127+
if err := parseErrorDetectionPatterns(setting, config); err != nil {
128+
return fmt.Errorf("failed to parse error detection patterns: %w", err)
129+
}
130+
124131
value, _ := setting.HubCatalogs.Load("default")
125132
catalogDefault, ok := value.(HubCatalog)
126133
if ok {
@@ -159,3 +166,61 @@ func startWithHTTPorHTTPS(url string) error {
159166
}
160167
return nil
161168
}
169+
170+
// parseErrorDetectionPatterns parses the error-detection-simple-regexp from ConfigMap
171+
// and populates ErrorDetectionSimpleRegexp as []string. It supports:
172+
// 1. Single string pattern (backward compatible)
173+
// 2. JSON array format: ["pattern1", "pattern2"]
174+
// 3. Newline-separated patterns (YAML multi-line).
175+
func parseErrorDetectionPatterns(setting *Settings, config map[string]string) error {
176+
regexpValue := config["error-detection-simple-regexp"]
177+
178+
// If not in config, use default
179+
if regexpValue == "" {
180+
regexpValue = "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
181+
}
182+
183+
regexpValue = strings.TrimSpace(regexpValue)
184+
185+
// Try to parse as JSON array first
186+
if strings.HasPrefix(regexpValue, "[") && strings.HasSuffix(regexpValue, "]") {
187+
var patterns []string
188+
if err := json.Unmarshal([]byte(regexpValue), &patterns); err == nil {
189+
// Validate each pattern
190+
for _, pattern := range patterns {
191+
if err := isValidRegex(pattern); err != nil {
192+
return fmt.Errorf("invalid regex in array: %w", err)
193+
}
194+
}
195+
setting.ErrorDetectionSimpleRegexp = patterns
196+
return nil
197+
}
198+
}
199+
200+
// Check if it contains newlines (multi-line YAML format)
201+
if strings.Contains(regexpValue, "\n") {
202+
lines := strings.Split(regexpValue, "\n")
203+
var patterns []string
204+
for _, line := range lines {
205+
line = strings.TrimSpace(line)
206+
if line == "" {
207+
continue
208+
}
209+
if err := isValidRegex(line); err != nil {
210+
return fmt.Errorf("invalid regex in multi-line format: %w", err)
211+
}
212+
patterns = append(patterns, line)
213+
}
214+
if len(patterns) > 0 {
215+
setting.ErrorDetectionSimpleRegexp = patterns
216+
return nil
217+
}
218+
}
219+
220+
// Single pattern (backward compatible)
221+
if err := isValidRegex(regexpValue); err != nil {
222+
return err
223+
}
224+
setting.ErrorDetectionSimpleRegexp = []string{regexpValue}
225+
return nil
226+
}

0 commit comments

Comments
 (0)