Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions config/300-repositories.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,34 @@ spec:
- roles
- secret_ref
type: object
error_detection:
description: |-
ErrorDetection configures error detection for this repository. Error detection scans
container logs for error patterns and creates annotations (currently GitHub-only).
The global error-detection-from-container-logs setting must be enabled for this to work.
If not specified, uses only global error detection pattern.
properties:
max_number_of_lines:
description: |-
MaxNumberOfLines specifies how many lines to scan from the end of container logs
when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
Higher values may increase memory usage. Use -1 for unlimited.
If not specified, uses the global setting (default: 50).
type: integer
patterns:
description: |-
Patterns is an array of regular expressions used to detect errors in container logs.
Each pattern must use named groups to capture: filename, line, and error.
The column group is optional. Example pattern:
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)

Multiple patterns can be specified to match different error formats.
Repository-specific patterns are tried first, followed by global patterns.
If not specified or empty, only the global error-detection-simple-regexp patterns are used.
items:
type: string
type: array
type: object
github:
properties:
comment_strategy:
Expand Down
17 changes: 16 additions & 1 deletion config/302-pac-configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,22 @@ data:
# memory usage. Use -1 for unlimited lines.
error-detection-max-number-of-lines: "50"

# The default regexp used when we use the simple error detection
# The default regexp(s) used for simple error detection.
# Supports multiple formats for backward compatibility:
#
# 1. Single pattern (backward compatible):
# error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
#
# 2. YAML list format (multiple patterns, one per line):
# error-detection-simple-regexp: |-
# ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
# ^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
# ^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
#
# 3. JSON array format:
# error-detection-simple-regexp: '["^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)", "^ERROR:.*"]'
#
# Each pattern must include named groups: filename, line, and error (column is optional)
error-detection-simple-regexp: |-
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)

Expand Down
98 changes: 98 additions & 0 deletions docs/content/docs/guide/repositorycrd.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,104 @@ spec:
comment_strategy: "disable_all"
```

## Error Detection

Error detection scans container logs for error patterns and creates inline
annotations on Pull Requests. This feature is currently **only supported for
GitHub Apps**.

By default, error detection uses the global pattern configured in the
`pipelines-as-code` ConfigMap via the `error-detection-simple-regexp` setting.
However, you can customize error detection patterns on a per-repository basis
using the Repository CR.

### Configuring Error Detection Patterns

You can specify multiple regex patterns to detect different error formats in
your repository:

```yaml
apiVersion: "pipelinesascode.tekton.dev/v1alpha1"
kind: Repository
metadata:
name: my-repo
spec:
url: "https://github.com/owner/repo"
settings:
error_detection:
patterns:
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
- "^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)"
max_number_of_lines: 100
```

**Pattern Requirements:**

Each pattern must use [named groups](https://www.regular-expressions.info/named.html) to capture:

- `filename`: The file path where the error occurred
- `line`: The line number
- `error`: The error message
- `column`: (optional) The column number

**Configuration Options:**

- `patterns`: Array of regex patterns. Repository-specific patterns are tried
first, followed by global patterns. If not specified or empty, only the
global patterns are used. **Note:** Providing an empty array does not disable
error detection; it falls back to using only the global patterns defined in
the `pipelines-as-code` ConfigMap.
- `max_number_of_lines`: Number of log lines to scan (overrides global
setting). Default is 50. Use -1 for unlimited.

{{< hint info >}}
**Global Override:** The global `error-detection-from-container-logs` setting
must be enabled (default: `true`) for error detection to work. If disabled
globally, repository-level settings cannot override it.
{{< /hint >}}
Comment on lines +238 to +242
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have kept a global override so admins could disable error detection at an org level (for whatever (?) reason).


### Examples

**Multiple error formats:**

```yaml
spec:
settings:
error_detection:
patterns:
# Standard format (make, gcc, eslint, etc.)
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
# Python traceback format
- 'File "(?P<filename>[^"]+)", line (?P<line>[0-9]+).*\n.*(?P<error>.*)'
# Custom CI format
- "^\\[(?P<filename>[^\\]]+)\\]:(?P<line>[0-9]+) - (?P<error>.*)"
max_number_of_lines: 200
```

**Using only global patterns:**

If you want to use only the global patterns defined in the `pipelines-as-code`
ConfigMap, simply omit the `error_detection` field or provide an empty
`patterns` array:

```yaml
spec:
settings:
error_detection:
patterns: [] # Uses only global patterns
max_number_of_lines: 100 # Can still override line count
```

{{< hint info >}}
**Pattern Priority:** When repository-specific patterns are defined, they are
tried first for each log line. If no repository pattern matches, the global
patterns are then tried. This allows you to add repository-specific patterns
while still benefiting from the global patterns as a fallback.
{{< /hint >}}

For more information about error detection and log snippets, see the
[Status documentation]({{< relref "/docs/guide/statuses.md" >}}).

## Concurrency

`concurrency_limit` allows you to define the maximum number of PipelineRuns running at any time for a Repository.
Expand Down
14 changes: 12 additions & 2 deletions docs/content/docs/guide/statuses.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,10 @@ You can customize the regular expression used for detecting errors with the
`error-detection-simple-regexp` setting. The regular expression uses [named
groups](https://www.regular-expressions.info/named.html) to provide flexibility
in specifying the matching criteria. The necessary groups for matching are
filename, line, and error (the column group is not used). The default regular
expression is defined in the configuration map.
filename, line, and error (the column group is optional). The default regular
expression is defined in the configuration map. Multiple patterns are supported
using multi-line YAML or JSON array format, allowing you to detect errors from
different tools with various output formats.

By default, Pipelines-as-Code searches for errors in only the last 50 lines of
the container logs. However, you can increase this limit by setting the
Expand All @@ -94,6 +96,14 @@ system will search through all available lines for errors. Keep in mind that
increasing this maximum number of lines may increase the memory usage of the
watcher.

{{< hint info >}}
**Repository-level configuration:** You can also configure error detection
patterns on a per-repository basis using the Repository CR. This allows you to
define multiple patterns and customize settings for individual repositories.
See the [Repository CR documentation]({{< relref "/docs/guide/repositorycrd.md#error-detection" >}})
for more details.
{{< /hint >}}

![annotations](/images/github-annotation-error-failure-detection.png)

## Namespace Event stream
Expand Down
1 change: 1 addition & 0 deletions docs/content/docs/install/operator_installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ spec:
hub-url: 'https://artifacthub.io'
hub-catalog-type: 'artifacthub'
error-detection-max-number-of-lines: '50'
# Single pattern example. For multiple patterns, use multi-line format (see settings docs)
error-detection-simple-regexp: >-
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+):([
]*)?(?P<error>.*)
Expand Down
33 changes: 33 additions & 0 deletions docs/content/docs/install/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,39 @@ A few settings are available to configure this feature:
By default the error detection only support a simple output, the way GCC or
Make will output error, which is supported by most linters and command line tools.

**Multiple Patterns Support:** You can now specify multiple regex patterns to
match different error formats. The setting supports three formats:

1. **Single pattern** (backward compatible):

```yaml
error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
```

2. **Multi-line YAML** (recommended for multiple patterns):

```yaml
error-detection-simple-regexp: |-
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
```

3. **JSON array format**:

```yaml
error-detection-simple-regexp: '["^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)", "^ERROR:.*"]'
```

Each pattern will be tried in order until one matches. This allows you to detect
errors from multiple tools with different output formats.

**Pattern Requirements:** Each pattern must use regexp named groups to capture:
* `(?P<filename>...)` - The file path where the error occurred
* `(?P<line>...)` - The line number
* `(?P<error>...)` - The error message
* `(?P<column>...)` - Column number (optional)

An example of an error that is supported is :

```console
Expand Down
32 changes: 32 additions & 0 deletions pkg/apis/pipelinesascode/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,13 @@ type Settings struct {
// AIAnalysis contains AI/LLM analysis configuration for automated CI/CD pipeline analysis.
// +optional
AIAnalysis *AIAnalysisConfig `json:"ai,omitempty"`

// ErrorDetection configures error detection for this repository. Error detection scans
// container logs for error patterns and creates annotations (currently GitHub-only).
// The global error-detection-from-container-logs setting must be enabled for this to work.
// If not specified, uses only global error detection pattern.
// +optional
ErrorDetection *ErrorDetectionSettings `json:"error_detection,omitempty"`
}

type GitlabSettings struct {
Expand All @@ -184,6 +191,28 @@ type GithubSettings struct {
CommentStrategy string `json:"comment_strategy,omitempty"`
}

// ErrorDetectionSettings configures how errors are detected from container logs and
// exposed as annotations on Pull Requests. Currently only supported for GitHub Apps.
type ErrorDetectionSettings struct {
// Patterns is an array of regular expressions used to detect errors in container logs.
// Each pattern must use named groups to capture: filename, line, and error.
// The column group is optional. Example pattern:
// ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
//
// Multiple patterns can be specified to match different error formats.
// Repository-specific patterns are tried first, followed by global patterns.
// If not specified or empty, only the global error-detection-simple-regexp patterns are used.
// +optional
Patterns []string `json:"patterns,omitempty"`

// MaxNumberOfLines specifies how many lines to scan from the end of container logs
// when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
// Higher values may increase memory usage. Use -1 for unlimited.
// If not specified, uses the global setting (default: 50).
// +optional
MaxNumberOfLines *int `json:"max_number_of_lines,omitempty"`
}

func (s *Settings) Merge(newSettings *Settings) {
if newSettings.PipelineRunProvenance != "" && s.PipelineRunProvenance == "" {
s.PipelineRunProvenance = newSettings.PipelineRunProvenance
Expand All @@ -197,6 +226,9 @@ func (s *Settings) Merge(newSettings *Settings) {
if newSettings.AIAnalysis != nil && s.AIAnalysis == nil {
s.AIAnalysis = newSettings.AIAnalysis
}
if newSettings.ErrorDetection != nil && s.ErrorDetection == nil {
s.ErrorDetection = newSettings.ErrorDetection
}
}

type Policy struct {
Expand Down
Loading