-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
fix: Improve log level detection with confidence scoring system #3070
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: canary
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR refactors the log type detection system to use a confidence-based scoring mechanism, significantly reducing false positives when classifying Docker log messages. The previous implementation incorrectly flagged informational messages containing words like "failed" (e.g., "Failed=0") as errors.
Key changes:
- Introduced a three-tier pattern matching system (structured, contextual, keyword) with weighted scores
- Added negative context detection to prevent false positives from phrases like "did not fail"
- Enhanced pattern coverage including emojis, HTTP status codes, and stack traces
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| apps/dokploy/components/dashboard/docker/logs/utils.ts | Replaced simple regex checks with a scoring-based classification system using structured, contextual, and keyword pattern arrays |
| apps/dokploy/test/logs/log-type.test.ts | Added comprehensive test suite with 25 test cases covering all log type detection scenarios including false positive prevention |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const isNegativeContext = (message: string, keyword: string): boolean => { | ||
| const lowerMessage = message.toLowerCase(); | ||
| const lowerKeyword = keyword.toLowerCase(); | ||
|
|
||
| const negativePatterns = [ | ||
| `(?:not|no|without|won't|didn't|doesn't|never)\\s+(?:\\w+\\s+){0,3}${lowerKeyword}`, |
Copilot
AI
Dec 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The regex pattern allows up to 3 words between negation and keyword ({0,3}), but this threshold appears arbitrary. Consider documenting why 3 words was chosen, or extracting it as a named constant (e.g., MAX_WORD_DISTANCE = 3) to make the intent clearer and the value easier to adjust if needed.
| const isNegativeContext = (message: string, keyword: string): boolean => { | |
| const lowerMessage = message.toLowerCase(); | |
| const lowerKeyword = keyword.toLowerCase(); | |
| const negativePatterns = [ | |
| `(?:not|no|without|won't|didn't|doesn't|never)\\s+(?:\\w+\\s+){0,3}${lowerKeyword}`, | |
| // Allow a small number of intermediate words between a negation term and the keyword | |
| // (e.g. "not at all really error"); 3 was chosen as a balance between recall and noise. | |
| const MAX_NEGATION_KEYWORD_WORD_DISTANCE = 3; | |
| const isNegativeContext = (message: string, keyword: string): boolean => { | |
| const lowerMessage = message.toLowerCase(); | |
| const lowerKeyword = keyword.toLowerCase(); | |
| const negativePatterns = [ | |
| `(?:not|no|without|won't|didn't|doesn't|never)\\s+(?:\\w+\\s+){0,${MAX_NEGATION_KEYWORD_WORD_DISTANCE}}${lowerKeyword}`, |
| score: 100, | ||
| type: "error", | ||
| }, | ||
| { pattern: /❌|🔴|🚨/i, score: 100, type: "error" }, |
Copilot
AI
Dec 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The case-insensitive flag i is unnecessary for emoji patterns since emojis don't have case variants. Consider removing the flag for clarity and minor performance improvement.
| { pattern: /^(?:warn(?:ing)?):?\s/i, score: 100, type: "warning" }, | ||
| { pattern: /level=(?:warn(?:ing)?)/i, score: 100, type: "warning" }, | ||
| { pattern: /severity=(?:warn(?:ing)?)/i, score: 100, type: "warning" }, | ||
| { pattern: /⚠️|⚠|⛔/i, score: 100, type: "warning" }, |
Copilot
AI
Dec 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The case-insensitive flag i is unnecessary for emoji patterns since emojis don't have case variants. Consider removing the flag for clarity and minor performance improvement.
| { pattern: /level=(?:debug|trace)/i, score: 100, type: "debug" }, | ||
|
|
||
| { pattern: /\[(?:success|ok|done)\]/i, score: 100, type: "success" }, | ||
| { pattern: /✓|√|✅|\[ok\]/i, score: 100, type: "success" }, |
Copilot
AI
Dec 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The case-insensitive flag i applies to the entire pattern, but is only meaningful for the \[ok\] portion. Consider splitting this into two patterns or documenting that the flag is intentionally applied to cover variations like [OK] and [Ok].
| { pattern: /✓|√|✅|\[ok\]/i, score: 100, type: "success" }, | |
| { pattern: /✓|√|✅/, score: 100, type: "success" }, | |
| { pattern: /\[ok\]/i, score: 100, type: "success" }, |
| for (const [type, score] of Object.entries(scores)) { | ||
| if (score > maxScore) { | ||
| maxScore = score; | ||
| detectedType = type as LogType; |
Copilot
AI
Dec 20, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When multiple log types have the same score, the detected type depends on object enumeration order, which may be unpredictable. Consider adding explicit tie-breaking logic (e.g., prioritizing 'error' over 'warning') to ensure consistent behavior when scores are equal.
| for (const [type, score] of Object.entries(scores)) { | |
| if (score > maxScore) { | |
| maxScore = score; | |
| detectedType = type as LogType; | |
| const LOG_TYPE_PRIORITY: Record<LogType, number> = { | |
| error: 0, | |
| warning: 1, | |
| success: 2, | |
| info: 3, | |
| debug: 4, | |
| }; | |
| for (const [type, score] of Object.entries(scores)) { | |
| const currentType = type as LogType; | |
| if ( | |
| score > maxScore || | |
| (score === maxScore && | |
| LOG_TYPE_PRIORITY[currentType] < LOG_TYPE_PRIORITY[detectedType]) | |
| ) { | |
| maxScore = score; | |
| detectedType = currentType; |
|
can you rsolve the conflicts? @Akinator31 |
ee7ce57 to
669516d
Compare
|
Rebased on canary and resolved conflicts @Siumauricio Changes made:
|
What is this PR about?
This PR improves the log type detection system in the Docker logs viewer to significantly reduce
false positives. The previous implementation used broad regex patterns that incorrectly flagged
informational messages as errors (e.g., "Failed=0" or "0 builds failed").
Technical Approach
The new implementation uses a confidence-based scoring system with three levels of pattern
matching:
Level 1 - Structured Patterns (Score: 100pts)
[ERROR],ERROR:,level=error,severity=warningLevel 2 - Contextual Patterns (Score: 50-70pts)
at Object.<anonymous> (/path/file.js:123:45)Level 3 - Keyword Patterns (Score: 20-30pts)
(conditional)
How Scoring Works
Example:
"⚠️ Token exchange failed with status: 400"scores:"Failed=0 Scanned=1"scores:Key Improvements
This is my proposed approach, but I'm very open to alternative implementations. Feel free to share any suggestions or concerns—I'm happy to adjust based on your feedback.
Checklist
Before submitting this PR, please make sure that:
canarybranch.Issues related (if applicable)
fixes #1996
Screenshots (if applicable)
Before: Logs like
time="2025-11-20T08:19:13Z" level=info msg="Session done" Failed=0wereincorrectly marked as errors
After: These logs are now correctly displayed as info, while real errors like
⚠️ Token exchange failed with status: 400 Bad Requestare properly detected