testiso.go: Add check for badness #3895

c4rt0 · 2024-10-01T19:41:30Z

This PR aims to address the first step of a suggestion in the #3788 issue.

(use checkConsole to verify console.txt and journal.txt for badness)

openshift-ci · 2024-10-01T19:41:34Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

mantle/platform/qemu.go

dustymabe · 2024-10-04T03:15:00Z

ok I think the strategy here should probably be to just call kola.CheckConsole directly from within awaitCompletion in testiso.go AFTER the errchan has been populated. i.e. not as a separate parallel running gofunc, but just at the end after the error channel has been filled.

There is precedent for calling CheckConsole() like this without a t *register.Test in kola check-console which I admit I didn't even know existed.

coreos-assembler/mantle/cmd/kola/console.go

Lines 77 to 81 in 7550a58

    
           _, badlines := kola.CheckConsole(console, nil) 
        
           for _, badline := range badlines { 
        
           	fmt.Printf("%v: %v\n", sourceName, badline) 
        
           	errorcount++ 
        
           }

So we should be able to do something like:

diff --git a/mantle/cmd/kola/testiso.go b/mantle/cmd/kola/testiso.go
index a925d3d25..e2cd761c1 100644
--- a/mantle/cmd/kola/testiso.go
+++ b/mantle/cmd/kola/testiso.go
@@ -729,7 +729,18 @@ func awaitCompletion(ctx context.Context, inst *platform.QemuInstance, outdir st
                }
        }()
        err := <-errchan
-       return time.Since(start), err
+       // The test is done so let's record the amount of elapsed time
+       elapsed = time.Since(start)
+       if err == nil; {
+               // No error so far; let's check the console/journal
+               for each of inst.builder.consoleFile and filepath.Join(outdir, "journal.txt") we need to {
+
+                       warnOnly, badlines := kola.CheckConsole(readfile, nil)
+                       for range badlines
+                               do something and set err to !nil if there was badness
+               }
+       }
+       return elapsed, err
 }

 func printResult(test string, duration time.Duration, err error) bool {

c4rt0 · 2024-10-15T12:58:01Z

I ran the code through the gofmt, yet it still fails the linter check. Can I get any pointers on this, please?
The error description does not seem to be precise enough for me to decypher it:

  Error: ineffectual assignment to err (ineffassign)

🤔

jbtrystram · 2024-10-15T14:04:52Z

I ran the code through the gofmt, yet it still fails the linter check. Can I get any pointers on this, please? The error description does not seem to be precise enough for me to decypher it:
  Error: ineffectual assignment to err (ineffassign)
🤔

the CI runs gofmt and golangci-lint, which are two different things : https://golangci-lint.run/

c4rt0 · 2024-10-15T16:06:31Z

I installed the version of golangci-lint as per the pipeline. The output seems to have nothing to do with code I modified. Here's the result of golangci-lint:

adamsky@fedorapc Work/coreos-assembler (PR/badness_1) » golangci-lint --version                                               
golangci-lint has version 1.55.1 built with go1.21.3 from 9b20d49d on 2023-10-25T10:03:43Z

adamsky@fedorapc Work/coreos-assembler (PR/badness_1) » golangci-lint run mantle/cmd/kola/testiso.go

mantle/cmd/kola/testiso.go:50:12: undefined: preRun (typecheck)
                PreRunE: preRun,
                         ^
mantle/cmd/kola/testiso.go:501:3: undefined: plog (typecheck)
                plog.Fatal(err)
                ^
mantle/cmd/kola/testiso.go:519:39: undefined: outputDir (typecheck)
        outputDir, err = kola.SetupOutputDir(outputDir, "testiso")
                                             ^
mantle/cmd/kola/testiso.go:525:29: undefined: outputDir (typecheck)
        reportDir := filepath.Join(outputDir, "reports")
                                   ^
mantle/cmd/kola/testiso.go:608:53: undefined: outputDir (typecheck)
                        duration, err = testPXE(ctx, inst, filepath.Join(outputDir, test))
                                                                         ^
mantle/cmd/kola/testiso.go:629:5: undefined: plog (typecheck)
                                plog.Fatalf("Unknown test name:%s", test)
                                ^
mantle/cmd/kola/testiso.go:633:4: undefined: plog (typecheck)
                        plog.Fatalf("Unknown test name:%s", test)
                        ^
mantle/cmd/kola/testiso.go:347:2: undefined: root (typecheck)
        root.AddCommand(cmdTestIso)

All golangci-lint errors I get for the harness.go, testiso.go and qemu.go seem to be unrelated to my modifications.

Is this common, or am I missing something?

HuijingHei · 2024-10-16T08:19:40Z

Maybe can try newer go version like 1.22, see coreos/repo-templates#248 and coreos/repo-templates#249 (as 1.21 is EOL)

c4rt0 · 2024-10-17T13:27:17Z

Maybe can try newer go version like 1.22, see coreos/repo-templates#248 and coreos/repo-templates#249 (as 1.21 is EOL)

I tried it and it didn't help:

adamsky@fedorapc Work/coreos-assembler (PR/badness_1) » golangci-lint --version                                                                                         1 ↵
golangci-lint has version 1.61.0 built with go1.23.1 from a1d6c560 on 2024-09-09T17:44:42Z
adamsky@fedorapc Work/coreos-assembler (PR/badness_1) » golangci-lint run mantle/cmd/kola/testiso.go                                                                    1 ↵
mantle/cmd/kola/testiso.go:50:12: undefined: preRun (typecheck)
                PreRunE: preRun,
                         ^
mantle/cmd/kola/testiso.go:501:3: undefined: plog (typecheck)
                plog.Fatal(err)
                ^
mantle/cmd/kola/testiso.go:519:39: undefined: outputDir (typecheck)
        outputDir, err = kola.SetupOutputDir(outputDir, "testiso")
                                             ^
mantle/cmd/kola/testiso.go:525:29: undefined: outputDir (typecheck)
        reportDir := filepath.Join(outputDir, "reports")
                                   ^
mantle/cmd/kola/testiso.go:608:53: undefined: outputDir (typecheck)
                        duration, err = testPXE(ctx, inst, filepath.Join(outputDir, test))
                                                                         ^
mantle/cmd/kola/testiso.go:629:5: undefined: plog (typecheck)
                                plog.Fatalf("Unknown test name:%s", test)
                                ^
mantle/cmd/kola/testiso.go:633:4: undefined: plog (typecheck)
                        plog.Fatalf("Unknown test name:%s", test)
                        ^
mantle/cmd/kola/testiso.go:347:2: undefined: root (typecheck)
        root.AddCommand(cmdTestIso)
        ^
adamsky@fedorapc Work/coreos-assembler (PR/badness_1) » golangci-lint run mantle/kola/harness.go    
mantle/kola/harness.go:1265:14: printf: non-constant format string in call to (*github.com/coreos/coreos-assembler/mantle/harness.H).Fatalf (govet)
                                c.Fatalf(errors.Wrapf(err, "kolet failed: %s", stderr).Error())
                                         ^

mantle/cmd/kola/testiso.go

mantle/kola/harness.go

mantle/platform/qemu.go

mantle/cmd/kola/testiso.go

After tests are complete this checks for bad lines in the console.txt and journal.txt files. Ref: coreos#3788 Co-authored-by: Dusty Mabe <[email protected]>

dustymabe · 2024-11-08T19:26:55Z

In the interest of time I'll push up a commit that is working in local testing.

c4rt0

/lgtm

c4rt0 · 2024-11-11T11:37:24Z

Thank you @dustymabe for the last update.

Changes implemented

openshift-ci bot added the do-not-merge/work-in-progress label Oct 1, 2024

c4rt0 requested review from jbtrystram and removed request for jbtrystram October 1, 2024 19:41

cverna reviewed Oct 2, 2024

View reviewed changes

mantle/platform/qemu.go Outdated Show resolved Hide resolved

c4rt0 force-pushed the PR/badness_1 branch 3 times, most recently from dd184f1 to fcb5eb7 Compare October 15, 2024 12:48

c4rt0 changed the title ~~qemu.go, harness.go: Add CheckConsoleForBadness & CheckConsoleText~~ testiso.go: Add go routine to handle badness Oct 15, 2024

dustymabe previously requested changes Oct 17, 2024

View reviewed changes

mantle/cmd/kola/testiso.go Show resolved Hide resolved

mantle/cmd/kola/testiso.go Outdated Show resolved Hide resolved

mantle/kola/harness.go Outdated Show resolved Hide resolved

mantle/platform/qemu.go Outdated Show resolved Hide resolved

c4rt0 force-pushed the PR/badness_1 branch 2 times, most recently from 03fc17e to 40290bd Compare November 7, 2024 18:26

c4rt0 changed the title ~~testiso.go: Add go routine to handle badness~~ testiso.go: Add check for badness Nov 7, 2024

c4rt0 marked this pull request as ready for review November 7, 2024 18:37

openshift-ci bot removed the do-not-merge/work-in-progress label Nov 7, 2024

dustymabe reviewed Nov 7, 2024

View reviewed changes

c4rt0 force-pushed the PR/badness_1 branch from 3d7109d to 4d50394 Compare November 8, 2024 15:19

dustymabe reviewed Nov 8, 2024

View reviewed changes

mantle/cmd/kola/testiso.go Outdated Show resolved Hide resolved

testiso.go: Add check for bad console/journal lines

dbe4048

After tests are complete this checks for bad lines in the console.txt and journal.txt files. Ref: coreos#3788 Co-authored-by: Dusty Mabe <[email protected]>

dustymabe force-pushed the PR/badness_1 branch from 4d50394 to dbe4048 Compare November 8, 2024 19:29

c4rt0 commented Nov 11, 2024

View reviewed changes

c4rt0 mentioned this pull request Nov 11, 2024

kola testiso tests should check for badness in console/journal output #3788

Closed

dustymabe approved these changes Nov 12, 2024

View reviewed changes

dustymabe merged commit 58df48e into coreos:main Nov 12, 2024
5 checks passed

c4rt0 deleted the PR/badness_1 branch November 12, 2024 09:37

testiso.go: Add check for badness #3895

testiso.go: Add check for badness #3895

Uh oh!

Conversation

c4rt0 commented Oct 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Oct 1, 2024

Uh oh!

Uh oh!

dustymabe commented Oct 4, 2024

Uh oh!

c4rt0 commented Oct 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jbtrystram commented Oct 15, 2024

Uh oh!

c4rt0 commented Oct 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuijingHei commented Oct 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

c4rt0 commented Oct 17, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dustymabe commented Nov 8, 2024

Uh oh!

c4rt0 left a comment

Choose a reason for hiding this comment

Uh oh!

c4rt0 commented Nov 11, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

c4rt0 commented Oct 1, 2024 •

edited

Loading

c4rt0 commented Oct 15, 2024 •

edited

Loading

c4rt0 commented Oct 15, 2024 •

edited

Loading

HuijingHei commented Oct 16, 2024 •

edited

Loading