Skip to content

Bug: Tests reusing onces dependency-cacheΒ #524

@bendiknesbo

Description

@bendiknesbo

Bug Description
We want to unit-test our mage-targets, but ended up in a weird behaviour where tests would success or fail based on which order they were ran in. This turns out to be because runDeps is loading a sync.Once from a global variable, and this onces is not cleared between tests (and there does not appear to be any way to clear this cache at all).

What did you do?
I have this Terraform-target for running terraform init in different directories:

// Init initializes a terraform project
func Init(ctx context.Context, directories []string) error {
	modules := []any{}
	for _, workDir := range directories {
		modules = append(modules, mg.F(initTerraform, workDir))
	}

	mg.SerialCtxDeps(ctx, modules...)
	return nil
}

And then I have the following unit-test:

func TestInitTarget(t *testing.T) {
	tests := []struct {
		name    string
		workdir string
		wantErr bool
	}{
		{
			name:    "Terraform Init should succeed",
			workdir: "tests/succes",
			wantErr: false,
		},
		{
			name:    "Terraform Init should fail",
			workdir: "tests/test-fail",
			wantErr: true,
		},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			t.Chdir(tt.workdir)
			gotErr := Init(context.Background(), []string{"."})
			if tt.wantErr && gotErr == nil {
				t.Error("expected error, but got nothing")
			} else if !tt.wantErr && gotErr != nil {
				t.Errorf("expected no error, but got following error: %v", gotErr)
			}
		})
	}
}

Along with some test-data directories, where tests/success contains terraform-code that should end up with a success on terraform init command, and tests/test-fail directory contains invalid terraform-code that should end up failing the command.

If I run both tests, then the second test-case will not pass, because it is expecting an error, but it is reusing the sync.Once from the successful test, which has already completed without error.

What did you expect to happen?

I expected the initTerraform function to be run for both unit-tests, but it is only run for the first test.

What actually happened?

The sync.Once is reused, even when the environment has changed.

Environment

  • Mage Version: v1.15.0
  • OS: Mac and Linux

Additional context

The only environment that changed in this instance was the current working-directory, but other parameters could have changed as well, like environment-variables, values smuggled in the context, and even the current time. I'm not sure if the correct way to solve this is to make the onceMap more aware of the environment, or if there should just exist a global function for unit-tests to clean up every global cache that mage adds, that can be invoked on test-startup (and would take a parameter like *testing.T or an interface that *testing.T implements, like Cleanup).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions