-
Notifications
You must be signed in to change notification settings - Fork 42
More tests #51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
More tests #51
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
7935dfb
accept a closure in RunAsStep
maxdml e156816
fix
maxdml 88cf424
fix
maxdml fb5b0b6
fix
maxdml dfcff91
fix
maxdml a62ee45
check special steps
maxdml 555c367
remove t.Fatal calls in wf/steps
maxdml 6d38d3f
fix race in step name map
maxdml 3ebbf9f
add test for running workflows in goroutines
maxdml 63e2c78
more concurrent tests
maxdml f81fff4
set version
maxdml cacb25c
allow custom workflow names
maxdml 83b8148
test conflicting execution of workflows / tasks of the same ID
maxdml 071eac6
nit
maxdml 1071108
test that running a recorded child workflow, from a parent, returns a…
maxdml 3069590
Merge branch 'main' into more-tests
maxdml a9a83a1
nits
maxdml e9d5c4f
fix test + nits
maxdml 04390cf
use a sync.Map
maxdml 2960cce
for now return a flat error if we try to enqueue a workflow in a diff…
maxdml File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,6 +8,7 @@ import ( | |
| "math" | ||
| "reflect" | ||
| "runtime" | ||
| "sync" | ||
| "time" | ||
|
|
||
| "github.com/google/uuid" | ||
|
|
@@ -204,10 +205,11 @@ type WrappedWorkflowFunc func(ctx DBOSContext, input any, opts ...WorkflowOption | |
| type workflowRegistryEntry struct { | ||
| wrappedFunction WrappedWorkflowFunc | ||
| maxRetries int | ||
| name string | ||
| } | ||
|
|
||
| // Register adds a workflow function to the registry (thread-safe, only once per name) | ||
| func registerWorkflow(ctx DBOSContext, workflowName string, fn WrappedWorkflowFunc, maxRetries int) { | ||
| func registerWorkflow(ctx DBOSContext, workflowFQN string, fn WrappedWorkflowFunc, maxRetries int, customName string) { | ||
| // Skip if we don't have a concrete dbosContext | ||
| c, ok := ctx.(*dbosContext) | ||
| if !ok { | ||
|
|
@@ -221,14 +223,15 @@ func registerWorkflow(ctx DBOSContext, workflowName string, fn WrappedWorkflowFu | |
| c.workflowRegMutex.Lock() | ||
| defer c.workflowRegMutex.Unlock() | ||
|
|
||
| if _, exists := c.workflowRegistry[workflowName]; exists { | ||
| c.logger.Error("workflow function already registered", "fqn", workflowName) | ||
| panic(newConflictingRegistrationError(workflowName)) | ||
| if _, exists := c.workflowRegistry[workflowFQN]; exists { | ||
| c.logger.Error("workflow function already registered", "fqn", workflowFQN) | ||
| panic(newConflictingRegistrationError(workflowFQN)) | ||
| } | ||
|
|
||
| c.workflowRegistry[workflowName] = workflowRegistryEntry{ | ||
| c.workflowRegistry[workflowFQN] = workflowRegistryEntry{ | ||
| wrappedFunction: fn, | ||
| maxRetries: maxRetries, | ||
| name: customName, | ||
| } | ||
| } | ||
|
|
||
|
|
@@ -274,6 +277,7 @@ func registerScheduledWorkflow(ctx DBOSContext, workflowName string, fn Workflow | |
| type workflowRegistrationParams struct { | ||
| cronSchedule string | ||
| maxRetries int | ||
| name string | ||
| } | ||
|
|
||
| type workflowRegistrationOption func(*workflowRegistrationParams) | ||
|
|
@@ -294,6 +298,12 @@ func WithSchedule(schedule string) workflowRegistrationOption { | |
| } | ||
| } | ||
|
|
||
| func WithWorkflowName(name string) workflowRegistrationOption { | ||
| return func(p *workflowRegistrationParams) { | ||
| p.name = name | ||
| } | ||
| } | ||
|
|
||
| // RegisterWorkflow registers the provided function as a durable workflow with the provided DBOSContext workflow registry | ||
| // If the workflow is a scheduled workflow (determined by the presence of a cron schedule), it will also register a cron job to execute it | ||
| // RegisterWorkflow is generically typed, providing compile-time type checking and allowing us to register the workflow input and output types for gob encoding | ||
|
|
@@ -347,7 +357,7 @@ func RegisterWorkflow[P any, R any](ctx DBOSContext, fn GenericWorkflowFunc[P, R | |
| } | ||
| return &workflowPollingHandle[any]{workflowID: handle.GetWorkflowID(), dbosContext: ctx}, nil // this is only used by recovery and queue runner so far -- queue runner dismisses it | ||
| }) | ||
| registerWorkflow(ctx, fqn, typeErasedWrapper, registrationParams.maxRetries) | ||
| registerWorkflow(ctx, fqn, typeErasedWrapper, registrationParams.maxRetries, registrationParams.name) | ||
|
|
||
| // If this is a scheduled workflow, register a cron job | ||
| if registrationParams.cronSchedule != "" { | ||
|
|
@@ -397,6 +407,7 @@ func WithApplicationVersion(version string) WorkflowOption { | |
| } | ||
| } | ||
|
|
||
| // An internal option we use to map the reflection function name to the registration options. | ||
| func withWorkflowName(name string) WorkflowOption { | ||
| return func(p *workflowParams) { | ||
| p.workflowName = name | ||
|
|
@@ -484,6 +495,9 @@ func (c *dbosContext) RunAsWorkflow(_ DBOSContext, fn WorkflowFunc, input any, o | |
| if registeredWorkflow.maxRetries > 0 { | ||
| params.maxRetries = registeredWorkflow.maxRetries | ||
| } | ||
| if len(registeredWorkflow.name) > 0 { | ||
| params.workflowName = registeredWorkflow.name | ||
| } | ||
|
|
||
| // Check if we are within a workflow (and thus a child workflow) | ||
| parentWorkflowState, ok := c.Value(workflowStateKey).(*workflowState) | ||
|
|
@@ -705,7 +719,12 @@ func setStepParamDefaults(params *StepParams, stepName string) *StepParams { | |
| BackoffFactor: 2.0, | ||
| BaseInterval: 100 * time.Millisecond, // Default base interval | ||
| MaxInterval: 5 * time.Second, // Default max interval | ||
| StepName: typeErasedStepNameToStepName[stepName], | ||
| StepName: func() string { | ||
| if value, ok := typeErasedStepNameToStepName.Load(stepName); ok { | ||
| return value.(string) | ||
| } | ||
| return "" // This should never happen | ||
| }(), | ||
| } | ||
| } | ||
|
|
||
|
|
@@ -719,15 +738,17 @@ func setStepParamDefaults(params *StepParams, stepName string) *StepParams { | |
| if params.MaxInterval == 0 { | ||
| params.MaxInterval = 5 * time.Second // Default max interval | ||
| } | ||
| if params.StepName == "" { | ||
| if len(params.StepName) == 0 { | ||
| // If the step name is not provided, use the function name | ||
| params.StepName = typeErasedStepNameToStepName[stepName] | ||
| if value, ok := typeErasedStepNameToStepName.Load(stepName); ok { | ||
| params.StepName = value.(string) | ||
| } | ||
| } | ||
|
|
||
| return params | ||
| } | ||
|
|
||
| var typeErasedStepNameToStepName = make(map[string]string) | ||
| var typeErasedStepNameToStepName sync.Map | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This map should be read-mostly. The first invocation of a step will store the reflection name. |
||
|
|
||
| func RunAsStep[R any](ctx DBOSContext, fn GenericStepFunc[R]) (R, error) { | ||
| if ctx == nil { | ||
|
|
@@ -742,7 +763,8 @@ func RunAsStep[R any](ctx DBOSContext, fn GenericStepFunc[R]) (R, error) { | |
|
|
||
| // Type-erase the function | ||
| typeErasedFn := StepFunc(func(ctx context.Context) (any, error) { return fn(ctx) }) | ||
| typeErasedStepNameToStepName[runtime.FuncForPC(reflect.ValueOf(typeErasedFn).Pointer()).Name()] = stepName | ||
| typeErasedFnName := runtime.FuncForPC(reflect.ValueOf(typeErasedFn).Pointer()).Name() | ||
| typeErasedStepNameToStepName.LoadOrStore(typeErasedFnName, stepName) | ||
|
|
||
| // Call the executor method and pass through the result/error | ||
| result, err := ctx.RunAsStep(ctx, typeErasedFn) | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's going on here? What's the difference between FQN and customName?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And the tests seem to use neither?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The FQN is the reflection name, which we get with
runtime.FuncForPC(reflect.ValueOf(fn).Pointer()).Name().The FQN is the only information we get at runtime when a user calls
RunAsWorkflow.A user can set a custom name when registering a workflow. There are now tests exercising this path.
The custom name is stored in the registry, alongside other registration-time parameters like
maxRetries.RunAsWorkflowresolve the workflow name at runtime when looking up registration options (from the registry.)