Skip to content

Add versioning support in durabletask-dotnet(phase1) #295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
skyao wants to merge 20 commits into
base: main
Choose a base branch
from
Open

Add versioning support in durabletask-dotnet(phase1) #295

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
380c622
Add a new constructor that sets the version expected by the client
skyao Apr 22, 2024
1976de6
Add new abstract method to get instance version
skyao Apr 22, 2024
4e1f8cb
Implemented new added abstract method to get instance version
skyao Apr 22, 2024
f6162ff
change to get instance version from this.innerContext.Version
skyao Apr 23, 2024
d564c79
Merge branch 'main' into versioning-phase1
skyao Apr 26, 2024
d9298d0
change InstanceVersion to virtual and return null be default to avoid…
skyao Apr 26, 2024
60c1341
Merge branch 'versioning-phase1' of github.com:skyao/durabletask-dotn…
skyao Apr 26, 2024
fc44a4a
Merge branch 'main' into versioning-phase1
skyao Apr 29, 2024
5b1b2b2
Merge branch 'main' into versioning-phase1
skyao May 7, 2024
cb86ecd
Merge branch 'main' into versioning-phase1
skyao May 21, 2024
83bc227
update constructor to handle null correctly
skyao May 21, 2024
f1619b1
Merge branch 'main' into versioning-phase1
skyao Jul 5, 2024
0045ced
update Equals/GetHashCode method to include version field
skyao Jul 10, 2024
2172e96
use value.ToString() to include version field
skyao Jul 10, 2024
5c3ad58
rollback file formator to use 4 space indentation
skyao Jul 16, 2024
7912d46
use Microsoft.Bcl.HashCode to do hash
skyao Jul 16, 2024
d015d3e
fix typo
skyao Jul 16, 2024
87b15e1
udpate documents for version
skyao Jul 18, 2024
6ecfde1
add static FromString() method and update the string conversion opera…
skyao Jul 18, 2024
220fb2d
Merge branch 'main' into versioning-phase1
skyao Aug 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
250 changes: 146 additions & 104 deletions src/Abstractions/TaskName.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,127 +8,169 @@ namespace Microsoft.DurableTask;
/// </summary>
public readonly struct TaskName : IEquatable<TaskName>
{
// TODO: Add detailed remarks that describe the role of TaskName
// TODO: Add detailed remarks that describe the role of TaskName

/// <summary>
/// Initializes a new instance of the <see cref="TaskName"/> struct.
/// </summary>
/// <param name="name">The name of the task. Providing <c>null</c> will yield the default struct.</param>
public TaskName(string name)
/// <summary>
/// Initializes a new instance of the <see cref="TaskName"/> struct.
/// </summary>
/// <param name="name">The name of the task. Providing <c>null</c> will yield the default struct.</param>
public TaskName(string name)
{
if (name is null)
{
if (name is null)
{
// Force the default struct when null is passed in.
this.Name = null!;
this.Version = null!;
}
else
{
this.Name = name;
this.Version = string.Empty; // expose setting Version only when we actually consume it.
}
// Force the default struct when null is passed in.
this.Name = null!;
this.Version = null!;
}

/// <summary>
/// Gets the name of the task without the version.
/// </summary>
/// <value>
/// The name of the activity task without the version.
/// </value>
public string Name { get; }

/// <summary>
/// Gets the version of the task.
/// </summary>
/// <remarks>
/// Task versions is currently locked to <see cref="string.Empty" /> as it is not yet integrated into task
/// identification. This is being left here as we intend to support it soon.
/// </remarks>
public string Version { get; }

/// <summary>
/// Implicitly converts a <see cref="TaskName"/> into a <see cref="string"/> of the <see cref="Name"/> property value.
/// </summary>
/// <param name="value">The <see cref="TaskName"/> to be converted into a string.</param>
public static implicit operator string(TaskName value) => value.Name;

/// <summary>
/// Implicitly converts a <see cref="string"/> into a <see cref="TaskName"/> value.
/// </summary>
/// <param name="value">The string to convert into a <see cref="TaskName"/>.</param>
public static implicit operator TaskName(string value) => string.IsNullOrEmpty(value) ? default : new(value);

/// <summary>
/// Compares two <see cref="TaskName"/> objects for equality.
/// </summary>
/// <param name="a">The first <see cref="TaskName"/> to compare.</param>
/// <param name="b">The second <see cref="TaskName"/> to compare.</param>
/// <returns><c>true</c> if the two <see cref="TaskName"/> objects are equal; otherwise <c>false</c>.</returns>
public static bool operator ==(TaskName a, TaskName b)
else
{
return a.Equals(b);
this.Name = name;
this.Version = string.Empty; // expose setting Version only when we actually consume it.
}
}

/// <summary>
/// Compares two <see cref="TaskName"/> objects for inequality.
/// </summary>
/// <param name="a">The first <see cref="TaskName"/> to compare.</param>
/// <param name="b">The second <see cref="TaskName"/> to compare.</param>
/// <returns><c>true</c> if the two <see cref="TaskName"/> objects are not equal; otherwise <c>false</c>.</returns>
public static bool operator !=(TaskName a, TaskName b)
/// <summary>
/// Initializes a new instance of the <see cref="TaskName"/> struct.
/// </summary>
/// <param name="name">The name of the task. Providing <c>null</c> will yield the default struct.</param>
/// <param name="version">The version of the task.</param>
public TaskName(string name, string version)
{
if (name is null)
Copy link
Member

@jviau jviau Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good opportunity for pattern matching. Can replace this if/else statements with:

    (this.Name, this.Version) = (name, version) switch
    {
        (null, null) => (null!, null!), // Force the default struct when null is passed in.
        (null, string _) => throw new ArgumentException("name must not be null when version is non-null"),
        (string _, null) => (name, string.Empty),
        (string _, string _) => (name, version),
    };

Also while we are at it, can replace the other ctor with:

    (this.Name, this.Version) = name switch
    {
        null => (null!, null!), // Force the default struct when null is passed in.
        string _ => (name, string.Empty),
    }

{
return !a.Equals(b);
if (version is null)
{
// Force the default struct when null is passed in.
this.Name = null!;
this.Version = null!;
}
else
{
throw new ArgumentException("name must not be null when version is non-null");
}
}

/// <summary>
/// Gets a value indicating whether to <see cref="TaskName"/> objects
/// are equal using value semantics.
/// </summary>
/// <param name="other">The other object to compare to.</param>
/// <returns><c>true</c> if the two objects are equal using value semantics; otherwise <c>false</c>.</returns>
public bool Equals(TaskName other)
else
{
return string.Equals(this.Name, other.Name, StringComparison.OrdinalIgnoreCase);
if (version is null)
{
this.Name = name;
this.Version = string.Empty; // fallback to the contructor without version parameter
}
else
{
this.Name = name;
this.Version = version;
}
}
}

/// <summary>
/// Gets a value indicating whether to <see cref="TaskName"/> objects
/// are equal using value semantics.
/// </summary>
/// <param name="obj">The other object to compare to.</param>
/// <returns><c>true</c> if the two objects are equal using value semantics; otherwise <c>false</c>.</returns>
public override bool Equals(object? obj)
{
if (obj is not TaskName other)
{
return false;
}
/// <summary>
/// Gets the name of the task without the version.
/// </summary>
/// <value>
/// The name of the activity task without the version.
/// </value>
public string Name { get; }

/// <summary>
/// Gets the version of the task.
/// </summary>
/// <remarks>
/// Task versions is currently locked to <see cref="string.Empty" /> as it is not yet integrated into task
/// identification. This is being left here as we intend to support it soon.
/// </remarks>
public string Version { get; }

/// <summary>
/// Implicitly converts a <see cref="TaskName"/> into a <see cref="string"/> of the <see cref="Name"/> property value.
/// </summary>
/// <param name="value">The <see cref="TaskName"/> to be converted into a string.</param>
public static implicit operator string(TaskName value) => value.ToString();

/// <summary>
/// Implicitly converts a <see cref="string"/> into a <see cref="TaskName"/> value.
/// </summary>
/// <param name="value">The string to convert into a <see cref="TaskName"/>.</param>
public static implicit operator TaskName(string value) => string.IsNullOrEmpty(value) ? default : new(value);

/// <summary>
/// Compares two <see cref="TaskName"/> objects for equality.
/// </summary>
/// <param name="a">The first <see cref="TaskName"/> to compare.</param>
/// <param name="b">The second <see cref="TaskName"/> to compare.</param>
/// <returns><c>true</c> if the two <see cref="TaskName"/> objects are equal; otherwise <c>false</c>.</returns>
public static bool operator ==(TaskName a, TaskName b)
{
return a.Equals(b);
}

return this.Equals(other);
/// <summary>
/// Compares two <see cref="TaskName"/> objects for inequality.
/// </summary>
/// <param name="a">The first <see cref="TaskName"/> to compare.</param>
/// <param name="b">The second <see cref="TaskName"/> to compare.</param>
/// <returns><c>true</c> if the two <see cref="TaskName"/> objects are not equal; otherwise <c>false</c>.</returns>
public static bool operator !=(TaskName a, TaskName b)
{
return !a.Equals(b);
}

/// <summary>
/// Gets a value indicating whether to <see cref="TaskName"/> objects
/// are equal using value semantics.
/// </summary>
/// <param name="other">The other object to compare to.</param>
/// <returns><c>true</c> if the two objects are equal using value semantics; otherwise <c>false</c>.</returns>
public bool Equals(TaskName other)
{
return string.Equals(this.Name, other.Name, StringComparison.OrdinalIgnoreCase)
&& string.Equals(this.Version, other.Version, StringComparison.OrdinalIgnoreCase);
}

/// <summary>
/// Gets a value indicating whether to <see cref="TaskName"/> objects
/// are equal using value semantics.
/// </summary>
/// <param name="obj">The other object to compare to.</param>
/// <returns><c>true</c> if the two objects are equal using value semantics; otherwise <c>false</c>.</returns>
public override bool Equals(object? obj)
{
if (obj is not TaskName other)
{
return false;
}

/// <summary>
/// Calculates a hash code value for the current <see cref="TaskName"/> instance.
/// </summary>
/// <returns>A 32-bit hash code value.</returns>
public override int GetHashCode()
return this.Equals(other);
}

/// <summary>
/// Calculates a hash code value for the current <see cref="TaskName"/> instance.
/// </summary>
/// <returns>A 32-bit hash code value.</returns>
public override int GetHashCode()
{
if (string.IsNullOrEmpty(this.Version))
{
return StringComparer.OrdinalIgnoreCase.GetHashCode(this.Name);
return StringComparer.OrdinalIgnoreCase.GetHashCode(this.Name);
}

/// <summary>
/// Gets the string value of the current <see cref="TaskName"/> instance.
/// </summary>
/// <returns>The name and optional version of the current <see cref="TaskName"/> instance.</returns>
public override string ToString()
return StringComparer.OrdinalIgnoreCase.GetHashCode(this.Name) * 31
+ StringComparer.OrdinalIgnoreCase.GetHashCode(this.Version);
}

/// <summary>
/// Gets the string value of the current <see cref="TaskName"/> instance.
/// </summary>
/// <returns>The name and optional version of the current <see cref="TaskName"/> instance.</returns>
public override string ToString()
{
if (string.IsNullOrEmpty(this.Version))
{
return this.Name;
}
else
{
if (string.IsNullOrEmpty(this.Version))
{
return this.Name;
}
else
{
return this.Name + ":" + this.Version;
}
return this.Name + ":" + this.Version;
Copy link
Member

@jviau jviau May 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider adding FromString semantics. But I wonder if that will be a breaking change? We have not restricted what characters can be used in a task name before this.

This change is going to introduce a confusing behavior:

TaskName name1 = new("MyTask", "2"); // Name = "MyTask", Version = "2'
TaskName name2 = name1.ToString(); // implicit operator convers. Name = "MyTask:2", Version = ""

Copy link
Member

@cgillum cgillum Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the implicit operator be updated to parse out the version?

I'm not super concerned about the breaking change since 99+% of .NET users should be relying on the function name, which already can't have special characters. However, we can take a look at Kusto data to confirm.

Copy link
Member

@jviau jviau Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the implicit operator would be changed to call FromString.

Copy link
Member

@jviau jviau Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have a discussion on what separator to use. I like @, as that is what we have used for distributed tracing to indicate the version - and ":" was used to separate the task type: {task_type}:{name}@{version}

However, we already use @ to separate entity name and key - will that be an issue? Or is that a good thing to use @ as a separator in both for consistency.

Copy link
Member

@cgillum cgillum Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using @ could break entity ID parsing. Looking at this code, I can see that entity IDs take the form of {name}@{key} and the code that parses this looks for the first instance of @ to separate the name from the key. If we inject a version into the name, we make it {name}@{version}@{key}, and the key for entities will be parsed as {version}@{key} instead of {key}.

Copy link
Member

@jviau jviau Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cgillum yeah I am trying to think through how that behaves. One thing is that it is a separate type for entities. We have a few options:

  1. use @ for separator. Entity instance IDs can then look like @{entity}@{version}@{key} (or we can do key first, then version). RISK: keys could already contain @. This could break those instances.
    • @sebastianburckhardt suggested we could prefix with double @@ to indicate this entity has a version field. So @myEntity@some@key -> no version field. @@myEntity@version@some@key -> version field present.
  2. use a different character. @{entity}?{version}@{key} (or key first). RISK: this character could already be used by customers in name or key
  3. Alternatively, we could opt to not support versioning of entities just yet

Copy link
Member

@jviau jviau Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to clarify that I am not opposed to a different separator char for version. We haven't officially GA'd distributed tracing v2, so we could update that spec with the new separator. But what one exactly?

Some that I have ruled out myself in the past:

  • / - avoided this so it wasn't confused with HTTP routes in distributed tracing
  • % - often used for encoding special chars, so wanted to avoid that
  • * - wild card, wanted to avoid that.
  • ! - used in netherite for partition targeting. We are looking to also support that in other backends.
  • : - used in distributed tracing to separate the task-type. ie: orchestration:name@version or activity:name@version

Copy link
Author

@skyao skyao Jul 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we merge this PR first and then open a new issue to trace and update the separator after we have a final decision?

Because today is my last day, I'm afraid that I can't continue to update the code in this PR because of permissions.

Copy link
Author

@skyao skyao Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cgillum @jviau please have a look at this PR and let's make the final decision.

}
}
}
5 changes: 5 additions & 0 deletions src/Abstractions/TaskOrchestrationContext.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ public abstract class TaskOrchestrationContext
/// </summary>
public abstract string InstanceId { get; }

/// <summary>
/// Gets the version of the current orchestration instance.
/// </summary>
public virtual string? InstanceVersion => null;

/// <summary>
/// Gets the parent instance or <c>null</c> if there is no parent orchestration.
/// </summary>
Expand Down
3 changes: 3 additions & 0 deletions src/Worker/Core/Shims/TaskOrchestrationContextWrapper.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ public TaskOrchestrationContextWrapper(
/// <inheritdoc/>
public override string InstanceId => this.innerContext.OrchestrationInstance.InstanceId;

/// <inheritdoc/>
public override string InstanceVersion => this.innerContext.Version;

/// <inheritdoc/>
public override ParentOrchestrationInstance? Parent => this.invocationContext.Parent;

Expand Down