-
Notifications
You must be signed in to change notification settings - Fork 329
FIX: ISXB-1674 - Input actions asset not converted correctly when upgrading from 1.14.1 (Regression) #2244
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
FIX: ISXB-1674 - Input actions asset not converted correctly when upgrading from 1.14.1 (Regression) #2244
Conversation
Codecov ReportAttention: Patch coverage is
@@ Coverage Diff @@
## develop #2244 +/- ##
===========================================
+ Coverage 68.14% 76.75% +8.60%
===========================================
Files 367 465 +98
Lines 53685 87938 +34253
===========================================
+ Hits 36584 67496 +30912
- Misses 17101 20442 +3341 Flags with carried forward coverage won't be shown. Click here to find out more.
... and 99 files with indirect coverage changes 🚀 New features to boost your workflow:
|
I noticed this PR title was incorrect so I changed it to remove that failure. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for looking into this, my main comment at this review round is to add inline comments explaining why we need to do the different parts since it is not clear to me from the code. I think this is important to make sure its maintainable.
if (mapJson.actions == null || mapJson.actions.Length == 0) | ||
continue; | ||
|
||
for (var ai = 0; ai < mapJson.actions.Length; ++ai) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be possible to add some inline comments to the different parts of this large for loop or split it up so that is clear what the intent is? It is quite difficult to review given that there is no motivation given for what we do here.
if (mapJson.actions == null || mapJson.actions.Length == 0) | ||
continue; | ||
|
||
for (var ai = 0; ai < mapJson.actions.Length; ++ai) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like 'ai' isn't really used so why not e.g foreach instead?
if (parts.Length == 0) | ||
continue; | ||
|
||
var tokens = new List<string>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is inside a loop I would recommend moving tokens
outside the loop and just do clear
here to avoid GC pressure.
if (tokens.Count == 0) | ||
continue; | ||
|
||
var parsed = new List<NameAndParameters>(tokens.Count); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also this list may be moved to outer scope to avoid allocating on every iteration.
foreach (var t in tokens) | ||
parsed.Add(NameAndParameters.Parse(t)); | ||
|
||
var rebuiltTokens = new List<string>(tokens.Count); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same with this list
continue; | ||
} | ||
|
||
var dict = new Dictionary<string, string>(nap.parameters.Count, System.StringComparer.OrdinalIgnoreCase); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This dict could also be outer scope?
var dict = nap.parameters.ToDictionary(p => p.name, p => p.value.ToString()); | ||
var anyChanged = false; | ||
foreach (var field in procType.GetFields(BindingFlags.Public | BindingFlags.Instance).Where(f => f.FieldType.IsEnum)) | ||
if (int.TryParse(rawVal, NumberStyles.Integer, CultureInfo.InvariantCulture, out var n)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do not understand what the goal is here, could you add inline comments explaining?
} | ||
} | ||
|
||
if (!changedThisProcessor) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also code from here to end is fuzzy to me, it would be helpful with inline comments explaining what we are trying to achieve in order to review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure I'll add comments in detail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a few things you mention in the PR description that I don't understand why they are needed:
Preserve original tokens and only touch the enum values
Formatting survive unchanged
I don't understand why this would be a requirement, I believe that if the user uses our editor to edit the asset that was written manually and has custom formatting that will overwrite the users formatting. Trying to parse json with string replaces, regex is a losing battle, the only way this won't be brittle is by parsing it, converting the data and serializing it back to json.
Avoid whole string rewrites, which previously caused mismatches between what the editor expected
As above, I don't understand why do we want to avoid string rewrites, my current thinking is that we didnt catch this for 2 reasons, we are not reusing the parsing and serializing funcitons that is used elsewhere, and we didnt test this case.
I believe the proper way of moving forward here is that we use the exact same code to parse and serialize json for this type that is used elsewhere in the editor, if we try to do anything different we are creating an opportunity for a bug to be hidden here.
var legacyJson = m_Asset.ToJson().Replace("\"version\": 1", "\"version\": 0").Replace("Custom(SomeEnum=10)", "Custom(SomeEnum=1)"); | ||
|
||
// Add a trailing processor to verify the semicolon separator is preserved. | ||
if (!legacyJson.Contains(";InvertVector2(invertX=true)")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact we have an conditional treatment on our test data before we act on it is not a good sign, ideally the data we use for testing is known ahead of time and we can just apply the function under test on it.
Whenever we need to bump the current version to 2 the replace above will fail, which is not what the test is trying to test.
I am wondering if a better approach here would be to just have a static json string here instead.
if (parsedJson.version >= JsonVersion.Version1) | ||
return; | ||
if ((parsedJson.maps?.Length ?? 0) > 0 && (parsedJson.version) < JsonVersion.Version1) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to do this parsing manually ourselves here? Is there anything special that we need in this parsing that is no needed elsewhere?
I'm assuming this data is used in other parts of the system, and there would be logic we can reuse... now if there is an issue with the parsing of jsonmaps we have to fix in 2 places...
"Expected a comma between the first and second processors, with InvertVector2 first." | ||
); | ||
|
||
Assert.Greater(migratedJson.IndexOf(";InvertVector2(invertX=true)", StringComparison.Ordinal), migratedJson.IndexOf("Custom(SomeEnum=20)", StringComparison.Ordinal), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm also wondering if asserting on a substring is the most robust way here... The substring might be there but other things could make the json parsing fail... What we want to make sure is that after the migration the json can be parsed without issues and that the enum values are correct.
We even have the migratedAsset object, so let's just assert that the processors are set correctly.
// Verify processor order and that enum was converted properly | ||
Assert.That(processors, Does.Contain("StickDeadzone"), "StickDeadzone processor missing."); | ||
Assert.That(processors, Does.Contain("InvertVector2(invertX=false)"), "InvertVector2 missing."); | ||
Assert.That(processors, Does.Contain("Custom(SomeEnum=20)"), "Custom(SomeEnum=1) should migrate to SomeEnum=20 (OptionB)."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test is not verifying that the processor string can be parsed properly, which is what we want... we could still have a ";" separating them and those asserts would most likely pass.
@ekcoh do you know how we could get access to the parsed list of processors so we could assert on that instead? I could only find InputControl.processors or InputActionState.processors, but I don't know how to go from an InputActionAsset to InputControl or InputActionState
It seems InputAction.ResolveBindings end up calling InputBindingResolver.AddActionMap which then calls InstantiateWithParameters and that leads to the parsing and creation of processors, but I am not sure how we could call that from a test and then retrieve the list of processors to assert on that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, since the API design of Input Actions is based on string you can put whatever into Processors. So this test needs to complemented with code resolving the actions and the only way to actually prove the processors is correctly parsed and applied I believe would be to enable the actions and do ReadValue on the action after being resolved. To drive that action you also likely need a fake device to drive it with e.g. 1.0 and then see that e.g. scale processor multiples by 10 or similar.
In CoreTests_Actions.cs there are various examples on how to achieve this, e.g. search for tests containing ReadValue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would have been trivial if the string based API design was not a thing and instead actions had a list of actual processors on them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @LeoUnity @ekcoh, our NameAndParameters.ParseMultiple() explicitly checks for both seperators ,
and ;
between processors entries. Either way we get the same result however I agree with semantic issues from string only assertions.
Should we go for
var parsed = NameAndParameters.ParseMultiple(action.processors).ToList(); Assert.That(parsed.Count, Is.EqualTo(expectedCount)); ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't it just work to do this? Its simple and very little code
var legacyJson = ....
var modernJson = ....
var ported = InputActionAsset.FromJson(legacyJson).ToJson();
Assert.That(modernJson, Is.EqualTo(ported));
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally you would have compared InputActionAsset objects but unfortunately it's not supported :/ Hence full cycle allows you to avoid the details. If porting is successful I would expect them both to be equal. Or even
var legacyJson = ....
var modernJson = ....
var ported = InputActionAsset.FromJson(legacyJson).ToJson();
Assert.That(InputActionAsset.FromJson(modernJson).ToJson(), Is.EqualTo(ported));
Since that wouldn't break either if formatting is updated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes it should work, earlier the cause of regression was not considering ;
separator in the migration logic it only splits ,
and keeps the ;
unchanged caused the serialization to break. Since now as @LeoUnity suggested the robust approach to use the NameAndParameters the above FromJson and ported json assertion should do good I believe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
our NameAndParameters.ParseMultiple() explicitly checks for both seperators , and ;
This is the source of the regression we have today and we are fixing on this PR. The comma and semicolon are both used, but in different contexts.
Take a look at this example, this should allow you to assert on the parsed processors:
var inputActionAsset = AssetDatabaseUtils.CreateAsset<InputActionAsset>();
var actionMap = inputActionAsset.AddActionMap("Action Map");
actionMap.AddAction("Action", InputActionType.Value, processors: "Custom(SomeEnum=10)").AddBinding("<Gamepad>/leftStick");
actionMap.ResolveBindings();
Assert.That(actionMap.m_State.processors[0].GetType(), Is.EqualTo(typeof(CustomProcessor)));
Assert.That((actionMap.m_State.processors[0] as CustomProcessor).SomeEnum, Is.EqualTo(SomeEnum.OptionA));
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as @LeoUnity, let's complement the test by making sure it parses correctly (logically) and then we can conclude this I believe.
Packages/com.unity.inputsystem/InputSystem/Utilities/NameAndParameters.cs
Show resolved
Hide resolved
|
||
var list = NameAndParameters.ParseMultiple(raw).ToList(); | ||
var rebuilt = new List<string>(list.Count); | ||
var converted = new List<NameAndParameters>(list.Count); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still do not see why this isn't moved to outer scope and reused over iterations instead of being allocated on every iteration? Am I missing something?
|
||
internal static NameAndParameters Create(string name, IList<NamedValue> parameters) | ||
{ | ||
var result = new NameAndParameters |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nitpick: Why create a result variable here instead of just
return new NameAndParameters { .... };
m_ListProperty.serializedObject.ApplyModifiedProperties(); | ||
} | ||
|
||
private static string ToSerializableString(IEnumerable<NameAndParameters> parametersForEachListItem) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this function existing if it only forwards to NameAndParameters.SerializeMultiple? Seems redundant
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe keep the old name instead?
Description
Bug: https://jira.unity3d.com/browse/ISXB-1674
Port: 1.14.X
Preserve original tokens and only touch the enum values. That means processor names, order, legacy processors, and formatting survive unchanged so the editor no longer collapses or marks them “Obsolete”.
Avoid whole string rewrites, which previously caused mismatches between what the editor expected and what was written back and only assign a new processor string if something actually changed.
Testing status & QA
Verified manually with the attached repro project.
Overall Product Risks
Complexity: 0
Halo Effect: 0
Comments to reviewers
Checklist
Before review:
Changed
,Fixed
,Added
sections.Area_CanDoX
,Area_CanDoX_EvenIfYIsTheCase
,Area_WhenIDoX_AndYHappens_ThisIsTheResult
.During merge:
NEW: ___
.FIX: ___
.DOCS: ___
.CHANGE: ___
.RELEASE: 1.1.0-preview.3
.After merge: