[BUG] DWI parse doesn't handle many well known older DWI files

[zaphod77]

Is there an existing issue for this?

• I have searched the existing issues and none matched.

Operating System

Windows 11

CPU

I5 8th gen

GPU

geforce gtx 1060 3GB

Storage

No response

Game Version

5.0.0-042

Game Mode

dance

Theme

Default theme

Describe the problem

Many DWI files that work fine on DWI itself and Stepamnia 3.9 do not play back properly. I have to edit them in stepmania 3.9 and save them to fix this. For an easily obtained example, grab the Bemaniruler originals pack, and try Angelfalls. Or DWInsanity Sandstorm.

They are fine in 3.9. I'm not sure when this got broken, but it's been broken ever since StepMania 5.0

Describe what should happen

DWI files that worked fine in Stepmania 3.9 should work fine here too. instead they tend to be vastly out of sync if they have extreme bpm changes, freeze arrows, or greater than 1/16th notes in them.

Relevant Log output

nothing relevant, the DWI parser simply doesn't parse things properly. StepMania 5.0 is the same.

[JoseVarelaP]

Taking a look into this. I recently noticed the holds part specifically and took care of that, but i do notice some other songs that parse differently.

[zaphod77]

Well turns out that other dwi files in packs are just flat out wrong. The only thing i could actually confirm wrong in outfox was hold parsing. That and it randomly saying there are fakes, when there aren't any.

Angefalls, for example, is simply incorrect in the stepmania pack. Fortunately i have the correct version of the file (the maniac only version), and was able to repair the all difficulties version. So many of these parse errors are actually errors in the dwi file. Just because you remember it working back in the day doesn't mean the DWI file is right.

the DWI format requires that every panel specified after the ! also be specified before the !. there are files that break this rule in packs. They are invalid. The bad Angelfalls that has steps other than maniac is one of these files. The way to test if a file is being parsed wrong is to test it in Dance With Intensity. I do have this, so please link me the offending simfiles so i can compare. The bad angefalls does not work in DWI either. The stepsets being significantly different numbers of measures is a big clue here.

I have confirmed that <> for multi panel hits is valid before the !, but not after it in Dance With Intensity itself. DWI does have an undocumented "use <> for 192 notes if there's a 0" in there, and there are dwis in the wild that do this. usually it's done to make DWI end earlier.

I recommend that malformed holds (missing the panel before the !) be logged at least, and possibly outright rejected.

Which dwi files still parse differently in Dance With Intensity and in Outfox?

[JoseVarelaP]

One that i still remember in my head going whack was “Today (FILO & PERI Remix)” on DWI Extreme: Glistening to Infinity 7. That one was making holds super longer. The fix I did does fix the holds, but the quants are still shifted after the first verse.

I do also have a build of DWI mounted on my pc to compare, but I’m nowhere close to my pc right now to test.

[zaphod77]

I have analyzed the file.
Basic and Another end on the same arrow. Basic has some arrows ended with "jumps" one foot goes up while other one goes down. Another does the same thing but with two freezes a measure.
Maniac has an odd freeze in measure 11. it starts on an 8th note, but is ended normally. By measure 21, things are back to normal, but the final measure is longer, having a freeze instead of a single arrow.
Smaniac has freezes starting on 8th notes in the first five measures. It also has holds ended inside a <>.

What difficulties are affected?

The files play correctly in dwi. It plays fine in stepmania 3.9. I tplays fine in stepmania 5 with my rewritten parser. i'm not sure if it plays fine i stock stepmania 5 yet. please test there.

I'm willing to send my rewrite if asked. Unlike the mind bending stepmania code, it doesn't keep jumping back and forth, and has a normal incrementer in the for loop. Upon finding a note, it frist checks to make sure there is actually a next note (compares i+1 with length-1). If there is a next character then it peesk at it to see if it's a !. If so, it increments i. then is does the same check to make SURE i +1 is still less than length-1. if so, it then advances again, and reads the character. If we haven't hit a break byy then it finally only advances the beat if we aren't in a multipanel hit.

It also logs overlapping ([{`, which as far as I know should never happen in a .dwi file and doesn't in any valid one. It still will change the beat increment, though, like the old code did. I do have dwi to test with, if we wish to replicate DWI's behavior, but i don't think that matters.

[zaphod77]

I can confirm treating any left bracket as a change to beat advance per number, and any right braket as a switch back to normal is correct, because of one particularly silly "It's Tricky" dwi file from tournamix 2 that tries to use (()) for 32nd notes, ((())) for 64th notes, and (((()))) for 128th notes to put the arrows on beat. it doesn't actually work, even in Dance With Intensity. you end up with a 16th note, and the first ) goes back to 8th notes.

[zaphod77]

my loader is here. feel free to port it into outfox and stress test it.

stepmania/stepmania#2302

[JoseVarelaP]

my loader is here. feel free to port it into outfox and stress test it.

stepmania/stepmania#2302

I’ll read it in probably 2-3 hours, cause I'm currently swarmed in celebrating new years (I wanna tackle this already)

[JoseVarelaP]

I have done a fix for the DWI parser, and checked out the files you suggested. Sandstorm passed with flying colors, but Angelfalls was a peculiar subject, as I realized that the bugged chart was also present in the actual DWI build.

0001-12015.comprimido.mp4

Do you happen to have a version of Angelfalls that has "proper" BPM speeds? Because this file is technically correct, bugs and all.