Request that AppData json files be formatted and sorted before saving to facilitate better version control

[dlg0]

At present, the AppData/*.json files are stored as a single long string. These strings contain both unformatted json and xml. This makes version control on these files impractical.

I'd request that prior to Veda saving these files, it applies a formatter to both the xml and json strings such that instead of blobs like

[{"CaseId":110,"CreatedOn":"2024-04-16 01:35","Description":"The parametric scenarios for the Climate Change Authority Project, initial years based on the reference case","EditedOn":"2024-05-07 00:10","EndingYear":"2060","FixResultFileName":"True","FixResultInfo":{"WorkTimesFolderPath":"C:\\Users\\gre538\\code\\Model_Aus_TIMES\\AppData\\GAMSSAVE","GdxElasticDermands":{"GdxSelectedFile":{"FileName":"","FilePath":"","IsSelected":false},"IsApplied":true},"GdxIre":{"GdxSelectedFile":{"FileName":"","FilePath":"","IsSelected":false},"IsApplied":true,"RadioSelection":1},"GdxUseSolution":{"FixYearsUpto":"2025","GdxSelectedFile":{"FileName":"cca_ref_p","FilePath":"C:\\Users\\gre538\\code\\Model_Aus_TIMES\\AppData\\GAMSSAVE\\cca_ref_p.gdx","IsSelected":true},"IsApplied":true,"RadioSelection":1},"IsApplyFixResult":true},"GAMSSourceFolder":"GAMS_SrcTIMES.v4.7.6","Name":"CCA_parscens","ParametricGroup":"all_austimes_CCA1234","ParametricGroupId":604,"PeriodsDefinition":"msy_2060_18p","PropertiesGroup":"save sol info (DLG)","PropertiesGroupId":608,"RegionGroup":"AllRegion","RegionGroupId":512,"ScenarioGroup":"CCA_parbase","ScenarioGroupId":605,"Solver":"cplex","SolverOptionFile":"BA121-7_nco","UserName":"gre538"},{"CaseId":104,"CreatedOn":"2024-01-16 06:04","Description":"demo2 taj CCA","EditedOn":"2024-01-16 13:49","EndingYear":"2050","FixResultFileName":"False","FixResultInfo":{"WorkTimesFolderPath":null,"GdxElasticDermands":{"GdxSelectedFile":{"FileName":"","FilePath":"","IsSelected":false},"IsApplied":false},"GdxIre":{"GdxSelectedFile":{"FileName":"","FilePath":"","IsSelected":false},"IsApplied":false,"RadioSelection":1},"GdxUseSolution":{"FixYearsUpto":"","GdxSelectedFile":{"FileName":"","FilePath":"","IsSelected":false},"IsApplied":false,"RadioSelection":1},"IsApplyFixResult":false},"GAMSSourceFolder":"GAMS_SrcTIMES.v4.7.6","Name":"demo2_CCA","ParametricGroup":"all_austimes_CCA-Copy","ParametricGroupId":530,"PeriodsDefinition":"msy_2050_5p","PropertiesGroup":"DefaultProperties-Copy","PropertiesGroupId":538,"RegionGroup":"AllRegion-Copy","RegionGroupId":531,"ScenarioGroup":"AllScenario-Copy","ScenarioGroupId":527,"Solver":"cplex","SolverOptionFile":"CPLEX","UserName":"kha082"},{"CaseId":109,"CreatedOn":"2024-02-13 22:21","Description":"The parametric scenarios for the Climate Change Authority Project, initial years based on the reference case","EditedOn":"2024-04-14 23:47","EndingYear":"2050","FixResultFileName":"True","FixResultInfo":{"WorkTimesFolderPath":"E:\\VEDA\\VEDA_Models\\Bri267_CCA\\AppData\\GAMSSAVE","GdxElasticDermands":{"GdxSelectedFile":{"FileName":"","FilePath":"","IsSelected":false},"IsApplied":true},"GdxIre":{"GdxSelectedFile":{"FileName":"","FilePath":"","IsSelected":false},"IsApplied":true,"RadioSelection":1},"GdxUseSolution":{"FixYearsUpto":"None","GdxSelectedFile":{"FileName":"cca__ref_p","FilePath":"E:\\VEDA\\VEDA_Models\\Bri267_CCA\\AppData\\GAMSSAVE\\cca__ref_p.gdx","IsSelected":true},"IsApplied":true,"RadioSelection":2},

we have files like

[
    {
        "CaseId": 110,
        "CreatedOn": "2024-04-16 01:35",
        "Description": "The parametric scenarios for the Climate Change Authority Project, initial years based on the reference case",
        "EditedOn": "2024-05-07 00:10",
        "EndingYear": "2060",
        "FixResultFileName": "True",
        "FixResultInfo": {
            "WorkTimesFolderPath": "C:\\Users\\gre538\\code\\Model_Aus_TIMES\\AppData\\GAMSSAVE",
            "GdxElasticDermands": {
                "GdxSelectedFile": {
                    "FileName": "",
                    "FilePath": "",
                    "IsSelected": false
                },
                "IsApplied": true
            },
            "GdxIre": {
                "GdxSelectedFile": {
                    "FileName": "",
                    "FilePath": "",
                    "IsSelected": false
                },
                "IsApplied": true,

I've also found that it is important to sort the files such that it allows useful merge between versions where chunks exists in one file but not another.

If it helps, I did whip up a python merge tool for the results and reports views json files which can be found here ... https://github.com/dlg0/austimes-tools/blob/master/src/austimes_tools/merge_appdata_json_files.py

[deepakgupta256]

We have implemented the requested formatting, which will be included in the next update. Please note that this may slightly increase the file size. However, even for large files, any extra time required for reading will be negligible and likely unnoticeable.