Testing of v2.0.0a1 (profile maintenance)

[ramirezfranciscof]

One last thing to test (you can even do this from your just migrated profiles from the profile migration test), is to run the maintenance on them. If you already backed up everything before the migration you don't need any other preparation, but if you only backed up the database you may want to now also create a backup of the repository before proceeding (if you have the space, the easiest is to just make a copy of your .aiida folder).

For this I would say:

1. Run verdi storage info --statistics (or --detailed instead if you updated after ♻️ REFACTOR: Storage info flag statistics -> detailed #5416) and record the stats on the repository:

  Packs: 1
  Objects:
    unpacked: 1
    packed: 26101
  Size (MB):
    unpacked: 0.00030994415283203125
    packed: 1223.617220878601
    other: 4.03125

2. Run the full maintenance timed (time verdi storage maintain --full). Note that this will ask for your confirmation ([y/N]) so accept that quickly so as not to interfere with the timing.
3. Run verdi storage info --statistics/--detailed again.

You can report the following:

1. Maintenance time.
2. Consistency: the sum between Objects->unpacked and Objects->packed should remain the same before/after.
3. Optimization: check by how much the total size (unpacked+packed+other) got reduced with the maintenance.

Afterwards you can also try re-running any of the "after migration" tests you may have used to check the consistency of your storage or just use it in case you detect any issue.

[giovannipizzi]

Profile: dispero2020-clone

output of verdi storage info --detailed (before maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 10
  Objects:
    unpacked: 0
    packed: 1055220
  Size (MB):
    unpacked: 0.0
    packed: 39215.35681724548
    other: 174.859375

output of verdi storage info --detailed (after maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 10
  Objects:
    unpacked: 0
    packed: 1055220
  Size (MB):
    unpacked: 0.0
    packed: 38475.280999183655
    other: 165.76953125

Time of verdi storage maintain --full: 6m12.015s

[giovannipizzi]

Profile: development-clone

output of verdi storage info --detailed (before maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 3
  Objects:
    unpacked: 0
    packed: 349629
  Size (MB):
    unpacked: 0.0
    packed: 9410.567880630493
    other: 57.720703125

output of verdi storage info --detailed (after maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 3
  Objects:
    unpacked: 0
    packed: 349629
  Size (MB):
    unpacked: 0.0
    packed: 9126.969013214111
    other: 57.720703125

Time of verdi storage maintain --full: 1m20s

[giovannipizzi]

Profile: bindingenergy2d-clone

output of verdi storage info --detailed (before maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 1
  Objects:
    unpacked: 0
    packed: 182248
  Size (MB):
    unpacked: 0.0
    packed: 2456.6087532043457
    other: 30.05078125

output of verdi storage info --detailed (after maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 1
  Objects:
    unpacked: 0
    packed: 182248
  Size (MB):
    unpacked: 0.0
    packed: 2245.629898071289
    other: 28.4453125

Time of verdi storage maintain --full: 31s

[giovannipizzi]

Profile: leonid (theossrv5)

output of verdi -p leonid storage info --detailed (before maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 6
  Objects:
    unpacked: 0
    packed: 195911
  Size (MB):
    unpacked: 0.0
    packed: 22208.83609008789
    other: 34.6591796875

output of verdi -p leonid storage info --detailed (after maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 6
  Objects:
    unpacked: 0
    packed: 195911
  Size (MB):
    unpacked: 0.0
    packed: 22010.268579483032
    other: 32.318359375

Time of time verdi -p leonid storage maintain --full: 2m25.267s

[giovannipizzi]

Profile: arsenene (theossrv5)

output of verdi -p arsenene storage info --detailed (before maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 5
  Objects:
    unpacked: 0
    packed: 20523
  Size (MB):
    unpacked: 0.0
    packed: 18059.172158241272
    other: 3.6171875

output of verdi -p arsenene storage info --detailed (after maintenance):

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 5
  Objects:
    unpacked: 0
    packed: 20523
  Size (MB):
    unpacked: 0.0
    packed: 18028.811470985413
    other: 3.3837890625

Time of time verdi -p arsenene storage maintain --full: 1m10.175s

[sphuber]

Only have a small dev profile locally. Already ran the maintain command before reading this post so don't have the stats before or the runtime, but it was of the order of a few seconds.

repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1
  Packs: 1
  Objects:
    unpacked: 0
    packed: 19844
  Size (MB):
    unpacked: 0.0
    packed: 42.047701835632324
    other: 3.0546875

[ramirezfranciscof]

@giovannipizzi thanks for the data points. I just have a question: did you run the maintenance on migrated profiles or in profiles resulting from importing an archive directly into a 2.0 installation?

[giovannipizzi]

@ramirezfranciscof always only on the migrated profiles