Skip to content

Low performance using HDF5 MPI parallel dependency #3

@antonelepfl

Description

@antonelepfl

This ticket is manly for Nuvla setup

Hi, I've been testing the performance and even though it seems like it writes in parallel at the end when I select with reports the cpus don't work at 100% during the simulation process making the simulation in general work slower than before.

Test with: Hippocampus Microcircuit (O1/20181114)

250 cells of cental column

Nodes Numprocs Cells ms
1 8 250 300

BB5

Module No Reports With Report Neuron Version Module
Old (serial) 06:57 07:44 7.6.2-2 2018-08-24 neurodamus/hippocampus
New (parallel) 08:00 07:15 7.6.2-2 2018-08-24 neurodamus-hippocampus

DOCKER (OpenStack VM)

Module No Reports With Report Neuron Version Module
Old (serial) 19:09 19:30 7.6.2-3 2018-08-28 neurodamus/hippocampus
New (parallel) 20:37 20:32 7.6.2-28 2018-10-18 neurodamus-hippocampus

DOCKER (Nuvla)

Module No Reports With Report Neuron Version Module
Old (serial) 12:15 22:33 7.6.2-3 2018-08-28 neurodamus/hippocampus
New (parallel) 13:52 32:45 7.6.2-28 2018-10-18 neurodamus-hippocampus

To test the raw read/write performance with the help of Judit we used IOR

Machine File Read Write NFS
OpenStack testFile.ior 2100 400 NO
Nuvla Worker testFile.ior 170 4 YES
Nuvla WorkerS testFile.ior 440 2.5 YES

Where WorkerS means 4 process 2 in each worker VM with MPI

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions