Layer Wise Activation Sparsity

[ben9809]

This pull request includes several changes to enhance the functionality and maintainability of the neurobench package. The most important change include adding a new metric for activation sparsity by layer, and enhancing the NeuronHook class to include layer names.

New Metric Addition:

• Added a new metric ActivationSparsityByLayer in neurobench/metrics/workload/activation_sparsity_by_layer.py to calculate the sparsity of model activations on a per-layer basis.
• Updated the __init__.py file in neurobench/metrics/workload to include the new ActivationSparsityByLayer metric in the __stateless__ list.

Hook Enhancements:

• Enhanced the NeuronHook class in neurobench/hooks/neuron.py to include an optional name parameter, allowing hooks to store the name of the layer they are attached to. [1] [2]
• Modified the is_activation_layer function and register_hooks method in neurobench/models/neurobench_model.py to include layer names when registering hooks. [1] [2]

[codecov]

Codecov Report

Attention: Patch coverage is 94.44444% with 2 lines in your changes missing coverage. Please review.

Project coverage is 77.83%. Comparing base (6cfbdbc) to head (10057d5).
Report is 8 commits behind head on dev.

Files with missing lines	Patch %	Lines
...h/metrics/workload/activation_sparsity_by_layer.py	92.59%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #252      +/-   ##
==========================================
+ Coverage   76.94%   77.83%   +0.89%     
==========================================
  Files          42       43       +1     
  Lines         772      803      +31     
  Branches      114      118       +4     
==========================================
+ Hits          594      625      +31     
+ Misses        133      132       -1     
- Partials       45       46       +1     

Flag	Coverage Δ
unittests	77.83% <94.44%> (+0.89%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jasonlyik]

Looks great, I think we should re-organize the metrics documentation setup. Right now we have pages for each of the metrics (https://neurobench.readthedocs.io/en/latest/metrics/index.html), but as there are more metrics this becomes less scalable. Potentially users may be confused whether the metrics written there are the only ones supported.

[jasonlyik]

@ben9809 It looks like there are test-code checks pending, though I can't tell which these refer to

[ben9809]

@ben9809 It looks like there are test-code checks pending, though I can't tell which these refer to

Fixed

[ben9809]

Looks great, I think we should re-organize the metrics documentation setup. Right now we have pages for each of the metrics (https://neurobench.readthedocs.io/en/latest/metrics/index.html), but as there are more metrics this becomes less scalable. Potentially users may be confused whether the metrics written there are the only ones supported.

Do you think we should include custom metrics under the metric page?

[jasonlyik]

Opened #254 to discuss metric documentation reorganization, this PR looks good to go