Keras v3 Support by calad0i · Pull Request #1116 · fastmachinelearning/hls4ml

calad0i · 2024-11-08T14:09:06Z

A# Description

Add keras v3 specific object-based parser and some layer handlers (no h5 or json loading supported). The current keras parser doesn't work with v3 functional models in general.

Type of change

New feature (non-breaking change which adds functionality)

Tests

test/pytest/test_keras_v3_api.py

Test Configuration:

Requires keras>=3.0. Skips the whole module if this requirement is not sufficed.

Checklist

I have read the guidelines for contributing.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation.
My changes generate no new warnings.
I have installed and run pre-commit on the files I edited or added.
I have added tests that prove my fix is effective or that my feature works.

calad0i · 2024-11-08T17:54:57Z

This PR will be rebased after its prerequisites are merged to resolve the conflicts.

jmitrevs · 2025-04-15T19:43:10Z

hls4ml/utils/config.py

+        if keras.__version__ > '3.0':
+            layer_list, *_ = hls4ml.converters.parse_keras_v3_model(model)
+        else:
+            model_arch = json.loads(model.to_json())


I don't know if the case was used in practice, but the old logic allows model to be a dict, and only if it's not not a dict, only then model_arch = json.loads(model.to_json()). Does the logic here work if model is a dict?

dict is a representation of a json. we work with a dict that was a json of a model whether made by us or, in the old days, by the user.

model.get_config has slightly different behavior, where objects may be passed as-is (not serialized). v2 converter assumes a dict of fully serialized model config, so serialize+deserialize is used here.

vloncar

cosmetics mostly. run ruff on changed files before merging

docs/frontend/keras.rst

vloncar · 2025-05-23T18:02:07Z

docs/frontend/keras.rst


+
+* `QKeras <https://github.com/fastmachinelearning/qkeras>`_
+    The equivalent QKeras API and its quantizers are also supported by ``hls4ml``. QKeras is not compatible with Keras v3.


Mention the alternatives (from Marius, HGQ2, via quantizers lib etc)

vloncar · 2025-05-23T18:11:51Z

docs/intro/setup.rst

-* `TensorFlow <https://pypi.org/project/tensorflow/>`_ (version 2.8 to 2.14) and `QKeras <https://pypi.org/project/qkeras/>`_ are required by the Keras converter. One may want to install newer versions of QKeras from GitHub. Newer versions of TensorFlow can be used, but QKeras and hl4ml do not currently support Keras v3.
+The ``hls4ml`` library requires python 3.10 or later, and depends on a number of Python packages and external tools for synthesis and simulation. Python dependencies are automatically managed by ``pip`` or ``conda``.
+
+The following Python packages are all optional and are only required if you intend to use the corresponding converter. Only install the packages you need.


the advice "only install packages you need" was already given above

vloncar · 2025-05-23T18:27:15Z

hls4ml/converters/__init__.py

 from hls4ml.converters.keras_to_hls import get_supported_keras_layers  # noqa: F401
 from hls4ml.converters.keras_to_hls import parse_keras_model  # noqa: F401
 from hls4ml.converters.keras_to_hls import keras_to_hls, register_keras_layer_handler
+from hls4ml.converters.keras_v3_to_hls import parse_keras_v3_model  # noqa: F401


rename keras_to_hls to keras_v2_to_hls

vloncar · 2025-05-23T18:27:30Z

hls4ml/backends/vivado/passes/einsum.py

+        # EinsumDense config
+        params = default_params.copy()
+        params['strategy'] = strategy
+        params['n_free0'] = node.attributes.attributes['n_free0']


use either node.attributes[] or node.get_attr()

vloncar · 2025-05-23T19:14:17Z

hls4ml/templates/vivado/nnet_utils/nnet_einsum.h

+    static const unsigned strategy;
+    static const unsigned reuse_factor;
+    static const unsigned multiplier_limit;
+    static const bool store_weights_in_bram = false; // NOT USED


remove it then

vloncar · 2025-05-23T19:26:27Z

test/pytest/test_keras_v3_api.py

@@ -0,0 +1,516 @@
+import math


keep for now for internal testing (as it is not triggered by the CI), but will be abandoned in the future in favor of spread out tests

vloncar · 2025-05-23T19:43:35Z

hls4ml/converters/keras_v3_to_hls.py

+def get_io_tensors(layer: 'keras.Layer', node_whitelist: set[int] | None = None):
+    '''Given a keras layer, return a list of tuples of input and output
+    tensors. If the layer is called only once (i.e., no shared layers),
+    the list will contain only one tuple.


add an example where this will happen

hls4ml/converters/keras_v3_to_hls.py

vloncar · 2025-05-23T19:50:27Z

hls4ml/converters/keras_v3_to_hls.py

+            return None
+        return handler(layer, inp_tensors, out_tensors)
+
+    def v2_call(


maybe add a comment on the rough list of layers that use this feature as this is to be deprecated

Can't really think of useful layers not yet covered. Haven't seen the fallback warning for a while.

vloncar · 2025-05-30T17:19:32Z

Two things to note for a future TODO:

RNN is not supported via v3 handler for now
We should get rid of n_in and n_out in pooling and standardize naming as with Conv

jmitrevs added this to the v1.1.0 milestone Nov 8, 2024

calad0i mentioned this pull request Nov 15, 2024

⚡ Add ESD Support to hls4ml #1129

Closed

2 tasks

vloncar mentioned this pull request Nov 22, 2024

KeyError: 'batch_input_shape' #1135

Closed

4 tasks

calad0i force-pushed the keras-v3 branch from 0b1918d to 6b8a44c Compare December 16, 2024 21:53

calad0i mentioned this pull request Dec 17, 2024

TF model implementation in tf version 2.18.0 #1150

Open

4 tasks

add keras v3 object parser

47f3e3c

calad0i force-pushed the keras-v3 branch from d2ccfb4 to 6f284ec Compare March 11, 2025 19:45

calad0i added 9 commits March 11, 2025 13:08

add keras v3 layer handlers

5debe71

expose kv3 parser to config interface

755be89

add kv3 converter test

a1c2227

einsumdense and einsum

067ef9e

add einsum templates

d8bb729

einsumdense test

303db72

support kv3 parsed batchnorm

56c0731

fix einsum/einsum dense regression issue

fe3fcd0

preemptive distributed_arithmetic flag for einsum ops

54a297e

calad0i force-pushed the keras-v3 branch from 1931653 to 54a297e Compare March 11, 2025 20:12

update doc for kv3

3509666

calad0i added the please test Trigger testing by creating local PR branch label Mar 11, 2025

more documentation

c81028e

vloncar self-requested a review March 11, 2025 21:40

vloncar self-assigned this Mar 11, 2025

bo3z modified the milestones: v1.1.0, v1.2.0 Apr 8, 2025

JanFSchulte mentioned this pull request Apr 15, 2025

Add support for einsum operation to pytorch parser (requires 1116) #1273

Merged

7 tasks

jmitrevs reviewed Apr 15, 2025

View reviewed changes

calad0i added 3 commits April 15, 2025 18:23

backport validate einsum function

eed6330

docstring style

cda903e

quote format

eccde4e

calad0i added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Apr 18, 2025

pre-commit update

8284757

calad0i added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Apr 18, 2025

Merge branch 'main' into keras-v3

35e94d0

calad0i added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Apr 18, 2025

calad0i mentioned this pull request May 5, 2025

error in config #1280

Open

vloncar requested changes May 23, 2025

View reviewed changes

calad0i added 12 commits May 27, 2025 07:00

kv3 handler update

e5ad92c

force keras>=3.10

6aec7f6

isolate merge handlers

64261aa

rm abomination

b8ed033

mv xpose config gen to utils

009ae8e

attributes.attributes -> attributes

3ea3490

isolate keras v2 and v3 to hls

5d4bdfe

update tests for api changes

150a3f6

update docs

ec914d1

mv einops to vivado backend, rm unused args

312328e

Merge branch 'main' into keras-v3

a15a353

post merge fix

9c585aa

calad0i requested a review from vloncar May 27, 2025 14:44

calad0i added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels May 27, 2025

calad0i added 2 commits May 30, 2025 02:02

quality-of-life changes

8153522

fix some qol changes

c4733b2

vloncar approved these changes May 30, 2025

View reviewed changes

vloncar merged commit dd75412 into fastmachinelearning:main May 30, 2025
5 checks passed



		* `QKeras <https://github.com/fastmachinelearning/qkeras>`_
		The equivalent QKeras API and its quantizers are also supported by ``hls4ml``. QKeras is not compatible with Keras v3.

Conversation

calad0i commented Nov 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of change

Tests

Checklist

Uh oh!

calad0i commented Nov 8, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vloncar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vloncar commented May 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

calad0i commented Nov 8, 2024 •

edited

Loading