Using Vitis on hls4ml #823

zyt1024 · 2023-06-28T06:48:43Z

zyt1024
Jun 28, 2023

Hello,I‘m using hls4ml[main],and The CNN model I used is as follows:

Model: "keras_baseline"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_2 (InputLayer)        [(None, 16, 128, 1)]      0         
                                                                 
 conv_0 (Conv2D)             (None, 16, 128, 128)      128       
                                                                 
 bn_conv_0 (BatchNormalizati  (None, 16, 128, 128)     512       
 on)                                                             
                                                                 
 conv_act_0 (Activation)     (None, 16, 128, 128)      0         
                                                                 
 conv_1 (Conv2D)             (None, 16, 128, 256)      32768     
                                                                 
 bn_conv_1 (BatchNormalizati  (None, 16, 128, 256)     1024      
 on)                                                             
                                                                 
 conv_act_1 (Activation)     (None, 16, 128, 256)      0         
                                                                 
 conv_2 (Conv2D)             (None, 16, 128, 2)        512       
                                                                 
 bn_conv_2 (BatchNormalizati  (None, 16, 128, 2)       8         
 on)                                                             
                                                                 
 conv_act_2 (Activation)     (None, 16, 128, 2)        0         
                                                                 
 flatten_1 (Flatten)         (None, 4096)              0         
                                                                 
 dense_0 (Dense)             (None, 1)                 4096      
                                                                 
 bn_dense_0 (BatchNormalizat  (None, 1)                4         
 ion)                                                            
                                                                 
 dense_act_0 (Activation)    (None, 1)                 0         
                                                                 
 dense_1 (Dense)             (None, 32)                32        
                                                                 
 bn_dense_1 (BatchNormalizat  (None, 32)               128       
 ion)                                                            
                                                                 
 dense_act_1 (Activation)    (None, 32)                0         
                                                                 
 output_dense (Dense)        (None, 2)                 66        
                                                                 
 output_softmax (Activation)  (None, 2)                0         
                                                                 
=================================================================
Total params: 39,278
Trainable params: 38,440
Non-trainable params: 838
_________________________________________________________________

For elements with a number greater than 4096, I have set Strategy=Resource and ReuseFactor=64, but when I synthesized, an error still occurred. The strategy I used is as follows:

import hls4ml
import plotting

hls_config_q = hls4ml.utils.config_from_keras_model(model, granularity='name')
hls_config_q['Model']['ReuseFactor'] = 16
hls_config_q['Model']['Precision'] = 'ap_fixed<16,6>'
hls_config_q['Model']['Strategy'] = 'Resource'
hls_config_q['LayerName']['output_softmax']['Strategy'] = 'Stable'
for layer in ['conv_0', 'conv_1', 'conv_2']:
    hls_config_q['LayerName'][layer]['ReuseFactor'] = 64
    hls_config_q['LayerName'][layer]['Strategy'] = 'Resource'
print("-----------------------------------")
plotting.print_dict(hls_config_q)
print("-----------------------------------")

cfg_q = hls4ml.converters.create_config(backend='Vitis')
cfg_q['IOType'] = 'io_stream'  # Must set this if using CNNs!
cfg_q['HLSConfig'] = hls_config_q
cfg_q['KerasModel'] = model
cfg_q['OutputDir'] = 'ioCNN/model_1/hls4ml_prj_vitis'
cfg_q['XilinxPart'] = 'xcu200-fsgd2104-2-e'

hls_model = hls4ml.converters.keras_to_hls(cfg_q)
hls_model.compile()

ERROR: [XFORM 203-103] Array 'w27' : partitioned elements number (32768) has exeeded the threshold (4096), which may cause long run-time.
ERROR: [HLS 200-1715] Encountered problem during source synthesis
ERROR: [HLS 200-70] Pre-synthesis failed.

May I ask if this is a bug? If not, how can I solve this issue? Thanks！

bo3z · 2023-06-29T19:47:59Z

bo3z
Jun 29, 2023
Maintainer

Can you include copies of the generated hls4ml_config.yaml and firmware/parameters.h. after compiling the model - maybe the configuration wasn't parsed properly in the last step.

3 replies

zyt1024 Jun 30, 2023
Author

yes,sure.
hls4ml_config.yaml

Backend: Vitis
ClockPeriod: 5
HLSConfig:
  LayerName:
    bn_dense_0:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        scale: fixed<16,6>
      Trace: false
    bn_dense_1:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        scale: fixed<16,6>
      Trace: false
    conv_0:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        weight: fixed<16,6>
      ReuseFactor: 64
      Strategy: Resource
      Trace: false
    conv_0_linear:
      Precision:
        result: fixed<16,6>
      Trace: false
    conv_1:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        weight: fixed<16,6>
      ReuseFactor: 64
      Strategy: Resource
      Trace: false
    conv_1_linear:
      Precision:
        result: fixed<16,6>
      Trace: false
    conv_2:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        weight: fixed<16,6>
      ReuseFactor: 64
      Strategy: Resource
      Trace: false
    conv_2_linear:
      Precision:
        result: fixed<16,6>
      Trace: false
    conv_3:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        weight: fixed<16,6>
      Trace: false
    conv_3_linear:
      Precision:
        result: fixed<16,6>
      Trace: false
    conv_act_0:
      Precision:
        result: fixed<16,6>
      Trace: false
    conv_act_1:
      Precision:
        result: fixed<16,6>
      Trace: false
    conv_act_2:
      Precision:
        result: fixed<16,6>
      Trace: false
    conv_act_3:
      Precision:
        result: fixed<16,6>
      Trace: false
    dense_0:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        weight: fixed<16,6>
      Trace: false
    dense_0_linear:
      Precision:
        result: fixed<16,6>
      Trace: false
    dense_1:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        weight: fixed<16,6>
      Trace: false
    dense_1_linear:
      Precision:
        result: fixed<16,6>
      Trace: false
    dense_act_0:
      Precision:
        result: fixed<16,6>
      Trace: false
    dense_act_1:
      Precision:
        result: fixed<16,6>
      Trace: false
    flatten:
      Precision:
        result: fixed<16,6>
      Trace: false
    input_1:
      Precision:
        result: fixed<16,6>
      Trace: false
    output_dense:
      Precision:
        bias: fixed<16,6>
        result: fixed<16,6>
        weight: fixed<16,6>
      Trace: false
    output_dense_linear:
      Precision:
        result: fixed<16,6>
      Trace: false
    output_softmax:
      Precision:
        result: fixed<16,6>
      Strategy: Stable
      Trace: false
  Model:
    BramFactor: 1000000000
    Precision: ap_fixed<16,6>
    ReuseFactor: 16
    Strategy: Resource
    TraceOutput: false
IOType: io_stream
KerasModel: !keras_model 'ioCNN/model_1/hls4ml_prj_vitis/keras_model.h5'
OutputDir: ioCNN/model_1/hls4ml_prj_vitis
Part: xcku115-flvb2104-2-i
ProjectName: myproject
Stamp: D8b4a35F
XilinxPart: xcu200-fsgd2104-2-e

bo3z Jun 30, 2023
Maintainer

Can you upload firmware/parameters.h as well?

zyt1024 Jul 2, 2023
Author

sure,I'm sorry for taking so long to reply to your message.
firmware/parameters.h

#ifndef PARAMETERS_H_
#define PARAMETERS_H_

#include "ap_fixed.h"
#include "ap_int.h"

#include "nnet_utils/nnet_code_gen.h"
#include "nnet_utils/nnet_helpers.h"
// hls-fpga-machine-learning insert includes
#include "nnet_utils/nnet_activation.h"
#include "nnet_utils/nnet_activation_stream.h"
#include "nnet_utils/nnet_conv2d.h"
#include "nnet_utils/nnet_dense.h"
#include "nnet_utils/nnet_dense_compressed.h"
#include "nnet_utils/nnet_dense_stream.h"
#include "nnet_utils/nnet_sepconv2d_stream.h"

// hls-fpga-machine-learning insert weights
#include "weights/w24.h"
#include "weights/b24.h"
#include "weights/w25.h"
#include "weights/b25.h"
#include "weights/w26.h"
#include "weights/b26.h"
#include "weights/w15.h"
#include "weights/b15.h"
#include "weights/w18.h"
#include "weights/b18.h"
#include "weights/w21.h"
#include "weights/b21.h"

// hls-fpga-machine-learning insert layer-config
// conv_0
struct config24_mult : nnet::dense_config {
    static const unsigned n_in = 1;
    static const unsigned n_out = 128;
    static const unsigned reuse_factor = 64;
    static const unsigned strategy = nnet::resource;
    static const unsigned n_zeros = 0;
    static const unsigned multiplier_limit = DIV_ROUNDUP(n_in * n_out, reuse_factor) - n_zeros / reuse_factor;
    typedef model_default_t accum_t;
    typedef bias24_t bias_t;
    typedef conv_0_weight_t weight_t;
    template<class x_T, class y_T>
    using product = nnet::product::mult<x_T, y_T>;
};

struct config24 : nnet::conv2d_config {
    static const unsigned pad_top = 0;
    static const unsigned pad_bottom = 0;
    static const unsigned pad_left = 0;
    static const unsigned pad_right = 0;
    static const unsigned in_height = 16;
    static const unsigned in_width = 128;
    static const unsigned n_chan = 1;
    static const unsigned filt_height = 1;
    static const unsigned filt_width = 1;
    static const unsigned kernel_size = filt_height * filt_width;
    static const unsigned n_filt = 128;
    static const unsigned stride_height = 1;
    static const unsigned stride_width = 1;
    static const unsigned out_height = 16;
    static const unsigned out_width = 128;
    static const unsigned reuse_factor = 64;
    static const unsigned n_zeros = 0;
    static const unsigned multiplier_limit =
        DIV_ROUNDUP(kernel_size * n_chan * n_filt, reuse_factor) - n_zeros / reuse_factor;
    static const bool store_weights_in_bram = false;
    static const unsigned strategy = nnet::resource;
    static const nnet::conv_implementation implementation = nnet::conv_implementation::linebuffer;
    static const unsigned min_height = 1;
    static const unsigned min_width = 1;
    static const ap_uint<filt_height * filt_width> pixels[min_height * min_width];
    static const unsigned n_partitions = 2048;
    static const unsigned n_pixels = out_height * out_width / n_partitions;
    template<class data_T, class CONFIG_T>
    using fill_buffer = nnet::FillConv2DBuffer<data_T, CONFIG_T>;
    typedef model_default_t accum_t;
    typedef bias24_t bias_t;
    typedef conv_0_weight_t weight_t;
    typedef config24_mult mult_config;
    template<unsigned K, unsigned S, unsigned W>
    using scale_index_height = nnet::scale_index_regular<K, S, W>;
    template<unsigned K, unsigned S, unsigned W>
    using scale_index_width = nnet::scale_index_regular<K, S, W>;
};
const ap_uint<config24::filt_height * config24::filt_width> config24::pixels[] = {1};

// conv_act_0
struct relu_config5 : nnet::activ_config {
    static const unsigned n_in = 262144;
    static const unsigned table_size = 1024;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned reuse_factor = 16;
    typedef conv_act_0_table_t table_t;
};

// conv_1
struct config25_mult : nnet::dense_config {
    static const unsigned n_in = 128;
    static const unsigned n_out = 256;
    static const unsigned reuse_factor = 64;
    static const unsigned strategy = nnet::resource;
    static const unsigned n_zeros = 0;
    static const unsigned multiplier_limit = DIV_ROUNDUP(n_in * n_out, reuse_factor) - n_zeros / reuse_factor;
    typedef model_default_t accum_t;
    typedef bias25_t bias_t;
    typedef conv_1_weight_t weight_t;
    template<class x_T, class y_T>
    using product = nnet::product::mult<x_T, y_T>;
};

struct config25 : nnet::conv2d_config {
    static const unsigned pad_top = 0;
    static const unsigned pad_bottom = 0;
    static const unsigned pad_left = 0;
    static const unsigned pad_right = 0;
    static const unsigned in_height = 16;
    static const unsigned in_width = 128;
    static const unsigned n_chan = 128;
    static const unsigned filt_height = 1;
    static const unsigned filt_width = 1;
    static const unsigned kernel_size = filt_height * filt_width;
    static const unsigned n_filt = 256;
    static const unsigned stride_height = 1;
    static const unsigned stride_width = 1;
    static const unsigned out_height = 16;
    static const unsigned out_width = 128;
    static const unsigned reuse_factor = 64;
    static const unsigned n_zeros = 0;
    static const unsigned multiplier_limit =
        DIV_ROUNDUP(kernel_size * n_chan * n_filt, reuse_factor) - n_zeros / reuse_factor;
    static const bool store_weights_in_bram = false;
    static const unsigned strategy = nnet::resource;
    static const nnet::conv_implementation implementation = nnet::conv_implementation::linebuffer;
    static const unsigned min_height = 1;
    static const unsigned min_width = 1;
    static const ap_uint<filt_height * filt_width> pixels[min_height * min_width];
    static const unsigned n_partitions = 2048;
    static const unsigned n_pixels = out_height * out_width / n_partitions;
    template<class data_T, class CONFIG_T>
    using fill_buffer = nnet::FillConv2DBuffer<data_T, CONFIG_T>;
    typedef model_default_t accum_t;
    typedef bias25_t bias_t;
    typedef conv_1_weight_t weight_t;
    typedef config25_mult mult_config;
    template<unsigned K, unsigned S, unsigned W>
    using scale_index_height = nnet::scale_index_regular<K, S, W>;
    template<unsigned K, unsigned S, unsigned W>
    using scale_index_width = nnet::scale_index_regular<K, S, W>;
};
const ap_uint<config25::filt_height * config25::filt_width> config25::pixels[] = {1};

// conv_act_1
struct relu_config9 : nnet::activ_config {
    static const unsigned n_in = 524288;
    static const unsigned table_size = 1024;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned reuse_factor = 16;
    typedef conv_act_1_table_t table_t;
};

// conv_2
struct config26_mult : nnet::dense_config {
    static const unsigned n_in = 256;
    static const unsigned n_out = 2;
    static const unsigned reuse_factor = 64;
    static const unsigned strategy = nnet::resource;
    static const unsigned n_zeros = 0;
    static const unsigned multiplier_limit = DIV_ROUNDUP(n_in * n_out, reuse_factor) - n_zeros / reuse_factor;
    typedef model_default_t accum_t;
    typedef bias26_t bias_t;
    typedef conv_2_weight_t weight_t;
    template<class x_T, class y_T>
    using product = nnet::product::mult<x_T, y_T>;
};

struct config26 : nnet::conv2d_config {
    static const unsigned pad_top = 0;
    static const unsigned pad_bottom = 0;
    static const unsigned pad_left = 0;
    static const unsigned pad_right = 0;
    static const unsigned in_height = 16;
    static const unsigned in_width = 128;
    static const unsigned n_chan = 256;
    static const unsigned filt_height = 1;
    static const unsigned filt_width = 1;
    static const unsigned kernel_size = filt_height * filt_width;
    static const unsigned n_filt = 2;
    static const unsigned stride_height = 1;
    static const unsigned stride_width = 1;
    static const unsigned out_height = 16;
    static const unsigned out_width = 128;
    static const unsigned reuse_factor = 64;
    static const unsigned n_zeros = 0;
    static const unsigned multiplier_limit =
        DIV_ROUNDUP(kernel_size * n_chan * n_filt, reuse_factor) - n_zeros / reuse_factor;
    static const bool store_weights_in_bram = false;
    static const unsigned strategy = nnet::resource;
    static const nnet::conv_implementation implementation = nnet::conv_implementation::linebuffer;
    static const unsigned min_height = 1;
    static const unsigned min_width = 1;
    static const ap_uint<filt_height * filt_width> pixels[min_height * min_width];
    static const unsigned n_partitions = 2048;
    static const unsigned n_pixels = out_height * out_width / n_partitions;
    template<class data_T, class CONFIG_T>
    using fill_buffer = nnet::FillConv2DBuffer<data_T, CONFIG_T>;
    typedef model_default_t accum_t;
    typedef bias26_t bias_t;
    typedef conv_2_weight_t weight_t;
    typedef config26_mult mult_config;
    template<unsigned K, unsigned S, unsigned W>
    using scale_index_height = nnet::scale_index_regular<K, S, W>;
    template<unsigned K, unsigned S, unsigned W>
    using scale_index_width = nnet::scale_index_regular<K, S, W>;
};
const ap_uint<config26::filt_height * config26::filt_width> config26::pixels[] = {1};

// conv_act_2
struct relu_config13 : nnet::activ_config {
    static const unsigned n_in = 4096;
    static const unsigned table_size = 1024;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned reuse_factor = 16;
    typedef conv_act_2_table_t table_t;
};

// dense_0
struct config15 : nnet::dense_config {
    static const unsigned n_in = 4096;
    static const unsigned n_out = 1;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned strategy = nnet::resource;
    static const unsigned reuse_factor = 16;
    static const unsigned n_zeros = 0;
    static const unsigned n_nonzeros = 4096;
    static const unsigned multiplier_limit = DIV_ROUNDUP(n_in * n_out, reuse_factor) - n_zeros / reuse_factor;
    static const bool store_weights_in_bram = false;
    typedef model_default_t accum_t;
    typedef bias15_t bias_t;
    typedef dense_0_weight_t weight_t;
    typedef layer15_index index_t;
    template<class x_T, class y_T>
    using product = nnet::product::mult<x_T, y_T>;
};

// dense_act_0
struct relu_config17 : nnet::activ_config {
    static const unsigned n_in = 1;
    static const unsigned table_size = 1024;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned reuse_factor = 16;
    typedef dense_act_0_table_t table_t;
};

// dense_1
struct config18 : nnet::dense_config {
    static const unsigned n_in = 1;
    static const unsigned n_out = 32;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned strategy = nnet::resource;
    static const unsigned reuse_factor = 16;
    static const unsigned n_zeros = 0;
    static const unsigned n_nonzeros = 32;
    static const unsigned multiplier_limit = DIV_ROUNDUP(n_in * n_out, reuse_factor) - n_zeros / reuse_factor;
    static const bool store_weights_in_bram = false;
    typedef model_default_t accum_t;
    typedef bias18_t bias_t;
    typedef dense_1_weight_t weight_t;
    typedef layer18_index index_t;
    template<class x_T, class y_T>
    using product = nnet::product::mult<x_T, y_T>;
};

// dense_act_1
struct relu_config20 : nnet::activ_config {
    static const unsigned n_in = 32;
    static const unsigned table_size = 1024;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned reuse_factor = 16;
    typedef dense_act_1_table_t table_t;
};

// output_dense
struct config21 : nnet::dense_config {
    static const unsigned n_in = 32;
    static const unsigned n_out = 2;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned strategy = nnet::resource;
    static const unsigned reuse_factor = 16;
    static const unsigned n_zeros = 0;
    static const unsigned n_nonzeros = 64;
    static const unsigned multiplier_limit = DIV_ROUNDUP(n_in * n_out, reuse_factor) - n_zeros / reuse_factor;
    static const bool store_weights_in_bram = false;
    typedef model_default_t accum_t;
    typedef output_dense_bias_t bias_t;
    typedef output_dense_weight_t weight_t;
    typedef layer21_index index_t;
    template<class x_T, class y_T>
    using product = nnet::product::mult<x_T, y_T>;
};

// output_softmax
struct softmax_config23 : nnet::activ_config {
    static const unsigned n_in = 2;
    static const unsigned table_size = 1024;
    static const unsigned io_type = nnet::io_stream;
    static const unsigned reuse_factor = 16;
    static const unsigned axis = -1;
    static const nnet::softmax_implementation implementation = nnet::softmax_implementation::stable;
    typedef output_softmax_exp_table_t exp_table_t;
    typedef output_softmax_inv_table_t inv_table_t;
};


#endif

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Using Vitis on hls4ml #823

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Using Vitis on hls4ml #823

Uh oh!

zyt1024 Jun 28, 2023

Replies: 1 comment · 3 replies

Uh oh!

bo3z Jun 29, 2023 Maintainer

Uh oh!

zyt1024 Jun 30, 2023 Author

Uh oh!

bo3z Jun 30, 2023 Maintainer

Uh oh!

zyt1024 Jul 2, 2023 Author

zyt1024
Jun 28, 2023

Replies: 1 comment 3 replies

bo3z
Jun 29, 2023
Maintainer

zyt1024 Jun 30, 2023
Author

bo3z Jun 30, 2023
Maintainer

zyt1024 Jul 2, 2023
Author