-
Notifications
You must be signed in to change notification settings - Fork 22
Open
Description
Small question about the biais header definition of nnlib-hifi4.
My original model using pytorch is defining this weigths and biases like this:
weight_ih_l -> (W_ir|W_iz|W_in)
weight_hh_l -> (W_hr|W_hz|W_hn)
bias_ih_l -> (b_ir|b_iz|b_in)
bias_hh_l -> (b_hr|b_hz|b_hn)
I can see that some c implementation are optimizing memory and computation by merging some bias like this:
b_r -> b_ir + b_hr
b_z -> b_iz + b_hz
This is logical after reviewing the original GRU formula (https://pytorch.org/docs/stable/generated/torch.nn.GRU.html)
So I am expecting 4 fields for the biais header:
b_r, b_z, b_hn and b_in.
But the biais header implementation for nnlib is only using 3 biases:
typedef struct _xa_nnlib_gru_biases_t
{
void *b_z; xa_nnlib_shape_t shape_b_z;
void *b_r; xa_nnlib_shape_t shape_b_r;
void *b_h; xa_nnlib_shape_t shape_b_h;
} xa_nnlib_gru_biases_t;
How can I obtain this b_h? What did I miss?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels