Spancat with a custom transformer from pretrained

[martyna-spyra]

Hi, I'm getting decent results on span categorisation task using this transformer+spancat pipeline. Now I'd like to customise a pretrained transformer model (deberta-v3-base) by increasing the number of hidden layers, such as in the example here. What would be the best approach within the Spacy config system? The curated transformers library or a Thinc wrapper for pytorch? What would the implementation of the latter look like for a pretrained model with an additional dropout layer?