train/use YDF random forests via Spacy

[rwst]

YDF [1] is a C++ open source implementation of random forests. It may sound odd but we have good results using one-hot text encoding after lemmatization as input to random forests on text classification tasks. We used several Python scripts to show this. Now the idea is to call spacy/prodigy from our Kotlin Multiplatform UI to annotate/train/predict with YDF models. We think we can simply call "python spacy" from Kotlin but we don't have an idea how to best include YDF with spacy? How to package spacy+YDF for our customers? Is there a YDF plugin that can be installed with spacy?

[1] https://github.com/google/yggdrasil-decision-forests

Thanks,

[rmitsch]

Hi @rwst, that's not too surprising. For some application bag-of-words models can perform decently.

Now the idea is to call spacy/prodigy from our Kotlin Multiplatform UI to annotate/train/predict with YDF models.

I recommend wrapping YDF models in a custom spaCy component. These can be part of your spaCy pipeline.

[rwst]

Exactly the pointer I needed, thanks!