How to do topic modeling before upserv? (preprocessing)

[themeaningofmeaning]

Has anyone figured out a way to do topic modeling for adding metadata on a document before up serving. This is typically added programmatically as follows:
`# Generate embeddings for the chunk
embedding = model.encode([chunk])

# Perform topic modeling for the chunk
**topics, _ = topic_model.fit_transform([chunk])**

# Prepare data for upsertion
data_to_upsert = {
    "text": chunk,
    "embedding": embedding[0],
    **"topic": topics[0]**
}

`

In my working flow below, which chunks document data and then embeds into Pincone, this would ideally be a single HuggingFace block somewhere before each document chunk embedding unserved to pinecone:

[HenryHengZJ]

dont think currently there is a way for that. but we are adding custom document loader which is a piece of JS code where you can basically customize to your needs

[themeaningofmeaning]

Yeah, it would be great to have a way of importing dependencies and writing functions (i.e. python, etc.) so we could do things like topic modeling with a few lines of code. Not sure if that is in the scope for this Flowise roadmap, but that would be a game changer for sure.