seed_phrase is ignored

[awlassche]

I tried out using seeded topic modelling, using pre-embedded documents:

model = KeyNMF(10, top_n=15, encoder=trf, vectorizer=CountVectorizer(stop_words=stop_words, min_df=50, max_df=0.85), seed_phrase='bog teater literatur')
model.fit(cleaned_corpus, embeddings=embeddings_matrix)

However, this gave me the same results as when I left out the seed_phrase:

model = KeyNMF(10, top_n=15, encoder=trf, vectorizer=CountVectorizer(stop_words=stop_words, min_df=50, max_df=0.85))
model.fit(cleaned_corpus, embeddings=embeddings_matrix)

Is it possible that the seed_phrase gets overwritten as soon as embeddings is defined?

EDIT: when I leave out embeddings=embeddings_matrix, I do get a different result. (But I want to load my pre-embedded documents.)

[x-tabdeveloping]

It's a bug! I'm fixing it!

[x-tabdeveloping]

Hi @awlassche what version of Turftopic are you using? I'm so far failing to reproduce this bug:

import numpy as np
from sklearn.datasets import fetch_20newsgroups

from turftopic import KeyNMF

corpus = fetch_20newsgroups(
    subset="all",
    remove=("headers", "footers", "quotes"),
).data
embeddings = np.load("_emb/20news_all-MiniLM.npy")

seeded_model = KeyNMF(
    10,
    seed_phrase="Is the death penalty moral?",
    random_state=42,
)
seeded_model.fit(corpus, embeddings=embeddings)

seedless_model = KeyNMF(
    10,
    random_state=42,
)
seedless_model.fit(corpus, embeddings=embeddings)


seeded_model.topics_df(top_k=10)
#   Topic ID                                    Highest Ranking
# 0        0  morality, objective, natural, subjective, soci...
# 1        1  armenian, armenians, genocide, armenia, turkis...
# 2        2  murder, punishment, death, penalty, innocent, ...
# 3        3  gun, guns, firearms, handgun, firearm, weapons...
# 4        4  israeli, israel, jews, palestinians, arab, jew...
# 5        5  god, sin, christians, christian, bible, sins, ...
# 6        6  crime, law, criminals, laws, crimes, criminal,...
# 7        7  moral, animals, killing, species, kill, religi...
# 8        8  immoral, morals, morally, behavior, homosexual...
# 9        9  bosnia, muslims, serbs, bosnian, holocaust, mu...
seedless_model.topics_df(top_k=10)
#   Topic ID                                    Highest Ranking
# 0        0  armenians, armenian, armenia, turks, turkish, ...
# 1        1  sale, price, shipping, offer, sell, mail, pric...
# 2        2  christians, christian, christianity, god, bibl...
# 3        3  encryption, chip, clipper, nsa, security, secu...
# 4        4  israel, israeli, jews, jewish, arab, palestini...
# 5        5  hockey, nhl, bruins, playoff, leafs, rangers, ...
# 6        6  windows, dos, file, files, software, program, ...
# 7        7  gun, guns, firearms, weapons, crime, law, amen...
# 8        8  monitor, card, ram, disk, mac, motherboard, vg...
# 9        9  braves, baseball, pitching, cubs, sox, mets, p...

[awlassche]

I am using version 0.14.1.

This is my code:

trf = SentenceTransformer('intfloat/multilingual-e5-large')

seeded_model = KeyNMF(10, top_n=15, encoder=trf, vectorizer=CountVectorizer(stop_words=stop_words, min_df=50, max_df=0.85), seed_phrase='Hvilke bøger er til salg?')
seeded_model.fit(cleaned_corpus, embeddings=embeddings_matrix)

seeded_model.print_topics()

Topic ID ┃ Highest Ranking                                                                                      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│        0 │ auction, offentlig, afholdt, bestaaende, auctionen, bortsolgt, tilhørende, sengeklæder,              │
│          │ næstkommende, begjæring                                                                              │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        1 │ varsel, anmelde, undertegnede, beviisliggjøre, indkaldes, afdøde, skifteforvalter, proclama,         │
│          │ vedkommende, fordringer                                                                              │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        2 │ thisted, forsendes, danmark, allernaadigst, privilegerede, brevposten, kongelig, lund, redigeres,    │
│          │ trykkes                                                                                              │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        3 │ undertegnede, odense, erholdes, tilkjøbs, henvende, billig, priser, erholde, kjøbenhavn, billige     │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        4 │ afholdes, auctioner, tilhørende, første, vedkommende, offentlige, auction, stillet, jfølge,          │
│          │ tilliggende                                                                                          │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        5 │ inden, bør, omkostninger, dage, sølv, betale, herreders, bemeldte, herredsfoged, uger                │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        6 │ bekjendtgjøres, efterretning, vedkommendes, aalborg, raadstue, vedkommende, ere, tillige, indtil,    │
│          │ kongelige                                                                                            │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        7 │ conditionerne, lysthavende, indbydes, eftersyn, thisted, auctionen, auctionsstedet, erfares,         │
│          │ begjæring, avertissements                                                                            │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        8 │ gaard, beliggende, gaarden, sogn, eiendommen, tilhørende, indrettet, eiendomme, ager, underhaanden   │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        9 │ førstkommende, odense, møde, kjøbmand, offentlig, kongelige, foretaget, vedkommendes, nielsen,       │
│          │ ifølge                                                                                                                                                                            

seedless_model = KeyNMF(10, top_n=15, encoder=trf, vectorizer=CountVectorizer(stop_words=stop_words, min_df=50, max_df=0.85))
seedless_model.fit(cleaned_corpus, embeddings=embeddings_matrix)

seedless_model.print_topics()

Topic ID ┃ Highest Ranking                                                                                      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│        0 │ auction, offentlig, afholdt, bestaaende, auctionen, bortsolgt, tilhørende, sengeklæder,              │
│          │ næstkommende, begjæring                                                                              │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        1 │ varsel, anmelde, undertegnede, beviisliggjøre, indkaldes, afdøde, skifteforvalter, proclama,         │
│          │ fordringer, creditorerne                                                                             │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        2 │ thisted, forsendes, danmark, allernaadigst, privilegerede, brevposten, kongelig, lund, redigeres,    │
│          │ trykkes                                                                                              │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        3 │ undertegnede, odense, erholdes, tilkjøbs, henvende, billig, erholde, priser, kjøbenhavn, gode        │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        4 │ afholdes, auctioner, tilhørende, første, vedkommende, offentlige, auction, stillet, jfølge,          │
│          │ tilliggende                                                                                          │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        5 │ inden, bør, omkostninger, dage, sølv, betale, herreders, bemeldte, herredsfoged, uger                │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        6 │ bekjendtgjøres, efterretning, vedkommendes, aalborg, raadstue, vedkommende, ere, tillige, indtil,    │
│          │ kongelige                                                                                            │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        7 │ conditionerne, lysthavende, indbydes, eftersyn, thisted, auctionen, auctionsstedet, erfares,         │
│          │ avertissements, begjæring                                                                            │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        8 │ gaard, beliggende, gaarden, sogn, eiendommen, tilhørende, indrettet, ager, underhaanden, eiendomme   │
├──────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│        9 │ førstkommende, odense, møde, kjøbmand, kongelige, offentlig, foretaget, vedkommendes, nielsen,       │
│          │ ifølge                                                                                               

[x-tabdeveloping]

@awlassche Can you try to check if the two models have the same parameters? It could very well be that the parameters are different but the top words do not change. This would include a) setting a random state by saying random_state=42 b) checking if the parameters are the same by saying:

print(np.allclose(seeded_model.components_, seedless_model.components_))

This should print True if the parameters are the same.

[awlassche]

It prints False (after setting the random state with both models).

[x-tabdeveloping]

So it seems that the parameters are not the same, it just so happens that the top words do not change.
I suspect that this has something to do with how E5 uses query: and passage: prefixes.
You said you tried it with the historic model too, does it result in the same thing?