Divide Overflow when subdividing groups

[DobromirM]

A division overflow can occur when subdividing groups multiple times.

Example:

from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import CountVectorizer
from turftopic import KeyNMF


def split_groups(groups, n):
    if n == 0:
        return

    for group in groups:
        group.divide(5)
        split_groups(group, n - 1)


if __name__ == '__main__':
    corpus = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes')).data

    corpus = corpus[0:250]

    vectorizer = CountVectorizer(min_df=5, max_df=0.8, stop_words='english')

    model = KeyNMF(n_components=5, vectorizer=vectorizer)

    topic_data = model.prepare_topic_data(corpus)

    split_groups(topic_data.hierarchy, 3)

Error:

/.venv1/lib/python3.10/site-packages/turftopic/models/wnmf.py:40: RuntimeWarning:

overflow encountered in divide

[DobromirM]

Updating:

turftopic/turftopic/models/wnmf.py

Line 38 in dc42d80

delta = numerator

to delta = numerator.astype(dtype=np.float128) seems to fix the issue.

[x-tabdeveloping]

Hey @DobromirM ! Thanks for looking into this and suggesting a fix. Since this only results in a warning, and I suspect it only makes a difference for words that are not significant for the topics, I did not consider it a serious issue.

If you think it's annoying, or you encounter problems down the line, you are more than welcome to submit a PR with the change :D