Skip to content

Modify reduce_dimensionality to use fit_transform #2347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jemather wants to merge 9 commits into from
Closed

Modify reduce_dimensionality to use fit_transform #2347

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
9f36539
Update _bertopic.py
jemather Apr 28, 2025
5bbac96
Update _bertopic.py
jemather May 13, 2025
896e38c
Update _bertopic.py
jemather May 13, 2025
70e2b4d
Update _bertopic.py
jemather May 13, 2025
933bca4
Update _bertopic.py
jemather May 13, 2025
a57a18d
Update _bertopic.py
jemather May 13, 2025
1a5ac77
Update _bertopic.py
jemather May 13, 2025
cc5c560
Update _bertopic.py
jemather May 13, 2025
7fdf885
Update _bertopic.py
jemather May 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions bertopic/_bertopic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3783,19 +3783,34 @@ def _reduce_dimensionality(
if partial_fit:
if hasattr(self.umap_model, "partial_fit"):
self.umap_model = self.umap_model.partial_fit(embeddings)
umap_embeddings = self.umap_model.transform(embeddings)
elif self.topic_representations_ is None:
self.umap_model.fit(embeddings)
umap_embeddings = self.umap_model.transform(embeddings)
else:
if hasattr(self.umap_model, "fit_transform"):
umap_embeddings = self.umap_model.fit_transform(embeddings)
else:
self.umap_model.fit(embeddings)
umap_embeddings = self.umap_model.transform(embeddings)

# Regular fit
else:
try:
# cuml umap needs y to be an numpy array
y = np.array(y) if y is not None else None
self.umap_model.fit(embeddings, y=y)
if hasattr(self.umap_model, "fit_transform"):
umap_embeddings = self.umap_model.fit_transform(embeddings, y=y)
else:
self.umap_model.fit(embeddings)
Copy link
Owner

@MaartenGr MaartenGr May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm missing the y=y here in the .fit. I can see it's there in the .fit_transform though.

Copy link
Author

@jemather jemather May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

Copy link
Owner

@MaartenGr MaartenGr May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw you moved the y=y to transform step but shouldn't it be used in the .fit instead?

Copy link
Author

@jemather jemather Jun 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it should. Sorry about that, I'll fix it.

Copy link
Owner

@MaartenGr MaartenGr Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In your latest commit, you moved the y parameter to the wrong line. It shouldn't be used in .transform but .fit instead.

umap_embeddings = self.umap_model.transform(embeddings, y=y)
except TypeError:
self.umap_model.fit(embeddings)
if hasattr(self.umap_model, "fit_transform"):
umap_embeddings = self.umap_model.fit_transform(embeddings)
else:
self.umap_model.fit(embeddings)
umap_embeddings = self.umap_model.transform(embeddings)

umap_embeddings = self.umap_model.transform(embeddings)
logger.info("Dimensionality - Completed \u2713")
return np.nan_to_num(umap_embeddings)

Expand Down
Loading