Skip to content

Fix Missing Import#187

Open
alex-jw-brooks wants to merge 1 commit intofoundation-model-stack:mainfrom
alex-jw-brooks:fix_import
Open

Fix Missing Import#187
alex-jw-brooks wants to merge 1 commit intofoundation-model-stack:mainfrom
alex-jw-brooks:fix_import

Conversation

@alex-jw-brooks
Copy link
Copy Markdown
Contributor

I noticed that the torch functional API is used when sampling is enabled in paged attention, but not imported - probably from copying from FMS and tweaking the generate implementation.

Adds it to prevent import errors if sampling is turned off.

Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Copy link
Copy Markdown
Collaborator

@matthew-pisano matthew-pisano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed this as well, looks good.

@matthew-pisano
Copy link
Copy Markdown
Collaborator

@Ssukriti @JRosenkranz Any issue with merging this in?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants