Skip to content

Conversation

@giladgd
Copy link
Member

@giladgd giladgd commented Oct 27, 2024

Description of change

  • feat: chat session response prefix
  • feat: improve context shift strategy
  • feat: use RAM and swap sizes in memory usage estimations
  • feat(inspect gguf command): print a single key flag
  • feat: faster building from source
  • fix: Electron crash with some models on macOS when not using Metal (via fix: use vm_allocate to allocate CPU backend buffer on macOS ggml-org/llama.cpp#9875)
  • fix: adapt to llama.cpp breaking changes
  • fix: improve CPU compatibility score
  • docs: mention the need to treat node-llama-cpp as an external module in Electron
  • docs: sitemap fixes
  • docs(README): make the logo be a link
  • chore: update issue config

Fixes #361

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run format to apply eslint formatting
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change
  • The new commits and pull request title follow conventions explained in pull request guidelines (PRs that do not follow this convention will not be merged)

@giladgd giladgd requested a review from ido-pluto October 27, 2024 00:04
@giladgd giladgd self-assigned this Oct 27, 2024
Copy link
Contributor

@ido-pluto ido-pluto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@giladgd giladgd merged commit ea12dc5 into master Oct 29, 2024
14 checks passed
@giladgd giladgd deleted the gilad/chatResponsePrefix branch October 29, 2024 23:54
@github-actions
Copy link

🎉 This PR is included in version 3.2.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Electron sample app crash on Mac with speciffic model Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf

3 participants