How to switch from CPU to GPU?

[Aleksi-learn-code]

How can I switch from CPU to GPU when it comes to calculations? can you help?

`using System;
using System.Windows.Forms;
using System.Threading;
using System.Threading.Tasks;
using OllamaSharp;
using OllamaSharp.Models.Chat;
using System.Xml.Linq;

namespace OllamaChatApp
{
public partial class Form1 : Form
{
private Chat _chat; // Entferne readonly

    public Form1()
    {
        InitializeComponent();
        InitializeOllama();
    }

    private void InitializeOllama()
    {
        var ollamaApiClient = new OllamaApiClient("http://localhost:11434");
        ollamaApiClient.SelectedModel = "llama3"; // Setzt das Modell
        _chat = new Chat(ollamaApiClient); // Modell ist über den Client festgelegt
    }

    private async void buttonsend_Click(object sender, EventArgs e)
    {
        if (string.IsNullOrWhiteSpace(richTextBoxinput.Text))
            return;

        string userInput = richTextBoxinput.Text;
        richTextBoxoutput.AppendText("You: " + userInput + Environment.NewLine);
        richTextBoxinput.Clear();

        // Hier kannst du die Rolle festlegen
        var systemMessage = "„You are a helpful assistant . They only speak German.";
        await _chat.SendAs(ChatRole.System, systemMessage, CancellationToken.None).StreamToEnd();
    
        richTextBoxoutput.AppendText("System: " + systemMessage + Environment.NewLine);

        richTextBoxoutput.AppendText("Assistant: ");
        buttonsend.Enabled = false;

        try
        {
            // Verwende die Methode Send für den Chat-Prozess
            var response = await _chat.Send(userInput, CancellationToken.None).StreamToEnd();
            richTextBoxoutput.AppendText(response);  // Zeigt die vollständige Antwort an
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
            MessageBox.Show($"Error: {ex.Message}", "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
        }
        finally
        {
            richTextBoxoutput.AppendText(Environment.NewLine + Environment.NewLine);
            buttonsend.Enabled = true;
        }
    }

}

}
`

[awaescher]

How can I switch from CPU to GPU when it comes to calculations?

What's the issue?

[Aleksi-learn-code]

So in the current version of the code, the calculation for the answer runs on the CPU, which unfortunately has an impact on speed. I had read at https://www.luisllamas.es/en/how-to-use-llama3-with-ollama-sharp/ that the GPU can also be used because it is faster. instead of the slow CPU. How can I implement this?

[awaescher]

This is something Ollama does by itself. Try to run Ollama and look how it deployed your model over CPU/GPU with ollama ps:

If you don't have enough VRAM, Ollama will not be able to fully load the model in GPU memory and with that the performance will drop significantly.

I guess this also happens if your GPU is not supported, you can check this here.