gpu nvidia wsl

Running Ollama with GPU support inside Docker on Windows 10 (WSL2) requires a specific "bridge" between your Windows hardware and the Linux container.

Since you are on **Windows 10**, ensure you are on version **21H2 or later** (Build 19044+) to have stable GPU paravirtualization support.

---

### Step 1: Install the Windows NVIDIA Driver
You do **not** install Linux drivers inside WSL. Instead, you install the standard Game Ready or Studio drivers on Windows.
* Download the latest driver from [nvidia.com/drivers](https://www.nvidia.com/Download/index.aspx).
* **Verify:** Open PowerShell and type `nvidia-smi`. You should see your GPU details and the CUDA version.

---

### Step 2: Install the NVIDIA Container Toolkit (Inside WSL)
Docker needs this toolkit to "see" your GPU through the WSL layer. Open your WSL terminal (e.g., Ubuntu) and run the following commands:

1.  **Configure the repository:**
    ```bash
    curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
        sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
        sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
    ```

2.  **Install the toolkit:**
    ```bash
    sudo apt-get update
    sudo apt-get install -y nvidia-container-toolkit
    ```

3.  **Restart Docker Desktop:** Right-click the Docker icon in your Windows system tray and select **Restart**. This ensures the new runtime is recognized.

---

### Step 3: Run Ollama with GPU Flags
Now you can launch the container. The `--gpus all` flag is the "magic" part that enables hardware acceleration.

```bash
docker run -d \
  --gpus=all \
  -v ollama:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  ollama/ollama
```

---

### Step 4: Verify GPU Usage
To make sure Ollama is actually using your GPU and not your CPU, run a model and check the logs:

1.  **Run a model:**
    ```bash
    docker exec -it ollama ollama run llama3.2
    ```
2.  **Check for GPU confirmation:**
    While the model is running, open a separate PowerShell window on Windows and run `nvidia-smi`. You should see a process named `ollama` or `ollama_llama_server` consuming VRAM.

---

### Troubleshooting Tips for Windows 10
* **WSL Version:** If it fails, run `wsl -l -v` in PowerShell. Ensure your distro says version **2**. If it says 1, run `wsl --set-version <distro_name> 2`.
* **Docker Settings:** In Docker Desktop settings, go to **General** and ensure "Use the WSL 2 based engine" is checked. Under **Resources > WSL Integration**, ensure your specific distro (e.g., Ubuntu) is toggled **On**.
* **Missing `libcuda.so`:** If the container complains about missing libraries, run `wsl --update` in PowerShell to ensure your WSL kernel is current.