Oobabooga Error Dump

2023-09-13 16:26:19 INFO:Loading TheBloke_MythoMix-L2-13B-GPTQ_gptq-4bit-32g-actorder_True...
2023-09-13 16:26:29 ERROR:Failed to load the model.
Traceback (most recent call last):
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\utils\cpp_extension.py", line 1893, in _run_ninja_build
    subprocess.run(
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\text-generation-webui\modules\ui_model_menu.py", line 194, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(shared.model_name, loader)
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\text-generation-webui\modules\models.py", line 77, in load_model
    output = load_func_map[loader](model_name)
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\text-generation-webui\modules\models.py", line 335, in ExLlamav2_loader
    from modules.exllamav2 import Exllamav2Model
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\text-generation-webui\modules\exllamav2.py", line 5, in <module>
    from exllamav2 import (
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\__init__.py", line 3, in <module>
    from exllamav2.model import ExLlamaV2
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\model.py", line 12, in <module>
    from exllamav2.linear import ExLlamaV2Linear
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\linear.py", line 4, in <module>
    from exllamav2 import ext
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\ext.py", line 118, in <module>
    exllamav2_ext = load \
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\utils\cpp_extension.py", line 1284, in load
    return _jit_compile(
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\utils\cpp_extension.py", line 1509, in _jit_compile
    _write_ninja_file_and_build_library(
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\utils\cpp_extension.py", line 1624, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\utils\cpp_extension.py", line 1909, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'exllamav2_ext': [1/6] cl /showIncludes -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc /Ox -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\ext.cpp /Foext.o
FAILED: ext.o
cl /showIncludes -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc /Ox -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\ext.cpp /Foext.o
Microsoft (R) C/C++ Optimizing Compiler Version 19.10.25019 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\c10/macros/Macros.h(3): fatal error C1083: Cannot open include file: 'cassert': No such file or directory
[2/6] cl /showIncludes -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc /Ox -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cpp\quantize_func.cpp /Foquantize_func.o
FAILED: quantize_func.o
cl /showIncludes -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc /Ox -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cpp\quantize_func.cpp /Foquantize_func.o
Microsoft (R) C/C++ Optimizing Compiler Version 19.10.25019 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\c10/macros/Macros.h(3): fatal error C1083: Cannot open include file: 'cassert': No such file or directory
[3/6] C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\bin\nvcc --generate-dependencies-with-compile --dependency-output rope.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -lineinfo -O3 -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\rope.cu -o rope.cuda.o
FAILED: rope.cuda.o
C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\bin\nvcc --generate-dependencies-with-compile --dependency-output rope.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -lineinfo -O3 -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\rope.cu -o rope.cuda.o
C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\rope.cu(39): error: expected a "("
          detected during instantiation of "void rope_cuda_kernel<use_half2>(half *, const half *, const half *, int, int, int, int, const uint32_t *) [with use_half2=true]"
(95): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\rope.cu(53): error: expected a "("
          detected during instantiation of "void rope_cuda_kernel<use_half2>(half *, const half *, const half *, int, int, int, int, const uint32_t *) [with use_half2=true]"
(95): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\rope.cu(39): error: expected a "("
          detected during instantiation of "void rope_cuda_kernel<use_half2>(half *, const half *, const half *, int, int, int, int, const uint32_t *) [with use_half2=false]"
(96): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\rope.cu(53): error: expected a "("
          detected during instantiation of "void rope_cuda_kernel<use_half2>(half *, const half *, const half *, int, int, int, int, const uint32_t *) [with use_half2=false]"
(96): here

4 errors detected in the compilation of "C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/exllamav2/exllamav2_ext/cuda/rope.cu".
rope.cu
[4/6] C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\bin\nvcc --generate-dependencies-with-compile --dependency-output pack_tensor.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -lineinfo -O3 -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu -o pack_tensor.cuda.o
FAILED: pack_tensor.cuda.o
C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\bin\nvcc --generate-dependencies-with-compile --dependency-output pack_tensor.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -lineinfo -O3 -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu -o pack_tensor.cuda.o
C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(132): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=2]"
(261): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(146): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=2]"
(261): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(168): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=2]"
(261): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(182): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=2]"
(261): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(210): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=2]"
(261): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(232): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=2]"
(261): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(132): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=3]"
(262): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(146): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=3]"
(262): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(168): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=3]"
(262): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(182): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=3]"
(262): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(210): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=3]"
(262): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(232): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=3]"
(262): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(132): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=4]"
(263): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(146): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=4]"
(263): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(168): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=4]"
(263): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(182): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=4]"
(263): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(210): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=4]"
(263): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(232): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=4]"
(263): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(132): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=5]"
(264): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(146): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=5]"
(264): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(168): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=5]"
(264): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(182): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=5]"
(264): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(210): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=5]"
(264): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(232): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=5]"
(264): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(132): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=6]"
(265): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(146): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=6]"
(265): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(168): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=6]"
(265): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(182): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=6]"
(265): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(210): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=6]"
(265): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(232): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=6]"
(265): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(132): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=8]"
(266): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(146): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=8]"
(266): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(168): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=8]"
(266): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(182): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=8]"
(266): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(210): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=8]"
(266): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\pack_tensor.cu(232): error: expected a "("
          detected during instantiation of "void pack_columns_kernel<bits>(const uint16_t *, uint32_t *, int, int) [with bits=8]"
(266): here

36 errors detected in the compilation of "C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/exllamav2/exllamav2_ext/cuda/pack_tensor.cu".
pack_tensor.cu
[5/6] C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\bin\nvcc --generate-dependencies-with-compile --dependency-output q_mlp.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -lineinfo -O3 -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\q_mlp.cu -o q_mlp.cuda.o
FAILED: q_mlp.cuda.o
C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\bin\nvcc --generate-dependencies-with-compile --dependency-output q_mlp.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\TH -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\torch\include\THC -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\include -IC:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -lineinfo -O3 -c C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\q_mlp.cu -o q_mlp.cuda.o
C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/torch/include\c10/macros/Macros.h(138): warning C4067: unexpected tokens following preprocessor directive - expected a newline
C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/torch/include\c10/macros/Macros.h(138): warning C4067: unexpected tokens following preprocessor directive - expected a newline
C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/include\thrust/detail/config/cpp_dialect.h:118: warning: Thrust requires at least MSVC 2019 (19.20/16.0/14.20). MSVC 2017 is deprecated but still supported. MSVC 2017 support will be removed in a future release. Define THRUST_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/torch/include\c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/torch/include\c10/core/TensorImpl.h(77): here

C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/torch/include\c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=true, <unnamed>=0]"
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=true, <unnamed>=0]"
C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/torch/include\c10/core/TensorImpl.h(2327): here
            instantiation of "__nv_bool c10::TensorImpl::SetDimsTemplate(c10::ArrayRef<T>) [with T=int64_t, <unnamed>=void]"
C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/torch/include\c10/core/TensorImpl.h(2337): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\q_mlp.cu(61): error: expected a "("
          detected during instantiation of "void silu_mul_kernel<use_half2>(half *, const half *, int, int) [with use_half2=true]"
(95): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\q_mlp.cu(67): error: expected a "("
          detected during instantiation of "void silu_mul_kernel<use_half2>(half *, const half *, int, int) [with use_half2=true]"
(95): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\q_mlp.cu(61): error: expected a "("
          detected during instantiation of "void silu_mul_kernel<use_half2>(half *, const half *, int, int) [with use_half2=false]"
(96): here

C:\Users\Zugzwang\Desktop\oobabooga_windows\installer_files\env\lib\site-packages\exllamav2\exllamav2_ext\cuda\q_mlp.cu(67): error: expected a "("
          detected during instantiation of "void silu_mul_kernel<use_half2>(half *, const half *, int, int) [with use_half2=false]"
(96): here

4 errors detected in the compilation of "C:/Users/Zugzwang/Desktop/oobabooga_windows/installer_files/env/lib/site-packages/exllamav2/exllamav2_ext/cuda/q_mlp.cu".
q_mlp.cu
ninja: build stopped: subcommand failed.