PS C:\Users\user> python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from optimum.onnxruntime import ORTModelForSequenceClassification
>>> from transformers import AutoTokenizer
... )
config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 829/829 [00:00<?, ?B/s]
C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\huggingface_hub\file_download.py:148: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\user\.cache\huggittps://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)
pytorch_model.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 17.6M/17.6M [00:00<00:00, 27.8MB/s]
vocab.txt: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 6.94MB/s]
tokenizer.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 466k/466k [00:00<00:00, 15.0MB/s]
special_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 112/112 [00:00<?, ?B/s]
Using the export variant default. Available variants are:
    - default: The default ONNX variant.
Using framework PyTorch: 2.2.2+cpu
Overriding 1 configuration item(s)
        - use_cache -> False
2024-04-08 17:55:23.6758212 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

*************** EP Error ***************
EP Error D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page  (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements),  make sure they're in the PATH, and that your GPU is supported.
 when using ['CUDAExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************
2024-04-08 17:55:23.7325638 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Traceback (most recent call last):
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page  (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements),  make sure they're in the PATH, and that your GPU is supported.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\optimum\onnxruntime\modeling_ort.py", line 663, in from_pretrained
    return super().from_pretrained(
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\optimum\modeling_base.py", line 401, in from_pretrained
    return from_pretrained_method(
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\optimum\onnxruntime\modeling_ort.py", line 545, in _from_transformers
    return cls._export(
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\optimum\onnxruntime\modeling_ort.py", line 604, in _export
    return cls._from_pretrained(
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\optimum\onnxruntime\modeling_ort.py", line 511, in _from_pretrained
    model = ORTModel.load_model(
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages\optimum\onnxruntime\modeling_ort.py", line 374, in load_model
    return ort.InferenceSession(

>>>
>>> tokenizer = AutoTokenizer.from_pretrained("philschmid/tiny-bert-sst2-distilled")
>>> inputs = tokenizer("expectations were low, actual enjoyment was high", return_tensors="pt", padding=True)
>>>
>>> outputs = ort_model(**inputs)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ort_model' is not defined
Terminate batch job (Y/N)? y
Looking in indexes: https://download.pytorch.org/whl/cu118
Requirement already satisfied: torchvision in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (0.17.2)
Collecting torchaudio
  Downloading https://download.pytorch.org/whl/cu118/torchaudio-2.2.2%2Bcu118-cp310-cp310-win_amd64.whl (4.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.0/4.0 MB 12.1 MB/s eta 0:00:00
Requirement already satisfied: networkx in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torch) (3.3)
Requirement already satisfied: fsspec in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torch) (2024.2.0)
Requirement already satisfied: sympy in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torch) (1.12)
Requirement already satisfied: jinja2 in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torch) (3.1.3)
Requirement already satisfied: filelock in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torch) (3.13.3)
Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torch) (4.11.0)
Requirement already satisfied: numpy in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torchvision) (1.26.4)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from torchvision) (10.3.0)
Collecting torch
  Downloading https://download.pytorch.org/whl/cu118/torch-2.2.2%2Bcu118-cp310-cp310-win_amd64.whl (2704.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.7/2.7 GB ? eta 0:00:00
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from jinja2->torch) (2.1.5)
Requirement already satisfied: mpmath>=0.19 in c:\users\user\.pyenv\pyenv-win\versions\3.10.6\lib\site-packages (from sympy->torch) (1.3.0)
Installing collected packages: torch, torchaudio
  Attempting uninstall: torch
    Found existing installation: torch 2.2.2
    Uninstalling torch-2.2.2:
      Successfully uninstalled torch-2.2.2
Successfully installed torch-2.2.2+cu118 torchaudio-2.2.2+cu118

[notice] A new release of pip available: 22.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
PS C:\Users\user> \
\ : The term '\' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ \
+ ~
    + CategoryInfo          : ObjectNotFound: (\:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS C:\Users\user>
PS C:\Users\user>
PS C:\Users\user>
PS C:\Users\user>
PS C:\Users\user> python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from optimum.onnxruntime import ORTModelForSequenceClassification
>>> from transformers import AutoTokenizer
>>>
>>> ort_model = ORTModelForSequenceClassification.from_pretrained(
...   "philschmid/tiny-bert-sst2-distilled",
...   export=True,
...   provider="CUDAExecutionProvider",
... )
Framework not specified. Using pt to export the model.
Using the export variant default. Available variants are:
    - default: The default ONNX variant.
Using framework PyTorch: 2.2.2+cu118
Overriding 1 configuration item(s)
        - use_cache -> False
2024-04-08 18:05:50.9940248 [W:onnxruntime:, session_state.cc:1166 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-04-08 18:05:51.0097430 [W:onnxruntime:, session_state.cc:1168 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
>>>
>>> tokenizer = AutoTokenizer.from_pretrained("philschmid/tiny-bert-sst2-distilled")
>>> inputs = tokenizer("expectations were low, actual enjoyment was high", return_tensors="pt", padding=True)
>>>
>>> outputs = ort_model(**inputs)
>>> assert ort_model.providers == ["CUDAExecutionProvider", "CPUExecutionProvider"]
>>> outputs
SequenceClassifierOutput(loss=None, logits=tensor([[ 0.2788, -0.0751]], device='cuda:0'), hidden_states=None, attentions=None)
>>> ort_model.providers
['CUDAExecutionProvider', 'CPUExecutionProvider']
>>>