D:\Projects\workspace>docker run --gpus all -it --rm -e CUDA_VISIBLE_DEVICES=0 --network host -v "D:/Projects/workspace" balacoon/tts_server:0.1 balacoon_tts_server 0.0.0.0 3333 16 16 /workspace/en_us_cmartic_jets_gpu.addon ============================= == Triton Inference Server == ============================= NVIDIA Release 22.08 (build 42766143) Triton Server Version 2.25.0 Copyright (c) 2018-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved. This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license I0608 04:11:03.406205 1 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x304600000' with size 268435456 I0608 04:11:03.406607 1 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864 I0608 04:11:03.409246 1 server.cc:561] +------------------+------+ | Repository Agent | Path | +------------------+------+ +------------------+------+ I0608 04:11:03.409316 1 server.cc:588] +---------+------+--------+ | Backend | Path | Config | +---------+------+--------+ +---------+------+--------+ I0608 04:11:03.409388 1 server.cc:631] +-------+---------+--------+ | Model | Version | Status | +-------+---------+--------+ I0608 04:11:03.445367 1 metrics.cc:650] Collecting metrics for GPU 0: NVIDIA GeForce GTX 1060 I0608 04:11:03.445744 1 tritonserver.cc:2214] +----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Option | Value |+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+| server_id | triton || server_version | 2.25.0 || server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_te || | nsor_data statistics trace || model_repository_path[0] | /opt/models || model_control_mode | MODE_EXPLICIT || strict_model_config | 1 || rate_limit | OFF || pinned_memory_pool_byte_size | 268435456 || cuda_memory_pool_byte_size{0} | 67108864 || response_cache_byte_size | 0 || min_supported_compute_capability | 6.0 || strict_readiness | 1 || exit_timeout | 30 |+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ terminate called after throwing an instance of 'std::invalid_argument' what(): pronunciation generation was not initialized, check addons passed PS D:\Projects\workspace> .\test.bat D:\Projects\workspace>docker run --gpus all -it --rm -e CUDA_VISIBLE_DEVICES=0 --network host -v "D:\Projects\workspace:/workspace" balacoon/tts_server:0.1 balacoon_tts_server 0.0.0.0 35565 16 16 /workspace/en_us_cmartic_jets_gpu.addon ============================= == Triton Inference Server == ============================= NVIDIA Release 22.08 (build 42766143) Triton Server Version 2.25.0 Copyright (c) 2018-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved. This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license I0608 04:11:29.428930 1 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x304600000' with size 268435456 I0608 04:11:29.429098 1 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864 I0608 04:11:29.432609 1 server.cc:561] +------------------+------+ | Repository Agent | Path | +------------------+------+ +------------------+------+ I0608 04:11:29.432676 1 server.cc:588] +---------+------+--------+ | Backend | Path | Config | +---------+------+--------+ +---------+------+--------+ I0608 04:11:29.432730 1 server.cc:631] +-------+---------+--------+ | Model | Version | Status | +-------+---------+--------+ +-------+---------+--------+ I0608 04:11:29.533630 1 metrics.cc:650] Collecting metrics for GPU 0: NVIDIA GeForce GTX 1060 I0608 04:11:29.533963 1 tritonserver.cc:2214] +----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Option | Value |+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+| server_id | triton || server_version | 2.25.0 || server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_te || | nsor_data statistics trace || model_repository_path[0] | /opt/models || model_control_mode | MODE_EXPLICIT || strict_model_config | 1 || rate_limit | OFF || pinned_memory_pool_byte_size | 268435456 || cuda_memory_pool_byte_size{0} | 67108864 || response_cache_byte_size | 0 || min_supported_compute_capability | 6.0 || strict_readiness | 1 || exit_timeout | 30 |+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ W0608 04:11:30.538457 1 metrics.cc:426] Unable to get power limit for GPU 0. Status:Success, value:0.000000 W0608 04:11:31.538758 1 metrics.cc:426] Unable to get power limit for GPU 0. Status:Success, value:0.000000 W0608 04:11:32.542208 1 metrics.cc:426] Unable to get power limit for GPU 0. Status:Success, value:0.000000 I0608 04:11:32.968885 1 model_lifecycle.cc:459] loading: tts_encoder:1 I0608 04:11:33.454696 1 libtorch.cc:1983] TRITONBACKEND_Initialize: pytorch I0608 04:11:33.454771 1 libtorch.cc:1993] Triton TRITONBACKEND API version: 1.10 I0608 04:11:33.454781 1 libtorch.cc:1999] 'pytorch' TRITONBACKEND API version: 1.10 I0608 04:11:33.454803 1 libtorch.cc:2032] TRITONBACKEND_ModelInitialize: tts_encoder (version 1) I0608 04:11:33.456529 1 libtorch.cc:313] Optimized execution is enabled for model instance 'tts_encoder' I0608 04:11:33.456754 1 libtorch.cc:332] Cache Cleaning is disabled for model instance 'tts_encoder' I0608 04:11:33.456834 1 libtorch.cc:349] Inference Mode is enabled for model instance 'tts_encoder' W0608 04:11:33.457320 1 libtorch.cc:454] NvFuser is disabled for model instance 'tts_encoder' I0608 04:11:33.475893 1 libtorch.cc:2076] TRITONBACKEND_ModelInstanceInitialize: tts_encoder (GPU device 0) I0608 04:11:34.961163 1 model_lifecycle.cc:693] successfully loaded 'tts_encoder' version 1 I0608 04:11:35.080046 1 model_lifecycle.cc:459] loading: tts_decoder:1 I0608 04:11:35.080350 1 libtorch.cc:2032] TRITONBACKEND_ModelInitialize: tts_decoder (version 1) I0608 04:11:35.081387 1 libtorch.cc:313] Optimized execution is enabled for model instance 'tts_decoder' I0608 04:11:35.081456 1 libtorch.cc:332] Cache Cleaning is disabled for model instance 'tts_decoder' I0608 04:11:35.081525 1 libtorch.cc:349] Inference Mode is enabled for model instance 'tts_decoder' W0608 04:11:35.081567 1 libtorch.cc:454] NvFuser is disabled for model instance 'tts_decoder' I0608 04:11:35.097893 1 libtorch.cc:2076] TRITONBACKEND_ModelInstanceInitialize: tts_decoder (GPU device 0) [W cuda_graph_fuser.h:17] Warning: RegisterCudaFuseGraph::registerPass() is deprecated. Please use torch::jit::fuser::cuda::setEnabled(). (function registerPass) I0608 04:11:37.504067 1 model_lifecycle.cc:693] successfully loaded 'tts_decoder' version 1 [INFO:/opt/balacoon_tts/build_server_on_docker/_deps/balacoon_neural-src/src/lib/triton_metrics_service.cc:128] 0.0.0.0:8002: metrics server