API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Mar 2nd, 2022
26
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 32.38 KB
Copied copy raw download clone embed print report
  1. ==========================
  2. === Riva Speech Skills ===
  3. ==========================
  4.  
  5. NVIDIA Release 21.12 (build 30304767)
  6.  
  7. Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.
  8.  
  9. Various files include modifications (c) NVIDIA CORPORATION. All rights reserved.
  10. NVIDIA modifications are covered by the license terms that apply to the underlying
  11. project or file.
  12.  
  13. CUDA Capability Major/Minor version number: 7.5
  14. Loading models from s3://tarteel-models/trtis-repo/1.8.0b0/75
  15. Loading model: CnLg-SpeUni256-EATL1300-streaming
  16. > Riva waiting for Triton server to load all models...retrying in 1 second
  17. > Riva waiting for Triton server to load all models...retrying in 1 second
  18. I0302 15:25:13.508607 78 metrics.cc:290] Collecting metrics for GPU 0: Tesla T4
  19. I0302 15:25:13.515606 78 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so
  20. I0302 15:25:13.550529 78 onnxruntime.cc:1970] TRITONBACKEND_Initialize: onnxruntime
  21. I0302 15:25:13.550561 78 onnxruntime.cc:1980] Triton TRITONBACKEND API version: 1.4
  22. I0302 15:25:13.550575 78 onnxruntime.cc:1986] 'onnxruntime' TRITONBACKEND API version: 1.4
  23. I0302 15:25:13.753683 78 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f6914000000' with size 268435456
  24. I0302 15:25:13.755203 78 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
  25. I0302 15:25:14.002457 78 backend_factory.h:45] Create TritonBackendFactory
  26. I0302 15:25:14.003820 78 plan_backend_factory.cc:49] Create PlanBackendFactory
  27. I0302 15:25:14.003842 78 plan_backend_factory.cc:56] Registering TensorRT Plugins
  28. I0302 15:25:14.003897 78 logging.cc:52] Registered plugin creator - ::GridAnchor_TRT version 1
  29. I0302 15:25:14.003917 78 logging.cc:52] Registered plugin creator - ::GridAnchorRect_TRT version 1
  30. I0302 15:25:14.003947 78 logging.cc:52] Registered plugin creator - ::NMS_TRT version 1
  31. I0302 15:25:14.003964 78 logging.cc:52] Registered plugin creator - ::Reorg_TRT version 1
  32. I0302 15:25:14.003976 78 logging.cc:52] Registered plugin creator - ::Region_TRT version 1
  33. I0302 15:25:14.003992 78 logging.cc:52] Registered plugin creator - ::Clip_TRT version 1
  34. I0302 15:25:14.004007 78 logging.cc:52] Registered plugin creator - ::LReLU_TRT version 1
  35. I0302 15:25:14.004018 78 logging.cc:52] Registered plugin creator - ::PriorBox_TRT version 1
  36. I0302 15:25:14.004036 78 logging.cc:52] Registered plugin creator - ::Normalize_TRT version 1
  37. I0302 15:25:14.004049 78 logging.cc:52] Registered plugin creator - ::ScatterND version 1
  38. I0302 15:25:14.004062 78 logging.cc:52] Registered plugin creator - ::RPROI_TRT version 1
  39. I0302 15:25:14.004076 78 logging.cc:52] Registered plugin creator - ::BatchedNMS_TRT version 1
  40. I0302 15:25:14.004086 78 logging.cc:52] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
  41. I0302 15:25:14.004094 78 logging.cc:52] Registered plugin creator - ::FlattenConcat_TRT version 1
  42. I0302 15:25:14.004106 78 logging.cc:52] Registered plugin creator - ::CropAndResize version 1
  43. I0302 15:25:14.004117 78 logging.cc:52] Registered plugin creator - ::DetectionLayer_TRT version 1
  44. I0302 15:25:14.004127 78 logging.cc:52] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
  45. I0302 15:25:14.004146 78 logging.cc:52] Registered plugin creator - ::EfficientNMS_TRT version 1
  46. I0302 15:25:14.004162 78 logging.cc:52] Registered plugin creator - ::Proposal version 1
  47. I0302 15:25:14.004184 78 logging.cc:52] Registered plugin creator - ::ProposalLayer_TRT version 1
  48. I0302 15:25:14.004199 78 logging.cc:52] Registered plugin creator - ::PyramidROIAlign_TRT version 1
  49. I0302 15:25:14.004212 78 logging.cc:52] Registered plugin creator - ::ResizeNearest_TRT version 1
  50. I0302 15:25:14.004228 78 logging.cc:52] Registered plugin creator - ::Split version 1
  51. I0302 15:25:14.004243 78 logging.cc:52] Registered plugin creator - ::SpecialSlice_TRT version 1
  52. I0302 15:25:14.004261 78 logging.cc:52] Registered plugin creator - ::InstanceNormalization_TRT version 1
  53. I0302 15:25:14.004272 78 ensemble_backend_factory.cc:47] Create EnsembleBackendFactory
  54. > Riva waiting for Triton server to load all models...retrying in 1 second
  55. W0302 15:25:14.336722 78 autofill.cc:243] Proceeding with simple config for now
  56. I0302 15:25:14.336747 78 model_config_utils.cc:637] autofilled config: name: "CnLg-SpeUni256-EATL1300-streaming"
  57. platform: "ensemble"
  58. max_batch_size: 64
  59. input {
  60. name: "AUDIO_SIGNAL"
  61. data_type: TYPE_FP32
  62. dims: -1
  63. }
  64. input {
  65. name: "SAMPLE_RATE"
  66. data_type: TYPE_UINT32
  67. dims: 1
  68. }
  69. input {
  70. name: "END_FLAG"
  71. data_type: TYPE_UINT32
  72. dims: 1
  73. }
  74. input {
  75. name: "CUSTOM_CONFIGURATION"
  76. data_type: TYPE_STRING
  77. dims: -1
  78. dims: 2
  79. }
  80. output {
  81. name: "FINAL_TRANSCRIPTS"
  82. data_type: TYPE_STRING
  83. dims: -1
  84. }
  85. output {
  86. name: "FINAL_TRANSCRIPTS_SCORE"
  87. data_type: TYPE_FP32
  88. dims: -1
  89. }
  90. output {
  91. name: "FINAL_WORDS_START_END"
  92. data_type: TYPE_INT32
  93. dims: -1
  94. dims: 2
  95. }
  96. output {
  97. name: "PARTIAL_TRANSCRIPTS"
  98. data_type: TYPE_STRING
  99. dims: -1
  100. }
  101. output {
  102. name: "PARTIAL_TRANSCRIPTS_STABILITY"
  103. data_type: TYPE_FP32
  104. dims: -1
  105. }
  106. output {
  107. name: "PARTIAL_WORDS_START_END"
  108. data_type: TYPE_INT32
  109. dims: -1
  110. dims: 2
  111. }
  112. output {
  113. name: "AUDIO_PROCESSED"
  114. data_type: TYPE_FP32
  115. dims: 1
  116. }
  117. parameters {
  118. key: "chunk_size"
  119. value {
  120. string_value: "1.2"
  121. }
  122. }
  123. parameters {
  124. key: "compute_timestamps"
  125. value {
  126. string_value: "True"
  127. }
  128. }
  129. parameters {
  130. key: "decoder_type"
  131. value {
  132. string_value: "greedy"
  133. }
  134. }
  135. parameters {
  136. key: "language_code"
  137. value {
  138. string_value: "ar-BH"
  139. }
  140. }
  141. parameters {
  142. key: "lattice_beam"
  143. value {
  144. string_value: "5"
  145. }
  146. }
  147. parameters {
  148. key: "left_padding_size"
  149. value {
  150. string_value: "2.4"
  151. }
  152. }
  153. parameters {
  154. key: "max_supported_transcripts"
  155. value {
  156. string_value: "1"
  157. }
  158. }
  159. parameters {
  160. key: "model_family"
  161. value {
  162. string_value: "riva"
  163. }
  164. }
  165. parameters {
  166. key: "ms_per_timestep"
  167. value {
  168. string_value: "80"
  169. }
  170. }
  171. parameters {
  172. key: "offline"
  173. value {
  174. string_value: "False"
  175. }
  176. }
  177. parameters {
  178. key: "right_padding_size"
  179. value {
  180. string_value: "2.4"
  181. }
  182. }
  183. parameters {
  184. key: "sample_rate"
  185. value {
  186. string_value: "16000"
  187. }
  188. }
  189. parameters {
  190. key: "streaming"
  191. value {
  192. string_value: "True"
  193. }
  194. }
  195. parameters {
  196. key: "type"
  197. value {
  198. string_value: "online"
  199. }
  200. }
  201. parameters {
  202. key: "vad"
  203. value {
  204. string_value: "True"
  205. }
  206. }
  207. ensemble_scheduling {
  208. step {
  209. model_name: "CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming"
  210. model_version: 1
  211. input_map {
  212. key: "AUDIO_SIGNAL"
  213. value: "AUDIO_SIGNAL"
  214. }
  215. input_map {
  216. key: "SAMPLE_RATE"
  217. value: "SAMPLE_RATE"
  218. }
  219. output_map {
  220. key: "AUDIO_FEATURES"
  221. value: "AUDIO_FEATURES"
  222. }
  223. output_map {
  224. key: "AUDIO_PROCESSED"
  225. value: "AUDIO_PROCESSED"
  226. }
  227. }
  228. step {
  229. model_name: "riva-trt-CnLg-SpeUni256-EATL1300-streaming-am-streaming"
  230. model_version: 1
  231. input_map {
  232. key: "audio_signal"
  233. value: "AUDIO_FEATURES"
  234. }
  235. output_map {
  236. key: "logprobs"
  237. value: "CHARACTER_PROBABILITIES"
  238. }
  239. }
  240. step {
  241. model_name: "CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming"
  242. model_version: 1
  243. input_map {
  244. key: "CLASS_LOGITS"
  245. value: "CHARACTER_PROBABILITIES"
  246. }
  247. output_map {
  248. key: "SEGMENTS_START_END"
  249. value: "SEGMENTS_START_END"
  250. }
  251. }
  252. step {
  253. model_name: "CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming"
  254. model_version: 1
  255. input_map {
  256. key: "CLASS_LOGITS"
  257. value: "CHARACTER_PROBABILITIES"
  258. }
  259. input_map {
  260. key: "CUSTOM_CONFIGURATION"
  261. value: "CUSTOM_CONFIGURATION"
  262. }
  263. input_map {
  264. key: "END_FLAG"
  265. value: "END_FLAG"
  266. }
  267. input_map {
  268. key: "SEGMENTS_START_END"
  269. value: "SEGMENTS_START_END"
  270. }
  271. output_map {
  272. key: "FINAL_TRANSCRIPTS"
  273. value: "FINAL_TRANSCRIPTS"
  274. }
  275. output_map {
  276. key: "FINAL_TRANSCRIPTS_SCORE"
  277. value: "FINAL_TRANSCRIPTS_SCORE"
  278. }
  279. output_map {
  280. key: "FINAL_WORDS_START_END"
  281. value: "FINAL_WORDS_START_END"
  282. }
  283. output_map {
  284. key: "PARTIAL_TRANSCRIPTS"
  285. value: "PARTIAL_TRANSCRIPTS"
  286. }
  287. output_map {
  288. key: "PARTIAL_TRANSCRIPTS_STABILITY"
  289. value: "PARTIAL_TRANSCRIPTS_STABILITY"
  290. }
  291. output_map {
  292. key: "PARTIAL_WORDS_START_END"
  293. value: "PARTIAL_WORDS_START_END"
  294. }
  295. }
  296. }
  297.  
  298. I0302 15:25:14.853070 78 autofill.cc:138] TensorFlow SavedModel autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming', unable to find savedmodel directory named 'model.savedmodel'
  299. I0302 15:25:14.939666 78 autofill.cc:151] TensorFlow GraphDef autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming', unable to find graphdef file named 'model.graphdef'
  300. I0302 15:25:15.057584 78 autofill.cc:164] PyTorch autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming', unable to find PyTorch file named 'model.pt'
  301. > Riva waiting for Triton server to load all models...retrying in 1 second
  302. I0302 15:25:15.281224 78 autofill.cc:196] ONNX autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming', unable to find onnx file or directory named 'model.onnx'
  303. > Riva waiting for Triton server to load all models...retrying in 1 second
  304. I0302 15:25:16.911249 78 logging.cc:49] [MemUsageChange] Init CUDA: CPU +320, GPU +0, now: CPU 340, GPU 314 (MiB)
  305. I0302 15:25:16.914647 78 logging.cc:49] Loaded engine size: 0 MB
  306. I0302 15:25:16.914762 78 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine begin: CPU 340 MiB, GPU 314 MiB
  307. E0302 15:25:16.934878 78 logging.cc:43] 1: [stdArchiveReader.cpp::StdArchiveReader::29] Error Code 1: Serialization (Serialization assertion magicTagRead == magicTag failed.Magic tag does not match)
  308. E0302 15:25:16.934920 78 logging.cc:43] 4: [runtime.cpp::deserializeCudaEngine::75] Error Code 4: Internal Error (Engine deserialization failed.)
  309. I0302 15:25:16.993806 78 logging.cc:49] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 340, GPU 314 (MiB)
  310. I0302 15:25:16.993842 78 logging.cc:49] Loaded engine size: 0 MB
  311. I0302 15:25:16.993910 78 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine begin: CPU 340 MiB, GPU 314 MiB
  312. E0302 15:25:16.994315 78 logging.cc:43] 1: [stdArchiveReader.cpp::StdArchiveReader::29] Error Code 1: Serialization (Serialization assertion magicTagRead == magicTag failed.Magic tag does not match)
  313. E0302 15:25:16.994341 78 logging.cc:43] 4: [runtime.cpp::deserializeCudaEngine::75] Error Code 4: Internal Error (Engine deserialization failed.)
  314. I0302 15:25:16.994375 78 autofill.cc:209] TensorRT autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming', unable to find a compatible plan file.
  315. W0302 15:25:16.994388 78 autofill.cc:243] Proceeding with simple config for now
  316. I0302 15:25:16.994396 78 model_config_utils.cc:637] autofilled config: name: "CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming"
  317. max_batch_size: 2048
  318. input {
  319. name: "CLASS_LOGITS"
  320. data_type: TYPE_FP32
  321. dims: -1
  322. dims: 257
  323. }
  324. input {
  325. name: "END_FLAG"
  326. data_type: TYPE_UINT32
  327. dims: 1
  328. }
  329. input {
  330. name: "SEGMENTS_START_END"
  331. data_type: TYPE_INT32
  332. dims: -1
  333. dims: 2
  334. }
  335. input {
  336. name: "CUSTOM_CONFIGURATION"
  337. data_type: TYPE_STRING
  338. dims: -1
  339. dims: 2
  340. }
  341. output {
  342. name: "FINAL_TRANSCRIPTS"
  343. data_type: TYPE_STRING
  344. dims: -1
  345. }
  346. output {
  347. name: "FINAL_TRANSCRIPTS_SCORE"
  348. data_type: TYPE_FP32
  349. dims: -1
  350. }
  351. output {
  352. name: "FINAL_WORDS_START_END"
  353. data_type: TYPE_INT32
  354. dims: -1
  355. dims: 2
  356. }
  357. output {
  358. name: "PARTIAL_TRANSCRIPTS"
  359. data_type: TYPE_STRING
  360. dims: -1
  361. }
  362. output {
  363. name: "PARTIAL_TRANSCRIPTS_STABILITY"
  364. data_type: TYPE_FP32
  365. dims: -1
  366. }
  367. output {
  368. name: "PARTIAL_WORDS_START_END"
  369. data_type: TYPE_INT32
  370. dims: -1
  371. dims: 2
  372. }
  373. instance_group {
  374. count: 1
  375. kind: KIND_GPU
  376. }
  377. optimization {
  378. cuda {
  379. output_copy_stream: true
  380. }
  381. }
  382. sequence_batching {
  383. max_sequence_idle_microseconds: 60000000
  384. control_input {
  385. name: "START"
  386. control {
  387. int32_false_true: 0
  388. int32_false_true: 1
  389. }
  390. }
  391. control_input {
  392. name: "READY"
  393. control {
  394. kind: CONTROL_SEQUENCE_READY
  395. int32_false_true: 0
  396. int32_false_true: 1
  397. }
  398. }
  399. control_input {
  400. name: "END"
  401. control {
  402. kind: CONTROL_SEQUENCE_END
  403. int32_false_true: 0
  404. int32_false_true: 1
  405. }
  406. }
  407. control_input {
  408. name: "CORRID"
  409. control {
  410. kind: CONTROL_SEQUENCE_CORRID
  411. data_type: TYPE_UINT64
  412. }
  413. }
  414. oldest {
  415. max_candidate_sequences: 2048
  416. preferred_batch_size: 32
  417. preferred_batch_size: 64
  418. max_queue_delay_microseconds: 1000
  419. }
  420. }
  421. parameters {
  422. key: "asr_model_delay"
  423. value {
  424. string_value: "-1"
  425. }
  426. }
  427. parameters {
  428. key: "chunk_size"
  429. value {
  430. string_value: "1.2"
  431. }
  432. }
  433. parameters {
  434. key: "compute_timestamps"
  435. value {
  436. string_value: "True"
  437. }
  438. }
  439. parameters {
  440. key: "decoder_num_worker_threads"
  441. value {
  442. string_value: "-1"
  443. }
  444. }
  445. parameters {
  446. key: "decoder_type"
  447. value {
  448. string_value: "greedy"
  449. }
  450. }
  451. parameters {
  452. key: "left_padding_size"
  453. value {
  454. string_value: "2.4"
  455. }
  456. }
  457. parameters {
  458. key: "max_execution_batch_size"
  459. value {
  460. string_value: "1024"
  461. }
  462. }
  463. parameters {
  464. key: "max_supported_transcripts"
  465. value {
  466. string_value: "1"
  467. }
  468. }
  469. parameters {
  470. key: "ms_per_timestep"
  471. value {
  472. string_value: "80"
  473. }
  474. }
  475. parameters {
  476. key: "right_padding_size"
  477. value {
  478. string_value: "2.4"
  479. }
  480. }
  481. parameters {
  482. key: "streaming"
  483. value {
  484. string_value: "True"
  485. }
  486. }
  487. parameters {
  488. key: "use_subword"
  489. value {
  490. string_value: "True"
  491. }
  492. }
  493. parameters {
  494. key: "use_vad"
  495. value {
  496. string_value: "True"
  497. }
  498. }
  499. parameters {
  500. key: "vocab_file"
  501. value {
  502. string_value: "/data/models/1.8.0b0/CnLg-SpeUni256-EATL1300-streaming-ctc-decoder-cpu-streaming/1/riva_decoder_vocabulary.txt"
  503. }
  504. }
  505. backend: "riva_asr_decoder"
  506. model_transaction_policy {
  507. }
  508.  
  509. > Riva waiting for Triton server to load all models...retrying in 1 second
  510. I0302 15:25:17.342891 78 autofill.cc:138] TensorFlow SavedModel autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming', unable to find savedmodel directory named 'model.savedmodel'
  511. I0302 15:25:17.402536 78 autofill.cc:151] TensorFlow GraphDef autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming', unable to find graphdef file named 'model.graphdef'
  512. I0302 15:25:17.467620 78 autofill.cc:164] PyTorch autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming', unable to find PyTorch file named 'model.pt'
  513. I0302 15:25:17.545857 78 autofill.cc:196] ONNX autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming', unable to find onnx file or directory named 'model.onnx'
  514. I0302 15:25:17.613234 78 autofill.cc:209] TensorRT autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming', unable to find a compatible plan file.
  515. W0302 15:25:17.613260 78 autofill.cc:243] Proceeding with simple config for now
  516. I0302 15:25:17.613267 78 model_config_utils.cc:637] autofilled config: name: "CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming"
  517. max_batch_size: 2048
  518. input {
  519. name: "AUDIO_SIGNAL"
  520. data_type: TYPE_FP32
  521. dims: -1
  522. }
  523. input {
  524. name: "SAMPLE_RATE"
  525. data_type: TYPE_UINT32
  526. dims: 1
  527. }
  528. output {
  529. name: "AUDIO_FEATURES"
  530. data_type: TYPE_FP32
  531. dims: 80
  532. dims: -1
  533. }
  534. output {
  535. name: "AUDIO_PROCESSED"
  536. data_type: TYPE_FP32
  537. dims: 1
  538. }
  539. instance_group {
  540. count: 1
  541. kind: KIND_GPU
  542. }
  543. optimization {
  544. cuda {
  545. output_copy_stream: true
  546. }
  547. }
  548. sequence_batching {
  549. max_sequence_idle_microseconds: 60000000
  550. control_input {
  551. name: "START"
  552. control {
  553. int32_false_true: 0
  554. int32_false_true: 1
  555. }
  556. }
  557. control_input {
  558. name: "READY"
  559. control {
  560. kind: CONTROL_SEQUENCE_READY
  561. int32_false_true: 0
  562. int32_false_true: 1
  563. }
  564. }
  565. control_input {
  566. name: "END"
  567. control {
  568. kind: CONTROL_SEQUENCE_END
  569. int32_false_true: 0
  570. int32_false_true: 1
  571. }
  572. }
  573. control_input {
  574. name: "CORRID"
  575. control {
  576. kind: CONTROL_SEQUENCE_CORRID
  577. data_type: TYPE_UINT64
  578. }
  579. }
  580. oldest {
  581. max_candidate_sequences: 2048
  582. preferred_batch_size: 256
  583. preferred_batch_size: 512
  584. max_queue_delay_microseconds: 1000
  585. }
  586. }
  587. parameters {
  588. key: "chunk_size"
  589. value {
  590. string_value: "1.2"
  591. }
  592. }
  593. parameters {
  594. key: "dither"
  595. value {
  596. string_value: "1e-05"
  597. }
  598. }
  599. parameters {
  600. key: "gain"
  601. value {
  602. string_value: "1.0"
  603. }
  604. }
  605. parameters {
  606. key: "left_padding_size"
  607. value {
  608. string_value: "2.4"
  609. }
  610. }
  611. parameters {
  612. key: "max_execution_batch_size"
  613. value {
  614. string_value: "1024"
  615. }
  616. }
  617. parameters {
  618. key: "mean"
  619. value {
  620. string_value: "-11.4412, -9.9334, -9.1292, -9.0365, -9.2804, -9.5643, -9.7342, -9.6925, -9.6333, -9.2808, -9.1887, -9.1422, -9.1397, -9.2028, -9.2749, -9.4776, -9.9185, -10.1557, -10.3800, -10.5067, -10.3190, -10.4728, -10.5529, -10.6402, -10.6440, -10.5113, -10.7395, -10.7870, -10.6074, -10.5033, -10.8278, -10.6384, -10.8481, -10.6875, -10.5454, -10.4747, -10.5165, -10.4930, -10.3413, -10.3472, -10.3735, -10.6830, -10.8813, -10.6338, -10.3856, -10.7727, -10.8957, -10.8068, -10.7373, -10.6108, -10.3405, -10.2889, -10.3922, -10.4946, -10.3367, -10.4164, -10.9949, -10.7196, -10.3971, -10.1734, -9.9257, -9.6557, -9.1761, -9.6653, -9.7876, -9.7230, -9.7792, -9.7056, -9.2702, -9.4650, -9.2755, -9.1369, -9.1174, -8.9197, -8.5394, -8.2614, -8.1353, -8.1422, -8.3430, -8.6655"
  621. }
  622. }
  623. parameters {
  624. key: "norm_per_feature"
  625. value {
  626. string_value: "True"
  627. }
  628. }
  629. parameters {
  630. key: "num_features"
  631. value {
  632. string_value: "80"
  633. }
  634. }
  635. parameters {
  636. key: "precalc_norm_params"
  637. value {
  638. string_value: "False"
  639. }
  640. }
  641. parameters {
  642. key: "precalc_norm_time_steps"
  643. value {
  644. string_value: "0"
  645. }
  646. }
  647. parameters {
  648. key: "right_padding_size"
  649. value {
  650. string_value: "2.4"
  651. }
  652. }
  653. parameters {
  654. key: "sample_rate"
  655. value {
  656. string_value: "16000"
  657. }
  658. }
  659. parameters {
  660. key: "stddev"
  661. value {
  662. string_value: "2.2668, 3.1642, 3.7079, 3.7642, 3.5349, 3.5901, 3.7640, 3.8424, 4.0145, 4.1475, 4.0457, 3.9048, 3.7709, 3.6117, 3.3188, 3.1489, 3.0615, 3.0362, 2.9929, 3.0500, 3.0341, 3.0484, 3.0103, 2.9474, 2.9128, 2.8669, 2.8332, 2.9411, 3.0378, 3.0712, 3.0190, 2.9992, 3.0124, 3.0024, 3.0275, 3.0870, 3.0656, 3.0142, 3.0493, 3.1373, 3.1135, 3.0675, 2.8828, 2.7018, 2.6296, 2.8826, 2.9325, 2.9288, 2.9271, 2.9890, 3.0137, 2.9855, 3.0839, 2.9319, 2.3512, 2.3795, 2.6191, 2.7555, 2.9326, 2.9931, 3.1543, 3.0855, 2.6820, 3.0566, 3.1272, 3.1663, 3.1836, 3.0018, 2.9089, 3.1727, 3.1626, 3.1086, 2.9804, 3.1107, 3.2998, 3.3697, 3.3716, 3.2487, 3.1597, 3.1181"
  663. }
  664. }
  665. parameters {
  666. key: "stddev_floor"
  667. value {
  668. string_value: "1e-05"
  669. }
  670. }
  671. parameters {
  672. key: "streaming"
  673. value {
  674. string_value: "True"
  675. }
  676. }
  677. parameters {
  678. key: "transpose"
  679. value {
  680. string_value: "False"
  681. }
  682. }
  683. parameters {
  684. key: "use_utterance_norm_params"
  685. value {
  686. string_value: "False"
  687. }
  688. }
  689. parameters {
  690. key: "window_size"
  691. value {
  692. string_value: "0.025"
  693. }
  694. }
  695. parameters {
  696. key: "window_stride"
  697. value {
  698. string_value: "0.01"
  699. }
  700. }
  701. backend: "riva_asr_features"
  702. model_transaction_policy {
  703. }
  704.  
  705. I0302 15:25:18.239271 78 autofill.cc:138] TensorFlow SavedModel autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming', unable to find savedmodel directory named 'model.savedmodel'
  706. > Riva waiting for Triton server to load all models...retrying in 1 second
  707. I0302 15:25:18.378833 78 autofill.cc:151] TensorFlow GraphDef autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming', unable to find graphdef file named 'model.graphdef'
  708. I0302 15:25:18.462903 78 autofill.cc:164] PyTorch autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming', unable to find PyTorch file named 'model.pt'
  709. I0302 15:25:18.571008 78 autofill.cc:196] ONNX autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming', unable to find onnx file or directory named 'model.onnx'
  710. I0302 15:25:18.698008 78 logging.cc:49] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 340, GPU 314 (MiB)
  711. I0302 15:25:18.698042 78 logging.cc:49] Loaded engine size: 0 MB
  712. I0302 15:25:18.698109 78 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine begin: CPU 340 MiB, GPU 314 MiB
  713. E0302 15:25:18.698500 78 logging.cc:43] 1: [stdArchiveReader.cpp::StdArchiveReader::29] Error Code 1: Serialization (Serialization assertion magicTagRead == magicTag failed.Magic tag does not match)
  714. E0302 15:25:18.698528 78 logging.cc:43] 4: [runtime.cpp::deserializeCudaEngine::75] Error Code 4: Internal Error (Engine deserialization failed.)
  715. I0302 15:25:18.698561 78 autofill.cc:209] TensorRT autofill: Internal: unable to autofill for 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming', unable to find a compatible plan file.
  716. W0302 15:25:18.698575 78 autofill.cc:243] Proceeding with simple config for now
  717. I0302 15:25:18.698589 78 model_config_utils.cc:637] autofilled config: name: "CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming"
  718. max_batch_size: 2048
  719. input {
  720. name: "CLASS_LOGITS"
  721. data_type: TYPE_FP32
  722. dims: -1
  723. dims: 257
  724. }
  725. output {
  726. name: "SEGMENTS_START_END"
  727. data_type: TYPE_INT32
  728. dims: -1
  729. dims: 2
  730. }
  731. instance_group {
  732. count: 1
  733. kind: KIND_CPU
  734. }
  735. optimization {
  736. cuda {
  737. output_copy_stream: true
  738. }
  739. }
  740. sequence_batching {
  741. max_sequence_idle_microseconds: 60000000
  742. control_input {
  743. name: "START"
  744. control {
  745. int32_false_true: 0
  746. int32_false_true: 1
  747. }
  748. }
  749. control_input {
  750. name: "READY"
  751. control {
  752. kind: CONTROL_SEQUENCE_READY
  753. int32_false_true: 0
  754. int32_false_true: 1
  755. }
  756. }
  757. }
  758. parameters {
  759. key: "chunk_size"
  760. value {
  761. string_value: "1.2"
  762. }
  763. }
  764. parameters {
  765. key: "ms_per_timestep"
  766. value {
  767. string_value: "80"
  768. }
  769. }
  770. parameters {
  771. key: "residue_blanks_at_end"
  772. value {
  773. string_value: "0"
  774. }
  775. }
  776. parameters {
  777. key: "residue_blanks_at_start"
  778. value {
  779. string_value: "-2"
  780. }
  781. }
  782. parameters {
  783. key: "streaming"
  784. value {
  785. string_value: "True"
  786. }
  787. }
  788. parameters {
  789. key: "use_subword"
  790. value {
  791. string_value: "True"
  792. }
  793. }
  794. parameters {
  795. key: "vad_start_history"
  796. value {
  797. string_value: "300"
  798. }
  799. }
  800. parameters {
  801. key: "vad_start_th"
  802. value {
  803. string_value: "0.2"
  804. }
  805. }
  806. parameters {
  807. key: "vad_stop_history"
  808. value {
  809. string_value: "2400"
  810. }
  811. }
  812. parameters {
  813. key: "vad_stop_th"
  814. value {
  815. string_value: "0.98"
  816. }
  817. }
  818. parameters {
  819. key: "vad_type"
  820. value {
  821. string_value: "ctc-vad"
  822. }
  823. }
  824. parameters {
  825. key: "vocab_file"
  826. value {
  827. string_value: "/data/models/1.8.0b0/CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming/1/riva_decoder_vocabulary.txt"
  828. }
  829. }
  830. backend: "riva_asr_vad"
  831. model_transaction_policy {
  832. }
  833.  
  834. > Riva waiting for Triton server to load all models...retrying in 1 second
  835. > Riva waiting for Triton server to load all models...retrying in 1 second
  836. > Riva waiting for Triton server to load all models...retrying in 1 second
  837. > Riva waiting for Triton server to load all models...retrying in 1 second
  838. > Riva waiting for Triton server to load all models...retrying in 1 second
  839. > Riva waiting for Triton server to load all models...retrying in 1 second
  840. > Riva waiting for Triton server to load all models...retrying in 1 second
  841. > Riva waiting for Triton server to load all models...retrying in 1 second
  842. > Riva waiting for Triton server to load all models...retrying in 1 second
  843. > Riva waiting for Triton server to load all models...retrying in 1 second
  844. > Riva waiting for Triton server to load all models...retrying in 1 second
  845. > Riva waiting for Triton server to load all models...retrying in 1 second
  846. I0302 15:25:30.676788 78 logging.cc:49] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 923, GPU 314 (MiB)
  847. I0302 15:25:30.676837 78 logging.cc:49] Loaded engine size: 291 MB
  848. I0302 15:25:30.676907 78 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine begin: CPU 923 MiB, GPU 314 MiB
  849. > Riva waiting for Triton server to load all models...retrying in 1 second
  850. > Riva waiting for Triton server to load all models...retrying in 1 second
  851. > Riva waiting for Triton server to load all models...retrying in 1 second
  852. I0302 15:25:33.633526 78 logging.cc:52] Using cublasLt a tactic source
  853. I0302 15:25:33.633630 78 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +491, GPU +212, now: CPU 1422, GPU 814 (MiB)
  854. I0302 15:25:33.633776 78 logging.cc:52] Using cuDNN as a tactic source
  855. > Riva waiting for Triton server to load all models...retrying in 1 second
  856. I0302 15:25:35.417042 78 logging.cc:49] [MemUsageChange] Init cuDNN: CPU +287, GPU +198, now: CPU 1709, GPU 1012 (MiB)
  857. I0302 15:25:35.419297 78 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1709, GPU 994 (MiB)
  858. I0302 15:25:35.419344 78 logging.cc:52] Deserialization required 4742097 microseconds.
  859. I0302 15:25:35.419426 78 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine end: CPU 1709 MiB, GPU 994 MiB
  860. I0302 15:25:35.439235 78 autofill.cc:209] TensorRT autofill: OK:
  861. I0302 15:25:35.439283 78 model_config_utils.cc:637] autofilled config: name: "riva-trt-CnLg-SpeUni256-EATL1300-streaming-am-streaming"
  862. platform: "tensorrt_plan"
  863. max_batch_size: 64
  864. input {
  865. name: "audio_signal"
  866. data_type: TYPE_FP32
  867. dims: 80
  868. dims: 601
  869. }
  870. output {
  871. name: "logprobs"
  872. data_type: TYPE_FP32
  873. dims: 76
  874. dims: 257
  875. }
  876. instance_group {
  877. count: 2
  878. kind: KIND_GPU
  879. }
  880. default_model_filename: "model.plan"
  881. dynamic_batching {
  882. preferred_batch_size: 32
  883. preferred_batch_size: 64
  884. max_queue_delay_microseconds: 1000
  885. preserve_ordering: true
  886. }
  887. optimization {
  888. cuda {
  889. output_copy_stream: true
  890. }
  891. }
  892. model_transaction_policy {
  893. }
  894.  
  895. I0302 15:25:35.439898 78 model_repository_manager.cc:749] AsyncLoad() 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming'
  896. > Riva waiting for Triton server to load all models...retrying in 1 second
  897. I0302 15:25:35.592525 78 model_repository_manager.cc:988] TriggerNextAction() 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming' version 1: 1
  898. I0302 15:25:35.592554 78 model_repository_manager.cc:1026] Load() 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming' version 1
  899. I0302 15:25:35.592567 78 model_repository_manager.cc:1045] loading: CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming:1
  900. I0302 15:25:35.692779 78 model_repository_manager.cc:749] AsyncLoad() 'riva-trt-CnLg-SpeUni256-EATL1300-streaming-am-streaming'
  901. I0302 15:25:35.692780 78 model_repository_manager.cc:1105] CreateInferenceBackend() 'CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming' version 1
  902. I0302 15:25:35.736442 78 model_repository_manager.cc:988] TriggerNextAction() 'riva-trt-CnLg-SpeUni256-EATL1300-streaming-am-streaming' version 1: 1
  903. I0302 15:25:35.736470 78 model_repository_manager.cc:1026] Load() 'riva-trt-CnLg-SpeUni256-EATL1300-streaming-am-streaming' version 1
  904. I0302 15:25:35.736482 78 model_repository_manager.cc:1045] loading: riva-trt-CnLg-SpeUni256-EATL1300-streaming-am-streaming:1
  905. I0302 15:25:35.836659 78 model_repository_manager.cc:749] AsyncLoad() 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming'
  906. I0302 15:25:35.836680 78 model_repository_manager.cc:1105] CreateInferenceBackend() 'riva-trt-CnLg-SpeUni256-EATL1300-streaming-am-streaming' version 1
  907. I0302 15:25:35.879046 78 model_repository_manager.cc:988] TriggerNextAction() 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming' version 1: 1
  908. I0302 15:25:35.879079 78 model_repository_manager.cc:1026] Load() 'CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming' version 1
  909. I0302 15:25:35.879092 78 model_repository_manager.cc:1045] loading: CnLg-SpeUni256-EATL1300-streaming-feature-extractor-streaming:1
  910. I0302 15:25:35.937681 78 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/riva_asr_vad/libtriton_riva_asr_vad.so
  911. I0302 15:25:35.945204 78 vad_library.cc:18] TRITONBACKEND_ModelInitialize: CnLg-SpeUni256-EATL1300-streaming-voice-activity-detector-ctc-streaming (version 1)
  912. I0302 15:25:35.948423 78 model_config_utils.cc:1524] ModelConfig 64-bit fields:
  913. I0302 15:25:35.948441 78 model_config_utils.cc:1526] ModelConfig::dynamic_batching::default_queue_policy::default_timeout_microseconds
  914. I0302 15:25:35.948455 78 model_config_utils.cc:1526] ModelConfig::dynamic_batching::max_queue_delay_microseconds
  915. I0302 15:25:35.948463 78 model_config_utils.cc:1526] ModelConfig::dynamic_batching::priority_queue_policy::value::default_timeout_microseconds
  916. I0302 15:25:35.948470 78 model_config_utils.cc:1526] ModelConfig::ensemble_scheduling::step::model_version
  917. I0302 15:25:35.948485 78 model_config_utils.cc:1526] ModelConfig::input::dims
  918. I0302 15:25:35.948504 78 model_config_utils.cc:1526] ModelConfig::input::reshape::shape
  919. I0302 15:25:35.948516 78 model_config_utils.cc:1526] ModelConfig::instance_group::secondary_devices::device_id
  920. I0302 15:25:35.948525 78 model_config_utils.cc:1526] ModelConfig::model_warmup::inputs::value::dims
  921. I0302 15:25:35.948536 78 model_config_utils.cc:1526] ModelConfig::optimization::cuda::graph_spec::graph_lower_bound::input::value::dim
  922. I0302 15:25:35.948550 78 model_config_utils.cc:1526] ModelConfig::optimization::cuda::graph_spec::input::value::dim
  923. I0302 15:25:35.948561 78 model_config_utils.cc:1526] ModelConfig::output::dims
  924. I0302 15:25:35.948570 78 model_config_utils.cc:1526] ModelConfig::output::reshape::shape
  925. I0302 15:25:35.948577 78 model_config_utils.cc:1526] ModelConfig::sequence_batching::direct::max_queue_delay_microseconds
  926. I0302 15:25:35.948583 78 model_config_utils.cc:1526] ModelConfig::sequence_batching::max_sequence_idle_microseconds
  927. I0302 15:25:35.948598 78 model_config_utils.cc:1526] ModelConfig::sequence_batching::oldest::max_queue_delay_microseconds
  928. I0302 15:25:35.948605 78 model_config_utils.cc:1526] ModelConfig::version_policy::specific::versions
  929. W:parameter_parser.cc:118: Parameter max_execution_batch_size could not be set from parameters
  930. W:parameter_parser.cc:119: Default value will be used
  931. W:parameter_parser.cc:118: Parameter max_execution_batch_size could not be set from parameters
  932. W:parameter_parser.cc:119: Default value will be used
  933. terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >'
  934. > Riva waiting for Triton server to load all models...retrying in 1 second
  935. > Riva waiting for Triton server to load all models...retrying in 1 second
  936. > Riva waiting for Triton server to load all models...retrying in 1 second
  937. > Riva waiting for Triton server to load all models...retrying in 1 second
  938. > Riva waiting for Triton server to load all models...retrying in 1 second
  939. > Riva waiting for Triton server to load all models...retrying in 1 second
  940. > Riva waiting for Triton server to load all models...retrying in 1 second
  941. > Riva waiting for Triton server to load all models...retrying in 1 second
  942. > Riva waiting for Triton server to load all models...retrying in 1 second
  943. > Riva waiting for Triton server to load all models...retrying in 1 second
  944. > Riva waiting for Triton server to load all models...retrying in 1 second
  945. > Riva waiting for Triton server to load all models...retrying in 1 second
  946. /opt/riva/start-riva.sh: line 4: 78 Aborted (core dumped) tritonserver --log-verbose=1 --log-info=true --log-warning=true --log-error=true --strict-model-config=false --model-control-mode=explicit "$LOAD_MODEL_STR" --model-repository "$MODEL_REPOSITORY"
  947. > Triton server died before reaching ready state. Terminating Riva startup.
Advertisement
RAW Paste Data Copied
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!