API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Mar 25th, 2023
61
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 31.90 KB | None | 0 0
raw download clone embed print report
  1. fauxpilot-copilot_proxy-1  | Returned completion in 82816.30563735962 ms 1679737428.549292
  2. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:43:48,549 :: 74.235.184.25:47144 - "POST /v1/completions HTTP/1.1" 200 OK
  3. fauxpilot-copilot_proxy-1  | Returned completion in 82816.67017936707 ms 1679737428.5498981
  4. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:43:48,550 :: 74.235.184.25:47152 - "POST /v1/completions HTTP/1.1" 200 OK
  5. fauxpilot-copilot_proxy-1  | Returned completion in 82816.81108474731 ms 1679737428.5502555
  6. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:43:48,550 :: 74.235.184.25:47158 - "POST /v1/completions HTTP/1.1" 200 OK
  7. fauxpilot-copilot_proxy-1  | Returned completion in 82816.92790985107 ms 1679737428.5505447
  8. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:43:48,550 :: 74.235.184.25:47164 - "POST /v1/completions HTTP/1.1" 200 OK
  9. fauxpilot-copilot_proxy-1  | Returned completion in 82817.0337677002 ms 1679737428.55082
  10. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:43:48,550 :: 74.235.184.25:47172 - "POST /v1/completions HTTP/1.1" 200 OK
  11. fauxpilot-copilot_proxy-1  | Returned completion in 82817.1739578247 ms 1679737428.5511227
  12. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:43:48,551 :: 74.235.184.25:47180 - "POST /v1/completions HTTP/1.1" 200 OK
  13. fauxpilot-copilot_proxy-1  | got request 1679737428.552212
  14. fauxpilot-triton-1         | W0325 09:43:49.554066 89 libfastertransformer.cc:1397] model fastertransformer, instance fastertransformer_0, executing 1 requests
  15. fauxpilot-triton-1         | W0325 09:43:49.554241 89 libfastertransformer.cc:638] TRITONBACKEND_ModelExecute: Running fastertransformer_0 with 1 requests
  16. fauxpilot-triton-1         | W0325 09:43:49.554345 89 libfastertransformer.cc:693] get total batch_size = 1
  17. fauxpilot-triton-1         | W0325 09:43:49.554452 89 libfastertransformer.cc:1051] get input count = 16
  18. fauxpilot-triton-1         | W0325 09:43:49.554566 89 libfastertransformer.cc:1117] collect name: start_id size: 4 bytes
  19. fauxpilot-triton-1         | W0325 09:43:49.554615 89 libfastertransformer.cc:1117] collect name: input_ids size: 540 bytes
  20. fauxpilot-triton-1         | W0325 09:43:49.554641 89 libfastertransformer.cc:1117] collect name: bad_words_list size: 8 bytes
  21. fauxpilot-triton-1         | W0325 09:43:49.554667 89 libfastertransformer.cc:1117] collect name: random_seed size: 4 bytes
  22. fauxpilot-triton-1         | W0325 09:43:49.554694 89 libfastertransformer.cc:1117] collect name: end_id size: 4 bytes
  23. fauxpilot-triton-1         | W0325 09:43:49.554720 89 libfastertransformer.cc:1117] collect name: input_lengths size: 4 bytes
  24. fauxpilot-triton-1         | W0325 09:43:49.554746 89 libfastertransformer.cc:1117] collect name: request_output_len size: 4 bytes
  25. fauxpilot-triton-1         | W0325 09:43:49.554772 89 libfastertransformer.cc:1117] collect name: runtime_top_k size: 4 bytes
  26. fauxpilot-triton-1         | W0325 09:43:49.554799 89 libfastertransformer.cc:1117] collect name: runtime_top_p size: 4 bytes
  27. fauxpilot-triton-1         | W0325 09:43:49.554825 89 libfastertransformer.cc:1117] collect name: is_return_log_probs size: 1 bytes
  28. fauxpilot-triton-1         | W0325 09:43:49.554851 89 libfastertransformer.cc:1117] collect name: stop_words_list size: 8 bytes
  29. fauxpilot-triton-1         | W0325 09:43:49.554877 89 libfastertransformer.cc:1117] collect name: temperature size: 4 bytes
  30. fauxpilot-triton-1         | W0325 09:43:49.554903 89 libfastertransformer.cc:1117] collect name: len_penalty size: 4 bytes
  31. fauxpilot-triton-1         | W0325 09:43:49.554929 89 libfastertransformer.cc:1117] collect name: beam_width size: 4 bytes
  32. fauxpilot-triton-1         | W0325 09:43:49.554956 89 libfastertransformer.cc:1117] collect name: beam_search_diversity_rate size: 4 bytes
  33. fauxpilot-triton-1         | W0325 09:43:49.554982 89 libfastertransformer.cc:1117] collect name: repetition_penalty size: 4 bytes
  34. fauxpilot-triton-1         | W0325 09:43:49.555007 89 libfastertransformer.cc:1130] the data is in CPU
  35. fauxpilot-triton-1         | W0325 09:43:49.555032 89 libfastertransformer.cc:1137] the data is in CPU
  36. fauxpilot-triton-1         | W0325 09:43:49.555203 89 libfastertransformer.cc:999] before ThreadForward 0
  37. fauxpilot-triton-1         | W0325 09:43:49.555361 89 libfastertransformer.cc:1006] after ThreadForward 0
  38. fauxpilot-triton-1         | I0325 09:43:49.555550 89 libfastertransformer.cc:834] Start to forward
  39. fauxpilot-triton-1         | I0325 09:44:06.115290 89 libfastertransformer.cc:836] Stop to forward
  40. fauxpilot-triton-1         | W0325 09:44:06.115451 89 libfastertransformer.cc:1161] Get output_tensors 0: output_ids
  41. fauxpilot-triton-1         | W0325 09:44:06.115483 89 libfastertransformer.cc:1171]     output_type: UINT32
  42. fauxpilot-triton-1         | W0325 09:44:06.115552 89 libfastertransformer.cc:1191]     output shape: [1, 1, 635]
  43. fauxpilot-triton-1         | W0325 09:44:06.115640 89 libfastertransformer.cc:1161] Get output_tensors 1: sequence_length
  44. fauxpilot-triton-1         | W0325 09:44:06.115690 89 libfastertransformer.cc:1171]     output_type: INT32
  45. fauxpilot-triton-1         | W0325 09:44:06.115747 89 libfastertransformer.cc:1191]     output shape: [1, 1]
  46. fauxpilot-triton-1         | W0325 09:44:06.115815 89 libfastertransformer.cc:1206] PERFORMED GPU copy: NO
  47. fauxpilot-triton-1         | W0325 09:44:06.115880 89 libfastertransformer.cc:780] get response size = 1
  48. fauxpilot-triton-1         | W0325 09:44:06.116036 89 libfastertransformer.cc:795] response is sent
  49. fauxpilot-copilot_proxy-1  | Returned completion in 17564.770460128784 ms 1679737446.1169825
  50. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:44:06,117 :: 74.235.184.25:45654 - "POST /v1/completions HTTP/1.1" 200 OK
  51. fauxpilot-copilot_proxy-1  | got request 1679737446.1195467
  52. fauxpilot-copilot_proxy-1  | got request 1679737446.1197827
  53. fauxpilot-copilot_proxy-1  | got request 1679737446.1199694
  54. fauxpilot-copilot_proxy-1  | got request 1679737446.120142
  55. fauxpilot-copilot_proxy-1  | got request 1679737446.1203208
  56. fauxpilot-copilot_proxy-1  | got request 1679737446.1205049
  57. fauxpilot-triton-1         | W0325 09:44:07.123815 89 libfastertransformer.cc:1397] model fastertransformer, instance fastertransformer_0, executing 1 requests
  58. fauxpilot-triton-1         | W0325 09:44:07.123867 89 libfastertransformer.cc:638] TRITONBACKEND_ModelExecute: Running fastertransformer_0 with 1 requests
  59. fauxpilot-triton-1         | W0325 09:44:07.123880 89 libfastertransformer.cc:693] get total batch_size = 1
  60. fauxpilot-triton-1         | W0325 09:44:07.123895 89 libfastertransformer.cc:1051] get input count = 16
  61. fauxpilot-triton-1         | W0325 09:44:07.123916 89 libfastertransformer.cc:1117] collect name: start_id size: 4 bytes
  62. fauxpilot-triton-1         | W0325 09:44:07.123938 89 libfastertransformer.cc:1117] collect name: input_ids size: 2696 bytes
  63. fauxpilot-triton-1         | W0325 09:44:07.123951 89 libfastertransformer.cc:1117] collect name: bad_words_list size: 8 bytes
  64. fauxpilot-triton-1         | W0325 09:44:07.123963 89 libfastertransformer.cc:1117] collect name: random_seed size: 4 bytes
  65. fauxpilot-triton-1         | W0325 09:44:07.123977 89 libfastertransformer.cc:1117] collect name: end_id size: 4 bytes
  66. fauxpilot-triton-1         | W0325 09:44:07.123990 89 libfastertransformer.cc:1117] collect name: input_lengths size: 4 bytes
  67. fauxpilot-triton-1         | W0325 09:44:07.124003 89 libfastertransformer.cc:1117] collect name: request_output_len size: 4 bytes
  68. fauxpilot-triton-1         | W0325 09:44:07.124018 89 libfastertransformer.cc:1117] collect name: runtime_top_k size: 4 bytes
  69. fauxpilot-triton-1         | W0325 09:44:07.124031 89 libfastertransformer.cc:1117] collect name: runtime_top_p size: 4 bytes
  70. fauxpilot-triton-1         | W0325 09:44:07.124043 89 libfastertransformer.cc:1117] collect name: is_return_log_probs size: 1 bytes
  71. fauxpilot-triton-1         | W0325 09:44:07.124055 89 libfastertransformer.cc:1117] collect name: stop_words_list size: 8 bytes
  72. fauxpilot-triton-1         | W0325 09:44:07.124067 89 libfastertransformer.cc:1117] collect name: temperature size: 4 bytes
  73. fauxpilot-triton-1         | W0325 09:44:07.124079 89 libfastertransformer.cc:1117] collect name: len_penalty size: 4 bytes
  74. fauxpilot-triton-1         | W0325 09:44:07.124090 89 libfastertransformer.cc:1117] collect name: beam_width size: 4 bytes
  75. fauxpilot-triton-1         | W0325 09:44:07.124104 89 libfastertransformer.cc:1117] collect name: beam_search_diversity_rate size: 4 bytes
  76. fauxpilot-triton-1         | W0325 09:44:07.124116 89 libfastertransformer.cc:1117] collect name: repetition_penalty size: 4 bytes
  77. fauxpilot-triton-1         | W0325 09:44:07.124128 89 libfastertransformer.cc:1130] the data is in CPU
  78. fauxpilot-triton-1         | W0325 09:44:07.124138 89 libfastertransformer.cc:1137] the data is in CPU
  79. fauxpilot-triton-1         | W0325 09:44:07.124160 89 libfastertransformer.cc:999] before ThreadForward 0
  80. fauxpilot-triton-1         | W0325 09:44:07.124253 89 libfastertransformer.cc:1006] after ThreadForward 0
  81. fauxpilot-triton-1         | I0325 09:44:07.124435 89 libfastertransformer.cc:834] Start to forward
  82. fauxpilot-triton-1         | I0325 09:44:26.125114 89 libfastertransformer.cc:836] Stop to forward
  83. fauxpilot-triton-1         | W0325 09:44:26.125305 89 libfastertransformer.cc:1161] Get output_tensors 0: output_ids
  84. fauxpilot-triton-1         | W0325 09:44:26.125327 89 libfastertransformer.cc:1171]     output_type: UINT32
  85. fauxpilot-triton-1         | W0325 09:44:26.125331 89 libfastertransformer.cc:1191]     output shape: [1, 1, 1174]
  86. fauxpilot-triton-1         | W0325 09:44:26.125363 89 libfastertransformer.cc:1161] Get output_tensors 1: sequence_length
  87. fauxpilot-triton-1         | W0325 09:44:26.125367 89 libfastertransformer.cc:1171]     output_type: INT32
  88. fauxpilot-triton-1         | W0325 09:44:26.125370 89 libfastertransformer.cc:1191]     output shape: [1, 1]
  89. fauxpilot-triton-1         | W0325 09:44:26.125385 89 libfastertransformer.cc:1206] PERFORMED GPU copy: NO
  90. fauxpilot-triton-1         | W0325 09:44:26.125393 89 libfastertransformer.cc:780] get response size = 1
  91. fauxpilot-triton-1         | W0325 09:44:26.125518 89 libfastertransformer.cc:795] response is sent
  92. fauxpilot-triton-1         | W0325 09:44:27.131119 89 libfastertransformer.cc:1397] model fastertransformer, instance fastertransformer_0, executing 1 requests
  93. fauxpilot-triton-1         | W0325 09:44:27.131176 89 libfastertransformer.cc:638] TRITONBACKEND_ModelExecute: Running fastertransformer_0 with 1 requests
  94. fauxpilot-triton-1         | W0325 09:44:27.131188 89 libfastertransformer.cc:693] get total batch_size = 1
  95. fauxpilot-triton-1         | W0325 09:44:27.131202 89 libfastertransformer.cc:1051] get input count = 16
  96. fauxpilot-triton-1         | W0325 09:44:27.131224 89 libfastertransformer.cc:1117] collect name: start_id size: 4 bytes
  97. fauxpilot-triton-1         | W0325 09:44:27.131247 89 libfastertransformer.cc:1117] collect name: input_ids size: 5444 bytes
  98. fauxpilot-triton-1         | W0325 09:44:27.131262 89 libfastertransformer.cc:1117] collect name: bad_words_list size: 8 bytes
  99. fauxpilot-triton-1         | W0325 09:44:27.131273 89 libfastertransformer.cc:1117] collect name: random_seed size: 4 bytes
  100. fauxpilot-triton-1         | W0325 09:44:27.131287 89 libfastertransformer.cc:1117] collect name: end_id size: 4 bytes
  101. fauxpilot-triton-1         | W0325 09:44:27.131301 89 libfastertransformer.cc:1117] collect name: input_lengths size: 4 bytes
  102. fauxpilot-triton-1         | W0325 09:44:27.131314 89 libfastertransformer.cc:1117] collect name: request_output_len size: 4 bytes
  103. fauxpilot-triton-1         | W0325 09:44:27.131328 89 libfastertransformer.cc:1117] collect name: runtime_top_k size: 4 bytes
  104. fauxpilot-triton-1         | W0325 09:44:27.131342 89 libfastertransformer.cc:1117] collect name: runtime_top_p size: 4 bytes
  105. fauxpilot-triton-1         | W0325 09:44:27.131355 89 libfastertransformer.cc:1117] collect name: is_return_log_probs size: 1 bytes
  106. fauxpilot-triton-1         | W0325 09:44:27.131368 89 libfastertransformer.cc:1117] collect name: stop_words_list size: 8 bytes
  107. fauxpilot-triton-1         | W0325 09:44:27.131380 89 libfastertransformer.cc:1117] collect name: temperature size: 4 bytes
  108. fauxpilot-triton-1         | W0325 09:44:27.131391 89 libfastertransformer.cc:1117] collect name: len_penalty size: 4 bytes
  109. fauxpilot-triton-1         | W0325 09:44:27.131403 89 libfastertransformer.cc:1117] collect name: beam_width size: 4 bytes
  110. fauxpilot-triton-1         | W0325 09:44:27.131418 89 libfastertransformer.cc:1117] collect name: beam_search_diversity_rate size: 4 bytes
  111. fauxpilot-triton-1         | W0325 09:44:27.131439 89 libfastertransformer.cc:1117] collect name: repetition_penalty size: 4 bytes
  112. fauxpilot-triton-1         | W0325 09:44:27.131452 89 libfastertransformer.cc:1130] the data is in CPU
  113. fauxpilot-triton-1         | W0325 09:44:27.131463 89 libfastertransformer.cc:1137] the data is in CPU
  114. fauxpilot-triton-1         | W0325 09:44:27.131485 89 libfastertransformer.cc:999] before ThreadForward 0
  115. fauxpilot-triton-1         | W0325 09:44:27.131530 89 libfastertransformer.cc:1006] after ThreadForward 0
  116. fauxpilot-triton-1         | I0325 09:44:27.131633 89 libfastertransformer.cc:834] Start to forward
  117. fauxpilot-triton-1         | I0325 09:44:49.196372 89 libfastertransformer.cc:836] Stop to forward
  118. fauxpilot-triton-1         | W0325 09:44:49.196562 89 libfastertransformer.cc:1161] Get output_tensors 0: output_ids
  119. fauxpilot-triton-1         | W0325 09:44:49.196581 89 libfastertransformer.cc:1171]     output_type: UINT32
  120. fauxpilot-triton-1         | W0325 09:44:49.196586 89 libfastertransformer.cc:1191]     output shape: [1, 1, 1861]
  121. fauxpilot-triton-1         | W0325 09:44:49.196621 89 libfastertransformer.cc:1161] Get output_tensors 1: sequence_length
  122. fauxpilot-triton-1         | W0325 09:44:49.196626 89 libfastertransformer.cc:1171]     output_type: INT32
  123. fauxpilot-triton-1         | W0325 09:44:49.196629 89 libfastertransformer.cc:1191]     output shape: [1, 1]
  124. fauxpilot-triton-1         | W0325 09:44:49.196644 89 libfastertransformer.cc:1206] PERFORMED GPU copy: NO
  125. fauxpilot-triton-1         | W0325 09:44:49.196654 89 libfastertransformer.cc:780] get response size = 1
  126. fauxpilot-triton-1         | W0325 09:44:49.196781 89 libfastertransformer.cc:795] response is sent
  127. fauxpilot-triton-1         | W0325 09:44:50.200287 89 libfastertransformer.cc:1397] model fastertransformer, instance fastertransformer_0, executing 1 requests
  128. fauxpilot-triton-1         | W0325 09:44:50.200342 89 libfastertransformer.cc:638] TRITONBACKEND_ModelExecute: Running fastertransformer_0 with 1 requests
  129. fauxpilot-triton-1         | W0325 09:44:50.200355 89 libfastertransformer.cc:693] get total batch_size = 1
  130. fauxpilot-triton-1         | W0325 09:44:50.200369 89 libfastertransformer.cc:1051] get input count = 16
  131. fauxpilot-triton-1         | W0325 09:44:50.200390 89 libfastertransformer.cc:1117] collect name: start_id size: 4 bytes
  132. fauxpilot-triton-1         | W0325 09:44:50.200411 89 libfastertransformer.cc:1117] collect name: input_ids size: 2392 bytes
  133. fauxpilot-triton-1         | W0325 09:44:50.200424 89 libfastertransformer.cc:1117] collect name: bad_words_list size: 8 bytes
  134. fauxpilot-triton-1         | W0325 09:44:50.200435 89 libfastertransformer.cc:1117] collect name: random_seed size: 4 bytes
  135. fauxpilot-triton-1         | W0325 09:44:50.200449 89 libfastertransformer.cc:1117] collect name: end_id size: 4 bytes
  136. fauxpilot-triton-1         | W0325 09:44:50.200462 89 libfastertransformer.cc:1117] collect name: input_lengths size: 4 bytes
  137. fauxpilot-triton-1         | W0325 09:44:50.200475 89 libfastertransformer.cc:1117] collect name: request_output_len size: 4 bytes
  138. fauxpilot-triton-1         | W0325 09:44:50.200491 89 libfastertransformer.cc:1117] collect name: runtime_top_k size: 4 bytes
  139. fauxpilot-triton-1         | W0325 09:44:50.200503 89 libfastertransformer.cc:1117] collect name: runtime_top_p size: 4 bytes
  140. fauxpilot-triton-1         | W0325 09:44:50.200515 89 libfastertransformer.cc:1117] collect name: is_return_log_probs size: 1 bytes
  141. fauxpilot-triton-1         | W0325 09:44:50.200528 89 libfastertransformer.cc:1117] collect name: stop_words_list size: 8 bytes
  142. fauxpilot-triton-1         | W0325 09:44:50.200541 89 libfastertransformer.cc:1117] collect name: temperature size: 4 bytes
  143. fauxpilot-triton-1         | W0325 09:44:50.200554 89 libfastertransformer.cc:1117] collect name: len_penalty size: 4 bytes
  144. fauxpilot-triton-1         | W0325 09:44:50.200567 89 libfastertransformer.cc:1117] collect name: beam_width size: 4 bytes
  145. fauxpilot-triton-1         | W0325 09:44:50.200581 89 libfastertransformer.cc:1117] collect name: beam_search_diversity_rate size: 4 bytes
  146. fauxpilot-triton-1         | W0325 09:44:50.200595 89 libfastertransformer.cc:1117] collect name: repetition_penalty size: 4 bytes
  147. fauxpilot-triton-1         | W0325 09:44:50.200608 89 libfastertransformer.cc:1130] the data is in CPU
  148. fauxpilot-triton-1         | W0325 09:44:50.200619 89 libfastertransformer.cc:1137] the data is in CPU
  149. fauxpilot-triton-1         | W0325 09:44:50.200641 89 libfastertransformer.cc:999] before ThreadForward 0
  150. fauxpilot-triton-1         | W0325 09:44:50.200745 89 libfastertransformer.cc:1006] after ThreadForward 0
  151. fauxpilot-triton-1         | I0325 09:44:50.200833 89 libfastertransformer.cc:834] Start to forward
  152. fauxpilot-triton-1         | I0325 09:45:08.857338 89 libfastertransformer.cc:836] Stop to forward
  153. fauxpilot-triton-1         | W0325 09:45:08.857530 89 libfastertransformer.cc:1161] Get output_tensors 0: output_ids
  154. fauxpilot-triton-1         | W0325 09:45:08.857547 89 libfastertransformer.cc:1171]     output_type: UINT32
  155. fauxpilot-triton-1         | W0325 09:45:08.857552 89 libfastertransformer.cc:1191]     output shape: [1, 1, 1098]
  156. fauxpilot-triton-1         | W0325 09:45:08.857583 89 libfastertransformer.cc:1161] Get output_tensors 1: sequence_length
  157. fauxpilot-triton-1         | W0325 09:45:08.857586 89 libfastertransformer.cc:1171]     output_type: INT32
  158. fauxpilot-triton-1         | W0325 09:45:08.857589 89 libfastertransformer.cc:1191]     output shape: [1, 1]
  159. fauxpilot-triton-1         | W0325 09:45:08.857603 89 libfastertransformer.cc:1206] PERFORMED GPU copy: NO
  160. fauxpilot-triton-1         | W0325 09:45:08.857608 89 libfastertransformer.cc:780] get response size = 1
  161. fauxpilot-triton-1         | W0325 09:45:08.857706 89 libfastertransformer.cc:795] response is sent
  162. fauxpilot-triton-1         | W0325 09:45:09.860855 89 libfastertransformer.cc:1397] model fastertransformer, instance fastertransformer_0, executing 1 requests
  163. fauxpilot-triton-1         | W0325 09:45:09.860905 89 libfastertransformer.cc:638] TRITONBACKEND_ModelExecute: Running fastertransformer_0 with 1 requests
  164. fauxpilot-triton-1         | W0325 09:45:09.860918 89 libfastertransformer.cc:693] get total batch_size = 1
  165. fauxpilot-triton-1         | W0325 09:45:09.860933 89 libfastertransformer.cc:1051] get input count = 16
  166. fauxpilot-triton-1         | W0325 09:45:09.860955 89 libfastertransformer.cc:1117] collect name: start_id size: 4 bytes
  167. fauxpilot-triton-1         | W0325 09:45:09.860977 89 libfastertransformer.cc:1117] collect name: input_ids size: 1624 bytes
  168. fauxpilot-triton-1         | W0325 09:45:09.860991 89 libfastertransformer.cc:1117] collect name: bad_words_list size: 8 bytes
  169. fauxpilot-triton-1         | W0325 09:45:09.861002 89 libfastertransformer.cc:1117] collect name: random_seed size: 4 bytes
  170. fauxpilot-triton-1         | W0325 09:45:09.861015 89 libfastertransformer.cc:1117] collect name: end_id size: 4 bytes
  171. fauxpilot-triton-1         | W0325 09:45:09.861028 89 libfastertransformer.cc:1117] collect name: input_lengths size: 4 bytes
  172. fauxpilot-triton-1         | W0325 09:45:09.861041 89 libfastertransformer.cc:1117] collect name: request_output_len size: 4 bytes
  173. fauxpilot-triton-1         | W0325 09:45:09.861054 89 libfastertransformer.cc:1117] collect name: runtime_top_k size: 4 bytes
  174. fauxpilot-triton-1         | W0325 09:45:09.861067 89 libfastertransformer.cc:1117] collect name: runtime_top_p size: 4 bytes
  175. fauxpilot-triton-1         | W0325 09:45:09.861079 89 libfastertransformer.cc:1117] collect name: is_return_log_probs size: 1 bytes
  176. fauxpilot-triton-1         | W0325 09:45:09.861092 89 libfastertransformer.cc:1117] collect name: stop_words_list size: 8 bytes
  177. fauxpilot-triton-1         | W0325 09:45:09.861104 89 libfastertransformer.cc:1117] collect name: temperature size: 4 bytes
  178. fauxpilot-triton-1         | W0325 09:45:09.861117 89 libfastertransformer.cc:1117] collect name: len_penalty size: 4 bytes
  179. fauxpilot-triton-1         | W0325 09:45:09.861130 89 libfastertransformer.cc:1117] collect name: beam_width size: 4 bytes
  180. fauxpilot-triton-1         | W0325 09:45:09.861145 89 libfastertransformer.cc:1117] collect name: beam_search_diversity_rate size: 4 bytes
  181. fauxpilot-triton-1         | W0325 09:45:09.861158 89 libfastertransformer.cc:1117] collect name: repetition_penalty size: 4 bytes
  182. fauxpilot-triton-1         | W0325 09:45:09.861171 89 libfastertransformer.cc:1130] the data is in CPU
  183. fauxpilot-triton-1         | W0325 09:45:09.861182 89 libfastertransformer.cc:1137] the data is in CPU
  184. fauxpilot-triton-1         | W0325 09:45:09.861205 89 libfastertransformer.cc:999] before ThreadForward 0
  185. fauxpilot-triton-1         | W0325 09:45:09.861351 89 libfastertransformer.cc:1006] after ThreadForward 0
  186. fauxpilot-triton-1         | I0325 09:45:09.861442 89 libfastertransformer.cc:834] Start to forward
  187. fauxpilot-triton-1         | I0325 09:45:27.665372 89 libfastertransformer.cc:836] Stop to forward
  188. fauxpilot-triton-1         | W0325 09:45:27.665556 89 libfastertransformer.cc:1161] Get output_tensors 0: output_ids
  189. fauxpilot-triton-1         | W0325 09:45:27.665575 89 libfastertransformer.cc:1171]     output_type: UINT32
  190. fauxpilot-triton-1         | W0325 09:45:27.665579 89 libfastertransformer.cc:1191]     output shape: [1, 1, 906]
  191. fauxpilot-triton-1         | W0325 09:45:27.665610 89 libfastertransformer.cc:1161] Get output_tensors 1: sequence_length
  192. fauxpilot-triton-1         | W0325 09:45:27.665617 89 libfastertransformer.cc:1171]     output_type: INT32
  193. fauxpilot-triton-1         | W0325 09:45:27.665620 89 libfastertransformer.cc:1191]     output shape: [1, 1]
  194. fauxpilot-triton-1         | W0325 09:45:27.665634 89 libfastertransformer.cc:1206] PERFORMED GPU copy: NO
  195. fauxpilot-triton-1         | W0325 09:45:27.665645 89 libfastertransformer.cc:780] get response size = 1
  196. fauxpilot-triton-1         | W0325 09:45:27.665752 89 libfastertransformer.cc:795] response is sent
  197. fauxpilot-triton-1         | W0325 09:45:28.671867 89 libfastertransformer.cc:1397] model fastertransformer, instance fastertransformer_0, executing 1 requests
  198. fauxpilot-triton-1         | W0325 09:45:28.671924 89 libfastertransformer.cc:638] TRITONBACKEND_ModelExecute: Running fastertransformer_0 with 1 requests
  199. fauxpilot-triton-1         | W0325 09:45:28.671937 89 libfastertransformer.cc:693] get total batch_size = 1
  200. fauxpilot-triton-1         | W0325 09:45:28.671955 89 libfastertransformer.cc:1051] get input count = 16
  201. fauxpilot-triton-1         | W0325 09:45:28.671977 89 libfastertransformer.cc:1117] collect name: start_id size: 4 bytes
  202. fauxpilot-triton-1         | W0325 09:45:28.672000 89 libfastertransformer.cc:1117] collect name: input_ids size: 6400 bytes
  203. fauxpilot-triton-1         | W0325 09:45:28.672013 89 libfastertransformer.cc:1117] collect name: bad_words_list size: 8 bytes
  204. fauxpilot-triton-1         | W0325 09:45:28.672024 89 libfastertransformer.cc:1117] collect name: random_seed size: 4 bytes
  205. fauxpilot-triton-1         | W0325 09:45:28.672038 89 libfastertransformer.cc:1117] collect name: end_id size: 4 bytes
  206. fauxpilot-triton-1         | W0325 09:45:28.672052 89 libfastertransformer.cc:1117] collect name: input_lengths size: 4 bytes
  207. fauxpilot-triton-1         | W0325 09:45:28.672066 89 libfastertransformer.cc:1117] collect name: request_output_len size: 4 bytes
  208. fauxpilot-triton-1         | W0325 09:45:28.672078 89 libfastertransformer.cc:1117] collect name: runtime_top_k size: 4 bytes
  209. fauxpilot-triton-1         | W0325 09:45:28.672091 89 libfastertransformer.cc:1117] collect name: runtime_top_p size: 4 bytes
  210. fauxpilot-triton-1         | W0325 09:45:28.672103 89 libfastertransformer.cc:1117] collect name: is_return_log_probs size: 1 bytes
  211. fauxpilot-triton-1         | W0325 09:45:28.672116 89 libfastertransformer.cc:1117] collect name: stop_words_list size: 8 bytes
  212. fauxpilot-triton-1         | W0325 09:45:28.672128 89 libfastertransformer.cc:1117] collect name: temperature size: 4 bytes
  213. fauxpilot-triton-1         | W0325 09:45:28.672140 89 libfastertransformer.cc:1117] collect name: len_penalty size: 4 bytes
  214. fauxpilot-triton-1         | W0325 09:45:28.672152 89 libfastertransformer.cc:1117] collect name: beam_width size: 4 bytes
  215. fauxpilot-triton-1         | W0325 09:45:28.672168 89 libfastertransformer.cc:1117] collect name: beam_search_diversity_rate size: 4 bytes
  216. fauxpilot-triton-1         | W0325 09:45:28.672196 89 libfastertransformer.cc:1117] collect name: repetition_penalty size: 4 bytes
  217. fauxpilot-triton-1         | W0325 09:45:28.672208 89 libfastertransformer.cc:1130] the data is in CPU
  218. fauxpilot-triton-1         | W0325 09:45:28.672217 89 libfastertransformer.cc:1137] the data is in CPU
  219. fauxpilot-triton-1         | W0325 09:45:28.672240 89 libfastertransformer.cc:999] before ThreadForward 0
  220. fauxpilot-triton-1         | W0325 09:45:28.672282 89 libfastertransformer.cc:1006] after ThreadForward 0
  221. fauxpilot-triton-1         | I0325 09:45:28.672449 89 libfastertransformer.cc:834] Start to forward
  222. fauxpilot-triton-1         | I0325 09:45:49.029485 89 libfastertransformer.cc:836] Stop to forward
  223. fauxpilot-triton-1         | W0325 09:45:49.029714 89 libfastertransformer.cc:1161] Get output_tensors 0: output_ids
  224. fauxpilot-triton-1         | W0325 09:45:49.029734 89 libfastertransformer.cc:1171]     output_type: UINT32
  225. fauxpilot-triton-1         | W0325 09:45:49.029739 89 libfastertransformer.cc:1191]     output shape: [1, 1, 2042]
  226. fauxpilot-triton-1         | W0325 09:45:49.029789 89 libfastertransformer.cc:1161] Get output_tensors 1: sequence_length
  227. fauxpilot-triton-1         | W0325 09:45:49.029798 89 libfastertransformer.cc:1171]     output_type: INT32
  228. fauxpilot-triton-1         | W0325 09:45:49.029801 89 libfastertransformer.cc:1191]     output shape: [1, 1]
  229. fauxpilot-triton-1         | W0325 09:45:49.029817 89 libfastertransformer.cc:1206] PERFORMED GPU copy: NO
  230. fauxpilot-triton-1         | W0325 09:45:49.029826 89 libfastertransformer.cc:780] get response size = 1
  231. fauxpilot-triton-1         | W0325 09:45:49.029977 89 libfastertransformer.cc:795] response is sent
  232. fauxpilot-triton-1         | W0325 09:45:50.035721 89 libfastertransformer.cc:1397] model fastertransformer, instance fastertransformer_0, executing 1 requests
  233. fauxpilot-triton-1         | W0325 09:45:50.035771 89 libfastertransformer.cc:638] TRITONBACKEND_ModelExecute: Running fastertransformer_0 with 1 requests
  234. fauxpilot-triton-1         | W0325 09:45:50.035784 89 libfastertransformer.cc:693] get total batch_size = 1
  235. fauxpilot-triton-1         | W0325 09:45:50.035803 89 libfastertransformer.cc:1051] get input count = 16
  236. fauxpilot-triton-1         | W0325 09:45:50.035826 89 libfastertransformer.cc:1117] collect name: start_id size: 4 bytes
  237. fauxpilot-triton-1         | W0325 09:45:50.035851 89 libfastertransformer.cc:1117] collect name: input_ids size: 5508 bytes
  238. fauxpilot-triton-1         | W0325 09:45:50.035863 89 libfastertransformer.cc:1117] collect name: bad_words_list size: 8 bytes
  239. fauxpilot-triton-1         | W0325 09:45:50.035875 89 libfastertransformer.cc:1117] collect name: random_seed size: 4 bytes
  240. fauxpilot-triton-1         | W0325 09:45:50.035889 89 libfastertransformer.cc:1117] collect name: end_id size: 4 bytes
  241. fauxpilot-triton-1         | W0325 09:45:50.035901 89 libfastertransformer.cc:1117] collect name: input_lengths size: 4 bytes
  242. fauxpilot-triton-1         | W0325 09:45:50.035916 89 libfastertransformer.cc:1117] collect name: request_output_len size: 4 bytes
  243. fauxpilot-triton-1         | W0325 09:45:50.035929 89 libfastertransformer.cc:1117] collect name: runtime_top_k size: 4 bytes
  244. fauxpilot-triton-1         | W0325 09:45:50.035943 89 libfastertransformer.cc:1117] collect name: runtime_top_p size: 4 bytes
  245. fauxpilot-triton-1         | W0325 09:45:50.035956 89 libfastertransformer.cc:1117] collect name: is_return_log_probs size: 1 bytes
  246. fauxpilot-triton-1         | W0325 09:45:50.035970 89 libfastertransformer.cc:1117] collect name: stop_words_list size: 8 bytes
  247. fauxpilot-triton-1         | W0325 09:45:50.035983 89 libfastertransformer.cc:1117] collect name: temperature size: 4 bytes
  248. fauxpilot-triton-1         | W0325 09:45:50.035996 89 libfastertransformer.cc:1117] collect name: len_penalty size: 4 bytes
  249. fauxpilot-triton-1         | W0325 09:45:50.036008 89 libfastertransformer.cc:1117] collect name: beam_width size: 4 bytes
  250. fauxpilot-triton-1         | W0325 09:45:50.036024 89 libfastertransformer.cc:1117] collect name: beam_search_diversity_rate size: 4 bytes
  251. fauxpilot-triton-1         | W0325 09:45:50.036036 89 libfastertransformer.cc:1117] collect name: repetition_penalty size: 4 bytes
  252. fauxpilot-triton-1         | W0325 09:45:50.036048 89 libfastertransformer.cc:1130] the data is in CPU
  253. fauxpilot-triton-1         | W0325 09:45:50.036058 89 libfastertransformer.cc:1137] the data is in CPU
  254. fauxpilot-triton-1         | W0325 09:45:50.036080 89 libfastertransformer.cc:999] before ThreadForward 0
  255. fauxpilot-triton-1         | W0325 09:45:50.036197 89 libfastertransformer.cc:1006] after ThreadForward 0
  256. fauxpilot-triton-1         | I0325 09:45:50.036365 89 libfastertransformer.cc:834] Start to forward
  257. fauxpilot-triton-1         | I0325 09:46:12.174991 89 libfastertransformer.cc:836] Stop to forward
  258. fauxpilot-triton-1         | W0325 09:46:12.175057 89 libfastertransformer.cc:1161] Get output_tensors 0: output_ids
  259. fauxpilot-triton-1         | W0325 09:46:12.175063 89 libfastertransformer.cc:1171]     output_type: UINT32
  260. fauxpilot-triton-1         | W0325 09:46:12.175068 89 libfastertransformer.cc:1191]     output shape: [1, 1, 1877]
  261. fauxpilot-triton-1         | W0325 09:46:12.175094 89 libfastertransformer.cc:1161] Get output_tensors 1: sequence_length
  262. fauxpilot-triton-1         | W0325 09:46:12.175101 89 libfastertransformer.cc:1171]     output_type: INT32
  263. fauxpilot-triton-1         | W0325 09:46:12.175104 89 libfastertransformer.cc:1191]     output shape: [1, 1]
  264. fauxpilot-triton-1         | W0325 09:46:12.175118 89 libfastertransformer.cc:1206] PERFORMED GPU copy: NO
  265. fauxpilot-triton-1         | W0325 09:46:12.175127 89 libfastertransformer.cc:780] get response size = 1
  266. fauxpilot-copilot_proxy-1  | Returned completion in 126056.41746520996 ms 1679737572.175964
  267. fauxpilot-triton-1         | W0325 09:46:12.175842 89 libfastertransformer.cc:795] response is sent
  268. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:46:12,176 :: 74.235.184.25:58080 - "POST /v1/completions HTTP/1.1" 200 OK
  269. fauxpilot-copilot_proxy-1  | Returned completion in 126056.7717552185 ms 1679737572.1765544
  270. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:46:12,176 :: 74.235.184.25:58092 - "POST /v1/completions HTTP/1.1" 200 OK
  271. fauxpilot-copilot_proxy-1  | Returned completion in 126056.95390701294 ms 1679737572.1769233
  272. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:46:12,177 :: 74.235.184.25:58096 - "POST /v1/completions HTTP/1.1" 200 OK
  273. fauxpilot-copilot_proxy-1  | Returned completion in 126057.10625648499 ms 1679737572.1772482
  274. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:46:12,177 :: 74.235.184.25:58100 - "POST /v1/completions HTTP/1.1" 200 OK
  275. fauxpilot-copilot_proxy-1  | Returned completion in 126057.24573135376 ms 1679737572.1775665
  276. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:46:12,177 :: 74.235.184.25:58104 - "POST /v1/completions HTTP/1.1" 200 OK
  277. fauxpilot-copilot_proxy-1  | Returned completion in 126057.34062194824 ms 1679737572.1778455
  278. fauxpilot-copilot_proxy-1  | INFO:     2023-03-25 09:46:12,177 :: 74.235.184.25:58114 - "POST /v1/completions HTTP/1.1" 200 OK
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!