Endpoint encountered an error.
You can try restarting it using the "retry" button above. Check  logs for more details.
[Server message]Endpoint failed to start
See details
Exit code: 1. Reason: n forward\n    out32, Sout32 = F.igemmlt(C32A, state.CxB, SA, state.SB)\n  File \"/opt/conda/lib/python3.11/site-packages/bitsandbytes/functional.py\", line 2295, in igemmlt\n    shapeA[-1] == shapeB[-1]\nAssertionError: Matmullt only supports A @ B^T. Inner matrix dimensions do not match: A @ B = torch.Size([1512, 2048]) @ torch.Size([3145728, 1])"},"target":"text_generation_launcher"}
{"timestamp":"2024-12-10T00:36:52.722801Z","level":"ERROR","message":"Server error: Matmullt only supports A @ B^T. Inner matrix dimensions do not match: A @ B = torch.Size([1512, 2048]) @ torch.Size([3145728, 1])","target":"text_generation_router_v3::client","filename":"backends/v3/src/client/mod.rs","line_number":45,"span":{"name":"warmup"},"spans":[{"max_batch_size":"None","max_input_length":"Some(1024)","max_prefill_tokens":1512,"max_total_tokens":"Some(1512)","name":"warmup"},{"name":"warmup"}]}
Error: Backend(Warmup(Generation("Matmullt only supports A @ B^T. Inner matrix dimensions do not match: A @ B = torch.Size([1512, 2048]) @ torch.Size([3145728, 1])")))
{"timestamp":"2024-12-10T00:36:52.792414Z","level":"ERROR","fields":{"message":"Webserver Crashed"},"target":"text_generation_launcher"}
{"timestamp":"2024-12-10T00:36:52.792434Z","level":"INFO","fields":{"message":"Shutting down shards"},"target":"text_generation_launcher"}
{"timestamp":"2024-12-10T00:36:52.849177Z","level":"INFO","fields":{"message":"Terminating shard"},"target":"text_generation_launcher","span":{"rank":0,"name":"shard-manager"},"spans":[{"rank":0,"name":"shard-manager"}]}
{"timestamp":"2024-12-10T00:36:52.849245Z","level":"INFO","fields":{"message":"Waiting for shard to gracefully shutdown"},"target":"text_generation_launcher","span":{"rank":0,"name":"shard-manager"},"spans":[{"rank":0,"name":"shard-manager"}]}
{"timestamp":"2024-12-10T00:36:54.050448Z","level":"INFO","fields":{"message":"shard terminated"},"target":"text_generation_launcher","span":{"rank":0,"name":"shard-manager"},"spans":[{"rank":0,"name":"shard-manager"}]}
Error: WebserverFailed