Running warmup..
Timing with prompt_tokens=1024, generation_tokens=512, batch_size=1.
Trial 1:  prompt_tps=1487.092, generation_tps=94.126, peak_memory=17.820, total_time=6.133
Trial 2:  prompt_tps=1407.693, generation_tps=94.080, peak_memory=17.820, total_time=6.175
Trial 3:  prompt_tps=1348.598, generation_tps=94.021, peak_memory=17.821, total_time=6.210
Trial 4:  prompt_tps=1369.096, generation_tps=93.845, peak_memory=17.821, total_time=6.208
Trial 5:  prompt_tps=1395.196, generation_tps=93.198, peak_memory=17.821, total_time=6.232
Averages: prompt_tps=1401.535, generation_tps=93.854, peak_memory=17.821