Running warmup..
Timing with prompt_tokens=1024, generation_tokens=512, batch_size=1.
Trial 1:  prompt_tps=1538.942, generation_tps=86.573, peak_memory=20.660, total_time=6.585
Trial 2:  prompt_tps=1350.163, generation_tps=87.959, peak_memory=20.660, total_time=6.584
Trial 3:  prompt_tps=1371.898, generation_tps=86.726, peak_memory=20.661, total_time=6.655
Trial 4:  prompt_tps=1314.767, generation_tps=84.044, peak_memory=20.661, total_time=6.877
Trial 5:  prompt_tps=1291.455, generation_tps=87.038, peak_memory=20.661, total_time=6.679
Averages: prompt_tps=1373.445, generation_tps=86.468, peak_memory=20.661