Running warmup..
Timing with prompt_tokens=1024, generation_tokens=512, batch_size=1.
Trial 1:  prompt_tps=1733.512, generation_tps=80.614, peak_memory=15.056, total_time=6.949
Trial 2:  prompt_tps=1647.329, generation_tps=80.654, peak_memory=15.056, total_time=6.977
Trial 3:  prompt_tps=1488.132, generation_tps=80.734, peak_memory=15.057, total_time=7.036
Trial 4:  prompt_tps=1456.780, generation_tps=80.669, peak_memory=15.057, total_time=7.057
Trial 5:  prompt_tps=1485.254, generation_tps=80.606, peak_memory=15.058, total_time=7.049
Averages: prompt_tps=1562.202, generation_tps=80.656, peak_memory=15.057