Running warmup..
Timing with prompt_tokens=1024, generation_tokens=512, batch_size=1.
Trial 1:  prompt_tps=3409.194, generation_tps=80.053, peak_memory=5.078, total_time=6.706
Trial 2:  prompt_tps=3468.612, generation_tps=81.251, peak_memory=5.079, total_time=6.606
Trial 3:  prompt_tps=3304.964, generation_tps=80.468, peak_memory=5.080, total_time=6.683
Trial 4:  prompt_tps=3293.373, generation_tps=81.114, peak_memory=5.080, total_time=6.632
Trial 5:  prompt_tps=3289.509, generation_tps=79.399, peak_memory=5.081, total_time=6.770
Averages: prompt_tps=3353.131, generation_tps=80.457, peak_memory=5.080