Running warmup..
Timing with prompt_tokens=1024, generation_tokens=512, batch_size=1.
Trial 1:  prompt_tps=1907.068, generation_tps=82.960, peak_memory=12.689, total_time=6.715
Trial 2:  prompt_tps=1766.014, generation_tps=82.832, peak_memory=12.690, total_time=6.768
Trial 3:  prompt_tps=1756.366, generation_tps=82.401, peak_memory=12.690, total_time=6.803
Trial 4:  prompt_tps=1745.797, generation_tps=82.419, peak_memory=12.690, total_time=6.805
Trial 5:  prompt_tps=1735.079, generation_tps=82.428, peak_memory=12.691, total_time=6.808
Averages: prompt_tps=1782.065, generation_tps=82.608, peak_memory=12.690