Running warmup..
Timing with prompt_tokens=1024, generation_tokens=512, batch_size=1.
Trial 1:  prompt_tps=7642.487, generation_tps=130.282, peak_memory=3.514, total_time=4.071
Trial 2:  prompt_tps=7607.715, generation_tps=130.588, peak_memory=3.515, total_time=4.062
Trial 3:  prompt_tps=7680.694, generation_tps=130.475, peak_memory=3.516, total_time=4.065
Trial 4:  prompt_tps=7929.018, generation_tps=130.588, peak_memory=3.516, total_time=4.057
Trial 5:  prompt_tps=7778.941, generation_tps=130.387, peak_memory=3.517, total_time=4.065
Averages: prompt_tps=7727.771, generation_tps=130.464, peak_memory=3.516