Puzzletron Progress 6/8 (calculating one block scores) takes 10 to 20 times more than in tutorial

[danielkorzekwa]

ModelOpt: release/0.44.0

Running torchrun --nproc_per_node 2 examples/puzzletron/main.py --config examples/puzzletron/configs/llama-3_1-8B_pruneffn_memory/llama-3_1-8B_pruneffn_memory.yaml 2>&1 | tee ./log.txt | grep "Puzzletron Progress"

takes 10 to 20 times longer than stated in the tutorial: https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/puzzletron

it is due to scoring.eval_samples: 128 in the examples/puzzletron/configs/llama-3_1-8B_pruneffn_memory/Llama-3_1-8B.yaml

suggestions:

• adjust tutorial and config file
• provide a better progress bar to indicate the remaining time