Hi, thank you for your great work and for releasing the code.
We are trying to reproduce the frozen-backbone results on LIBERO-Long. In our experiments, we froze all layers except the ActionQuery and the action head, and followed the hyperparameters reported in the paper as closely as possible.
However, our reproduced performance is still far below the reported 86.4% success rate on LIBERO-Long. Several tasks, such as task 8 and 6, achieve less than 50% success rate. Could you please advise what might be the possible reasons for this gap?
We would also like to ask whether training with 8 GPUs and 4 GPUs is equivalent if both use a batch size of 16 and train for 15,000 steps.
Thank you very much for your guidance.
Hi, thank you for your great work and for releasing the code.
We are trying to reproduce the frozen-backbone results on LIBERO-Long. In our experiments, we froze all layers except the ActionQuery and the action head, and followed the hyperparameters reported in the paper as closely as possible.
However, our reproduced performance is still far below the reported 86.4% success rate on LIBERO-Long. Several tasks, such as task 8 and 6, achieve less than 50% success rate. Could you please advise what might be the possible reasons for this gap?
We would also like to ask whether training with 8 GPUs and 4 GPUs is equivalent if both use a batch size of 16 and train for 15,000 steps.
Thank you very much for your guidance.