Thanks for your code! I'd like to know how to calculate the total_batch_size? Is total_batch_size = batch_size(config.yaml) * gpus * accumulation_steps?
But in your example of trainning for SSV2, you set nproc_per_node=8, accumulation-steps =8, and in config.yaml batch_size=8, it can not calculate 256 for total_batch_size
Thanks for your code! I'd like to know how to calculate the total_batch_size? Is total_batch_size = batch_size(config.yaml) * gpus * accumulation_steps?
But in your example of trainning for SSV2, you set nproc_per_node=8, accumulation-steps =8, and in config.yaml batch_size=8, it can not calculate 256 for total_batch_size