Why batch training with maximum batchsize using PyTorch dataloader exhibits worse performance than inputing entire datasets to the networks?
While experimenting with PyTorch for neural network training, we encounter a choice: Should we load data in batches using PyTorch’s DataLoader, or should we input the entire dataset at once directly into the model (No GPU memory issues)? I was thinking that using DataLoader with a batch size equal to the entire dataset should mirror the performance of directly loading the full dataset. However, observations indicate otherwise.