I am training the plane fully-connected neural nets and I noticed that after interrupting it the character of training changes. I interrupt it to check the results. And then I continue the process, but sometimes it reaches a plateau immediately after interrupting.

Why it has place and is it a good way to interrupt training to check the results and then continue the process?
I use the next code for training:

iters = 10000
optimizer=torch.optim.LBFGS(model.parameters(), lr=0.001)
def train():
    for step in iters:
        def closure():
            optimizer.zero_grad()
            loss  = model(input)
            loss.backward()
            return loss
        optimizer.step(closure)
        if step % 2 == 0:
            current_loss = closure().item()
            pbar.set_description("Step: %d | Loss: %.6f" %
                                 (step, current_loss))
train()

Thanks for the answer!