Batch size

[crisbodnar]

I see that this implementation (and all the others I've seen online) use a batch size of 16. The paper also mentions this as the default batch size they tried. Is there a reason why the batch size is kept so small? 32 or 64 would speed up learning. Have you tried bigger batch sizes?

[edubois]

Usually, small batch sizes avoid being stuck in local minima thanks to noisier gradients.