Try higher batch size with module limited training #7
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Large batch size, especially if it fits the entire dataset, might have a profound effect on training with reduced blocks/roles/modules. Try this out and see what happens. Also it'd be good to figure out how much VRAM we're saving.