The following was tested with Arduino 1.8.5.
It doesn't matter if a sketch is loaded in the editor.
Plug the USBtiny compatible in to your computer via USB. Then for each ATMEGA328P-PU you want to burn the bootloader onto:
Repeat the above as needed for additional ICs.