Unfortunately, drill bits are machine tool accessories and when hand driven tend to bend and break easily and the better their quality, more easily they break
A drill press is a good solution, but bending is not the only issue: plastic shavings get stuck in the process of drilling (also tend to melt at high speed) twisting the tool and again leading to breaks, so low speed, frequent extraction and cleaning from plastic the drill bit are good preventive measure (also when using a pin vice). Last but worthy, in my opinion: try with cheap, low quality bits: they are flexible and not hardened, it's difficult to start drilling and keeping straight, you can't drill metals, but they don't break easily.
I hope this can be useful, good luck. Ciao.
Giuseppe