A drill press, or milling machine, is very useful, but the main thing they do is keep the drill aligned - if you use a pin vice by hand, any twitch will snap a drill. Likewise, a power tool has no 'feel', the other killer. Solution, use a pin vice by hand; but put the pin vice in a drill press or equivalent to keep it aligned - but loose in the chuck, powered by hand. The drill press chuck will prevent hand twitches from snapping drills, and your feel will reduce the breakage from jamming. Of course since you're only using the drill press for alignment, you can cobble up something out of wood or whatever to avoid shelling out for the machine; just a hole to loosely fit the shank of the pin vice is all you need.
Of, course breaking those teeny drills is inevitable, but you can have some fun lowering the attrition.
- Bill