You can make a bait by hand, and use rasps, files, sandpaper, and a dovetail saw or dremel tool to cut slots and drill holes, but, to make four or five at a time, power tools are the only way to go.
A lot depends on what kind of lures you're making.
If you're making poppers and cylindrical lures, a lathe is probably all you'd need, along with a cordless drill.
For cranks and swimbaits, it gets a little more complicated.
If you're rough cutting blank stock and cutting joints and lip slots, a small table saw. You could use a radial arm saw, but a table saw is much safer for most things.
If you're predrilling for hook hangers, weighting, and hinge joints, a drill press. A cordless drill would work, but a drill press is much more accurate.
If you're rough cutting to shape, a small bandsaw. This is to cut down on sanding.
If you're refining the shape, to get to ready to paint, an oscillating sander. I have a combo spindle/belt sander. I only use the belt part. Depending on the belt, I can rough out a blank in about two minutes, and refine the shape and finish sand in another two minutes. I use a dust collector with my sander, and it's still dusty work. A mask and back fan are essential for me.
Choose your poison.