I tried making a clay master once. I found it extremely difficult to control the shape, particularly maintaining symmetry. It is OK for making worms and simple rolled shapes. Also, clay can give problems when removing from the PoP, if the section is too thin.
I am currently using balsa. True, balsa is delicate and can be prone to break when removing from the PoP. I counter this by sealing well with 50/50 PVA (elmers). This adds rigidity, in addition to a polyester resin top coat.
If your master has legs, pincers etc, there is nothing wrong with making your master from several materials, each more suitable for the individual components of the lure.
More experienced members could help you more if you were to post a pic of the pattern that you are trying to make. If you don't want to spill the beans, then a more graphic explanation of the shape or nature of the components of the lure, eg fat body, thin tail, 3mm dia legs etc.
Also, if the initial clay pattern is not 100% correct to your design, car filler can be added and sanded using a dremel. Two part car filler is very convenient for this purpose, as it hardens very fast and is very strong, much stronger than the clay.
Post some pics and let the members help you.