If the paint is coming off in sheets it means the clear is adhering to the paint just fine unless the clear is peeling separately from the paint....it's the paint to primer adhesion or the primer to wood that is the problem. Most likely the primer to wood.
Reason being is that each layer adheres to the previous...top coat to paint, paint to primer, primer to sealer, sealer to wood. Where it stops peeling is where the problem is. If you can't see primer anymore then it's primer to wood. If you lose topcoat and paint but the primer stays on the wood then it's paint to primer adhesion which is bad. The weakest adhesion point is where it happens.
Most times when you see a paint adhesion problem like this where it comes off in sheets you can usually find that the lure swelled from water intrusion....when it swells it no longer allows the coating to adhere due to the new larger size and it comes off in sheets like you describe.
A few tests you can do....measure the diameter of a lure that has paint on it, then dunk it in water....then measure it again. If it swells significantly then it's absorbing water and you need to change your sealer.
Also take a lure and cut a crosshatch pattern into it...4-5 lines in each direction like a tic tac toe board...then cover it with a real strong packing type of tape to simulate ASTM adhesion testing tape....rub the tape down so you know it's sticking good....then pull the tape up....good adhesion shouldn't allow much if any paint to come up on the tape....if your tape is full of paint etc then there's your problem. What comes up on the tape will also give you a clue where the adhesion problem is.