If your rear bearing isn't close, it's not doing it's job.
Remember what the bearings are for. They are to keep the blade from twisting or wandering while you're cutting. They aren't necessary for your blade to stay on the wheel when the saw is running with no load from a work piece. If you need the bearings to keep you blade on your wheels you don't have the saw adjusted properly.
I only use a roller bearing behind my blade.
Because the blade you want to use is so small, there is very little flat area behind the teeth for the bearings to ride on, so metal bearings wind up getting scared up by the blade, and the blade dulls quickly.
I use fiber blocks on the sides with 1/8" to 1/4" blades
I use metal bearings for wider blades.
I adjust the rear bearing so it just clears the blade when the saw is running, and comes into contact with the back of the blade when I apply pressure with the work piece. Then I move the side bearings until they are also just clear of the blade when it's running. I'm talking about the thickness of two pieces of paper. Constant contact with the rear bearing will make your blade wander, because it is being pushed forward on the wheels.
For lure making, 1/4" is the widest blade I use, because the wider the blade, the wider the turning radius.
So thinner blades make it easier to follow contour lines.
Once you get your saw set up right, you shouldn't need to adjust it again until you change your blade.
I remove the tension on the blade after each session, and retension it when I'm going to use the saw again. That saves both the blade, and the wheel tires.