The flat fall jigs need to have the line straight down in the water. Anything more than 10-15 degrees out of vertical and they don't produce the action they're designed to - no more reaction bites. If your Speed Over Ground on GPS is more than about 0.3 - 0.4 mph, the line will have too much angle.
Solutions:
- wait for a calm day
- wait for slack tide
- anchor over a bait ball and hope it stays put
- use heavier jigs
- use the kicker in reverse to back troll into the wind/current, may require shifting in and out of gear
- get a MinnKota electric trolling motor with 'virtual anchor' mode
- fish shallower (less time for boat drift to pull the line sideways)
I have some 5 oz and 6 oz Spinnows and found they fished better once I switched over to the double assist hook setup that the Shimano jigs use. The assist hooks are sold in pairs already rigged and ready to install on the jig of your choice, available in either 5/0 or 7/0.