Do you clip the dummy right to the downrigger ball, and about 3-4ft of thick mono?
How far up the cable do you run the lure/bait? Do you go just high enough up the cable to exceed the the length of the line on the dummy? And how many pulls is typical before clipping on?
