Q: Swirl marks made from semi abrasive sponge.

I washed my Jeep Grand Cherokee and on parts with tough stuff or sap i used a semi abrasive sponge. It worked great and left no marks. Then a few days later I do the same thing on the Volvo but now there are scratches Ina swirl pattern on the parts where there were really sticky stuff. I used kit scratch out in it. I got a sock and poured some on th sock and rubbed on in a circular motion. Then when dry I rub off with a new sock. It looks awesome for all of 30 minutes and it comes back. I love cars and am so anal about its conditions. There are probably 4-5 noticeable spots all over the once perfect silver paint job. Do I need a different brand of scratch out? A different technique? Multiple layers? PLEAE HELP!!!!! This needs to be fixed. Thanks!!

My car has 146000 miles.
My car has an automatic transmission.