I only just last week tried the Duragloss cleaner on anything, and yes, I tested a spot on the old caulk which is what turned gummy. It didn't clean the stains from the silicone caulk, but didn't seem to damage it either. It cleaned up the mildew spots on the old caulk pretty well, but at a cost.
The dull paint on the frame is fortunately only something you'd notice with really close inspection, and I think some wax will shine it back up. Or worst case, a little buffing. I'm not worried about it, but of course wouldn't do it again. I asked because it occurred to me that it would be difficult to re-caulk around the windows with the gutters in place.