He was a gold glover because of his bat, as many GG winners were over the years. Sure, the award is supposed to be about defense only, but it's been obvious to any fan how many times the bat was what the vote was about.
Edit to add: he wasn't "a gold glover early on" - he won a GG the season before the Cardinals traded for him, and only that season. That's it.
Raphael Palmiero won a GG at first base for a season in which he played fewer than 30 games(28) at the position(first base) - 1999, look it up. There are other examples, but that one takes the cake.
In short, if you refuse to look at advanced defensive metrics to judge defense, then don't bring up a [shirt]ty fielder winning a GG.