"I work at Visa. It's not the credit card company you deal with it's the bank. Banks will place a hold on any disputed billing. Just simply call your bank and they will investigate. If it is in fact a fraudulent charge the charge is removed. If they have evidence that says otherwise they will provide proof you made the purchase."
The above from troutbum, and to verify his statement: my bank (Chase) recently called me to question a suspicious charge on my debit Visa card. Sure enough, it was bogus, I had been scammed! The bank immediately refunded the $200 the scammer had managed to withdraw, cancelled my card and mailed me a new one.
The bank will trust you unless there is evidence to show you did make the purchase or the charge, says my local branch manager. He also said that it is not a matter of "if" you will be scammed, it's "when", because virtually everyone gets scammed eventually, but if you catch it in a timely fashion, or the bank does, your money is returned, the charge denied.
So...let's worry about something else, like North Korea nuking us.