From memory earlier Fiats used a 8 amp fuse for cigar lighters, what amperage is your USB charger?
Early fuse holders were not very good and made worse when people used aluminium fuses instead of the original brass ones as they quickly corrode and make poor contact with the fuse holder.
If your charger needs more than that then maybe a separate circuit may be needed.