Larger fees. You can't keep your gold at home or in the financial institution's safe deposit box. In its place, you have to pay a custodian to store and insure, as well as purchase, ship, and transport, the precious metals you maintain while in the IRA. Gold IRA custodial fees are usually increased than typical IRA management fees, too. Birch Gold