Very (very) unlikely. Pure gold and silver minerals are hypoallergenic. That is 24K/999 gold and 999 silver. If your skin is reacting to jewelry, it is most likely reacting to the non-precious metal in the gold or silver alloy your piece is made of.
So, if you are noticing discomfort when wearing a sterling silver piece, you may be allergic to copper or zinc, as sterling silver is 92.5% pure fine silver and 7.5% other metals, copper being the most common.
If your skin is reacting to gold jewelry, it is most likely caused by one of the non-precious metals in the gold alloy – copper, zinc, nickel, and others. Nickel is the most common culprit for allergic reactions, and it is used in most white gold alloys. Copper, zinc, or palladium allergies are very rare. If you are white gold fan, you may find this information useful.
Any reputable jeweler must be able to tell you the exact content of their gold and silver alloys. It is not common for us to clutter every product listing with that information because too many technical details can be overwhelming for customers. But it should always be available upon request.