Food safety is directly related to human health and has attracted intense attention all over the world. Surface-enhanced Raman scattering (SERS), as a rapid and selective technique, has been widely applied in monitoring food safety. SERS substrates, as an essential factor for sensing design, greatly influence the analytical performance. Currently, nanostructure-based SERS substrates have garnered significant interest due to their excellent merits in improving the sensitivity, specificity, and stability, holding great potential for the rapid and accurate sensing of food contaminants in complex matrices. This review summarizes the fundamentals of Raman spectroscopy and the used nanostructures for designing the SERS platform, including precious metal nanoparticles, metal–organic frameworks, polymers, and semiconductors. Moreover, it introduces the mechanisms and applications of nanostructures for enhancing SERS signals for monitoring hazardous substances, such as foodborne bacteria, pesticide and veterinary drug residues, food additives, illegal adulterants, and packaging material contamination. Finally, with the continuous progress of nanostructure technology and the continuous improvement of SERS technology, its application prospect in food safety testing will be broader.