To explore the effects of different soil conditions on the quality and safety, nutritional quality and yield of ginseng fruit, six typical soils, including Wenshan mining area (treatment I), Honghe mining area (treatment II), a high background value area in Qujing (treatment III), a low risk area in Kunming (treatment IV), a medium risk area in Kunming (treatment V), a high risk area in Kunming (treatment VI), were selected to carry out pot experiment. The contents of soil nutrients and heavy metals were systematically determined, and their relationships with fruit quality (soluble solids, total acid, total sugar, vitamin C, amino acid, and calcium), safety index and yield were analyzed. The results showed that the content of heavy metal cadmium in soil of most treatments was higher than the specified value except treatment IV, the content of arsenic in soil of treatment Ⅵ exceeded the specified limit of heavy metal concentration, and the content of heavy metal in ginseng fruit was not detected to exceed the standard. The quality, soluble solids, total acids, total sugars, vitamin C, amino acids and calcium contents of ginseng fruit were determined by principal component analysis and correlation analysis. The results of fruit quality index analysis showed that total potassium and total acid, amino acid and total phosphorus, citric acid and total potassium were negatively correlated in different soil organic matter and fruit nutritional quality. The first three principal components were extracted by principal component analysis and the cumulative contribution rate was 81.524%. The comprehensive evaluation index model of ginseng fruit nutritional components under different soil conditions was F=49.537F1+31.987F2, and the comprehensive scores of Ⅴ (Kunming 2), Ⅳ (Kunming 1) and Ⅲ (Qujing) were the highest, and the quality was the best. There was no significant difference in yield and fruit weight among the six treatments, and the difference in fruit weight between III (Qujing) and VI (Kunming 3) was the largest. By measuring, analyzing and studying the effects of different soil background values on the quality, safety, nutritional quality and yield of ginseng fruit, it is helpful to reveal the internal relationship between soil factors and the growth and development of ginseng fruit, provide theoretical basis and practical guidance for formulating scientific planting management measures, help to improve the yield and quality of ginseng fruit, ensure food safety, and promote the sustainable development of agriculture.