Sweet potato is one of the essential starch materials for food and feed world widely. However, its application is limited by the low viscosity and poor stability of native starches. Herein, sweet potato starch is modified by octenyl succinic anhydride (OSA) in the aqueous system. The optimization for esterification is studied using response surface methodology. The characteristics of OSA-modified sweet potato starch are investigated, and its emulsifying properties as macromolecular and particulate stabilizers in the oil-in-water systems are compared. The optimal process conditions for OSA modification are: reaction time 4.0 h, temperature 38.8 °C, pH of the reaction system 8.2, starch concentration 32.2%, OSA amount 3.0%. The degree of substitution is 0.0185, and reaction efficiency is 79.7% under the optimum conditions. The new peaks at 1725 and 1573 cm⁻¹ are observed by Fourier transform infrared spectroscopy. Compared with native starch, OSA-modified sweet potato starch exhibits higher viscosities, improved paste clarity, and decreased retrogradation. The confocal laser scanning microscope results show that the gelatinized starch macromolecule and ungelatinized starch particles can arrange at the water–oil interface. The OSA-modified sweet potato starch is both an effective macromolecular emulsifier and a particulate stabilizer, effectively stabilizing oil-in-water emulsions.