The good lubrication performance of the nuclear power gearbox is an important guarantee for the reliable operation of the nuclear power circulating pump,and the fully considering the influence of tooth surface morphology and tooth profile modification on internal and external meshing gear pairs is a prerequisite for accurately evaluating lubrication characteristics.The thermal elastohydrodynamic lubrication model of the helical gears of the planetary transmission system in the nuclear power circulating pump was established under typical working conditions.The meshing state of the helical gear pair was geometrically equivalent to the contact problem of a conical roller,and by considering the contact deformation and tooth profile bias of the helical gears,the oil film thickness,lubrication pressure,friction stress,and flash temperature at different positions in the contact area of internal and external meshing gear pairs were calculated.Considering the tooth surface running-in effect,the moving average filtering method was used to smooth the initial morphology without running-in,and the influence of the tooth surface topography on the lubrication behavior was analyzed.The tooth profile was modified to improve the lubrication characteristics.The results show that surface roughness and tooth profile modification have significant influences on lubrication performance,and the rough morphology of the tooth surface can cause a decrease in oil film thickness,so as to affect the lubrication characteristics and reduce the load-bearing capacity of the oil film.By modifying the tooth profile,the geometric transition at the gear meshing boundary can be improved,the stress concentration and surface temperature in this area can be reduced,and the lubrication state at the end of the meshing line can be significantly improved.