The reliability of submersible motors is directly related to the operational life of offshore oil-gas resources exploration and development equipment, crude oil production, and drilling costs. Temperature field model and coupling mode are the key factors for accurate analysis of cooling system. First, considering different equivalent modes of convective heat transfer of housing surface, two temperature field models are established using the finite volume method (FVM). Second, considering the influence of temperature and magnetic field, the coupling mode between heat loss (HL) and temperature of submersible motor is analyzed, and the importance of oil friction loss and copper loss as variables is pointed out. Considering the influence relationship among the components of the cooling system, the coupling mode between the radiator temperature field and the cooling system internal flow field is obtained. For a simulated circulation flow, the necessity of rotor iron loss calculation under the orthogonal decomposition correction model and rotor loss analysis is explained. Finally, the design procedure of the cooling system is proposed to obtain the design range between the structural parameters of the radiator and the circulation flow. Through the analysis of the influence factors of the cooling system, the matching mode of the same structural parameters under the two models is determined. These models are experimentally validated, and a more reasonable model is obtained under the proposed coupling mode in this paper, which can provide necessary insights for future research on submersible motor cooling systems.