In this paper, the mode coupling between bending, stretching, and torsional deformations is mainly studied by presenting an analytical model of a rotating cantilever beam with pre-twist angle and arbitrary cross section. Equations of motion of the beam are derived using Hamilton's principle. The Coriolis effect due to the coupling of the bending deformation and stretching deformation, the eccentricity caused by inconsistency between elastic center and centroid, spin softening effect, stress stiffening effect, shear deformation, and rotary inertia are included in the model. Equations of motion are solved by the Rayleigh-Ritz method. The natural frequencies obtained by the proposed analytical modal are in good agreement with those obtained by finite element method (FEM) which proved the accuracy of the analytical model. Finally, the coupling between different mode components is studied in detail based on a quantitative method. The transformation/conversion between different mode components is revealed, the influence of rotational speed, setting angle, and pre-twist angle on this conversion mode is studied. Results show that a specific mode shape is usually composed of multiple mode components. The essence of mode coupling is the coupling between different mode components. The influence of rotational speed, setting angle, and pre-twist angle on the mode coupling is that they cause the transformation/conversion between different mode components.