In this investigation, a three-dimensional (3D) finite element model (FEM) was developed to study fretting wear of Hertzian circular and line contacts. The wear law incorporated in this model is based on the accumulated dissipated energy (ADE). A stress-based damage mechanics finite element model using the ADE was developed to determine wear of non-conformal bodies in contact. Voronoi tessellation was used to simulate the microstructure of the materials during the fretting process. In order to simulate the wear area in fretting contacts, a material removal approach was developed and implemented in the model. The FEM was used to investigate partial slip regimes under various operating conditions. The normal and shear surface tractions for the circular and line contacts were applied to the domain in order to improve the computational efficiency. The calculated wear volume rate using the FE model is in good agreement with the wear coefficient available in the open literature. The influence of modulus of elasticity, hardness, and coefficient of friction on the partial slip fretting phenomenon were studied. In order to verify the model, several fretting wear tests were conducted using AISI 8620 steel and AISI 1566 steel in a partial slip regime of circular contact configuration. The properties for each material such as the modulus of elasticity, hardness, and the grain size were measured experimentally and compared with the model. For the defined load and displacement amplitude of the experimental fretting tests, both materials have shown a partial slip behavior in the initial cycles and then transition to a gross slip regime. The numerical model predicted the worn surface and wear-rate in partial slip regime which corroborated well with these experimental test results.