This paper presents a finite element model (FEM) to investigate the effect of prior austenite grain refinement on rolling contact fatigue (RCF). RCF life was determined using continuum damage mechanics (CDM), which simulated material deterioration as a function of cycle. Continuum damage mechanics calculations in this investigation considered the subsurface shear (orthogonal) reversal to be responsible for RCF failure. To establish the CDM critical parameters—resistance stress (σr) and damage rate exponent (m)—torsion stress-life data from open literature of three different grain sizes for the same material was used. It was observed from the torsion S-N (stress-life) data that the resistance stress exhibits a linear relationship with grain diameter. As grain diameter was refined, the resistance stress was found to increase. The damage rate exponent (m) displayed no relation to grain diameter; hence, the average value from the three torsion S-N curves was used in this investigation. In order to assess the effect of grain refinement on RCF life, a series of unique material microstructures were constructed using the Voronoi tessellation process at eight mean grain diameters. Finite element (FE) simulations were devised at three contact pressures, typical of heavily loaded lubricated contacts, and the RCF life was determined for each set of microstructures of a given mean grain diameter. The RCF results at the eight grain diameters indicate that fatigue performance is improved exponentially with finer grain diameter. The observed life improvements from the RCF simulations resulting from grain refinement exhibit good corroboration with existing experimental results found in open literature. A single predictive fatigue life equation was constructed from this investigation’s RCF simulations to evaluate the stochastic RCF performance, given grain diameter and contact pressure, of non-conformal contacts.