This paper studies the effect of covariance regularization for high dimensional data for classification. This is done primarily by fitting a mixture of Gaussians where the covariance matrix is regularized to each class. Three data sets from various domains are used to suggest the results are applicable to any domain where covariance regularization is required. The regularization needs of the data when pre-processed using two dimensionality reduction techniques, the popular principal component analysis (PCA) and the up-and-coming random projection, are also compared. Observations include that using a large amount of covariance regularization consistently provides classification accuracy as good if not...