Microwave imaging offers excellent potential for breast cancer detection. Deep learning is state-of-the-art in biomedical imaging, which has been successfully applied for biomedical image classifications. This paper investigates a deep neural network (DNN) based classification method for identifying breast lesion in holographic microwave image (HMI). A computer model is developed to demonstrate the proposed method under practical consideration. Various experiments are carried out to evaluate the proposed DNN-based HMI for breast lesion classification. Results have shown that the proposed method could serve as a helpful imaging tool for automatically classifying different types of breast tissues.