This technical brief proposes a defect recognition model to recognize four typical defects of phased array ultrasonic testing (PA-UT) images for electrofusion (EF) joints. PA-UT has been proved to be the most feasible way to inspect defects in EF joints of polyethylene pipes. The recognition of defects in PA-UT images relies on the experience of operators, resulting in inconsistent defective detection rate and low recognition speed. The proposed recognition model was composed of an anomaly detection model and a defect detection model. The anomaly detection model recognized anomalies in PA-UT images, meeting the requirement of real-time recognition for practical inspection. The defect detection model classified and located defects in abnormal PA-UT images, achieving high accuracy of defects recognition. By comparing detection models, optimizing parameters and augmenting dataset, the anomaly detection model and defect detection model reached a good combination of accuracy and speed.