In a deep drawing process, the contact sliding between the surfaces of a metal sheet and a die/mold under high contact stresses brings about wear of both the counterpart surfaces. This paper aims to investigate the influence of tool and workpiece properties on the wear behaviors subjected to cylinder-on-flat sliding conditions. The effects of different tool materials (Cr12MoV, SKD11, and DC53), radii, surface hardness, and surface treatment processes including thermal diffusion (TD) coating, nitriding coating, and vacuum heat treatment (VHT) were investigated. Two types of workpieces with high tensile strength (DP980 and QP980) were also applied in the tests. The friction coefficient, surface morphology, and material removal were analyzed with the aid of load cells installed and microscopy measurements. It was found that the DC53 tool experiences the smallest tool wear and material transfer. The least material transfer is attributed to its low adhesion to workpiece material, which also results in more wear debris and a higher friction coefficient. The TD coated tool surface can further improve the wear resistance and bring about the lowest friction coefficient. The debris density associated with the TD coating is also the smallest. These findings provide a useful guideline for the design and optimization of molds/dies for the deep drawing processes of sheet metals.