The aerodynamic and aeroelastic behavior of an engine-like transonic compressor stage is investigated in this paper. Simulations were carried out for the transonic compressor at the Technical University of Darmstadt using the AU3D flow solver. A comparison with previous experimental investigations shows close agreement for both steady and transient data. The simulations enable a detailed flow analysis during spike stall inception and nonsynchronous vibrations (NSV) at operating points close to the stall limit. The aerodynamic structure of shear layer fluctuations in the rotor tip region during the initial phase of spike stall inception is investigated. Already before the stall limit is reached, such fluctuations can occur. In this case, the compressor continues to operate on the aerodynamically stable characteristic and NSV might be excited due to the unsteady blade force. This paper concludes with a joint analytical interpretation of the shear layer fluctuations observed at the beginning of spike stall and during near stall NSV by basic flow principles.