An elasto-plastic pendulum performing large rigid-body rotations and vibrating in the small-strain regime is studied. Spatial distribution of plastic zones within the beam-type pendulum is taken into account. The problem is described by a differential algebraic system of equations for the flexural coordinates and the rotation of the pendulum. This system of equations can be interpreted as a model for an elastic background pendulum under the action of additional sources, formed by the plastic parts of strain. Since the elastic pendulum is a Hamiltonian system, it is possible to control the motion of the elastic pendulum by means of a collocated PD-controller. Especially, we consider the problem of bringing the tip of the elastic pendulum into its upward (inverted) position by means of a control moment acting at the fixed end. We then apply the controller designed for the elastic pendulum to the elasto-plastic model, assuming that the yield level of the material was lowered considerably by some environmental influences. Since the effect of plasticity is dissipative, the controlled elasto-plastic pendulum turns out to reach an equilibrium position, which however does not exactly coincide with the upward target position. This deviation is due to the permanent deformations induced in the pendulum by plasticity, and it is demonstrated in a numerical study.