A friction pendulum experiment equipment was introduced, and a movement differential equation of this friction pendulum system in its stable swing state was established. The effect of relative angular velocity and relative angular acceleration of the friction pairs on the stability of self-excited vibration of the friction pendulum was calculated and discussed. The result shows that a variable friction controlled by the relative acceleration is the primary factor causing the self-excited vibration of system. So the effective control method of friction self-excited vibration in engineering practice is to make the friction between the friction pairs not to change with the relative velocity, but not to fluctuate with the relative acceleration.