Gear mesh = Number of teeth x shaft speed. Vibration analysis is a excellent tool for measuring gear mesh frequencys. You'll want to use acceleration measurments, and time waveform. It is important to set your frequency range (Fmax) high enough to see at least 3 times gear mesh frequency.
When teeth begin to wear, two things will happen. The first is the sidebands of gear mesh will increase in level. The sidebands will correspond to the speed of the gear with the wear.
The second thing that occurs is that the natural frequency of the gear is excited. this peak will also exhibit sidebands, and as a natural frequency, it is likely to have a broader base.
__________________
"You can't launch rockets out of a canoe."
Major concerns with gear box are broken / chipped teeth, high backlash, tooth wear, misalignment, eccentricity. Time waveform , spike energy, FFT analysis are best suited to vibration analysis of gear boxes. Proper installation procedures adherence is necessary for efficient and trouble free operation.