Puzzling magnetic data on the Zn1-xMnxO system such as a small magnetization values or a large negative values of the Curie-Weiss temperature have been obtained in many experimental investigations. Here we report element-specific structural and magnetic investigations on a high-quality Zn0.95Mn0.05O nanocrystalline sample. Combining low-temperature x-ray absorption spectroscopy and theoretical simulations, we show that the formation of substitutional spin-antiparallel pairs induces a large local distortion involving a contraction of the Mn-Mn distance and a reduced Mn-O-Mn bond angle. The first-principles calculation considering hole-doping reveals that such a distortion can result in a localized hole around a dopant atom, generating a ferrimagnetic ordering with a magnetization of 0.45 mu(B)/Mn. This result may give a new insight for a better understanding of the reported magnetic data.