Possible Lorentz invariance violation (LIV) has been investigated for a long time based on observations of GRBs. These arguments relied on the assumption that photons with different energy are emitted at the same place and time. In this work, we try to take account of the intrinsic time delay Delta t(int) between emissions of low and high energy photons by using the magnetic jet model. The possible LIV effects are discussed in a unified scenario both for long and short Fermi-detected GRBs. This leads to a unique quantum gravity energy scale M(1)c(2) similar to 1.0 x 10(20) GeV respecting the linear dispersion relation. (C) 2012 Elsevier B.V. All rights reserved.