The electrodynamics with a Chern-Simons term p(mu)A(nu)(F) over tilde (mu nu) violates Lorentz and CPT symmetries with a non-vanishing p(mu). For a fixed vector p(mu), in this Letter we point out that the energy-momentum tensor of this theory coupled to the gravity minimally is symmetric but not divergence free, which consequently makes the gravitational field equation inconsistent. To preserve the consistency, we introduce a Chern-Simons term in the gravity sector with the coefficient determined by the Lorentz and CPT violating term in the electromagnetic field. Further we study the phenomenologies of the model oil the cosmic microwave background radiation and the relic gravitational waves. (C) 2009 Elsevier B.V. All rights reserved.