The high polarization observed in the prompt phase of some gamma-ray bursts invites extensive study of the emission mechanism. In this paper, we investigate the polarization properties of the synchrotron self-Compton (SSC) process from a highly relativistic jet. Amagnetic-dominated, baryon-loaded jet ejected from the central engine travels with a large Lorentz factor. Shells with slightly different velocities collide with each other and produce shocks. The shocks accelerate electrons to a power-law distribution and, at the same time, magnify the magnetic field. Electrons move in the magnetic field and produce synchrotron photons. Synchrotron photons suffer from the Compton scattering (CS) process and then are detected by an observer located slightly off-axis. We analytically derive the formulae of photon polarization in the SSC process in two magnetic configurations: a magnetic field in the shock plane and perpendicular to the shock plane. We show that photons induced by the SSC process can be highly polarized, with the maximum polarization Pi similar to 24% in the energy band [0.5, 5] MeV. The polarization depends on the viewing angles, peaking in the plane perpendicular to the magnetic field. In the energy band [0.05, 0.5] MeV, in which most gamma-ray polarimeters are active, the polarization is about twice that in the Thomson limit, reaching Pi similar to 20%. This implies that the Klein-Nishina effect, which is often neglected in the literature, should be carefully considered.