Twelve mortars were mixed and tested to explore the effect of gypsum on the compressive strength development and shrinkage strain of alkali-activated mortars. Powder typed sodium silicate and ground granulated blast-furnace slag were employed as alkaline activator and source material, respectively, to produce cementless mortar. The main variables investigated were alkali quality coefficient combining the concentration of activator and main compositions in source material, and the adding amount of gypsum ranged between 1 and 5% with respect to the weight of binder. Initial flow, compressive strength development, modulus of rupture, and shrinkage strain behavior of mortar specimens were measured. In addition, the hydration production of alkali-activated pastes with gypsum was traced using X-ray diffraction and energy-dispersive X-ray analysis combined with scanning electron microscope image. Test results showed that the initial flow of slag-based alkali-activated mortar was little influenced by the adding amount of gypsum. On the other hand, the effect of gypsum on the compressive strength of mortar specimens was dependent on the alkali quality coefficient, indicating that the compressive strength increased with the increase of the adding amount of gypsum until a certain limit, beyond which the strength decreased slowly. Shrinkage strain of mortar tested was little influenced by the adding amount of gypsum because no ettringite as hydration product was generated. However, the adding of gypsum had a beneficial effect on reducing the microcrack in the alkali-activated mortar.