Grassland recovery from degradation is increasingly occurring worldwide. Diverse managements have been considered as effective ways to restore degraded grassland, but it remains unclear how semi-arid grasslands respond to long-term grazing exclusion and fenced mowing. Here, a study was conducted under open grazing, grazing exclusion and fenced mowing in a semi-arid grassland on the Loess Plateau. We measured plant species composition and diversity, plant production, surface litter and soil water and carbon content. Shifts in grassland management led to significant divergence in plant community composition. Long-term grazing exclusion (35 years) significantly increased plant biomass, surface litter, soil water and carbon storage, but suppressed plant diversity compared to open grazing. Conversely, fenced mowing significantly increased plant diversity accompanying with a weak effect on soil carbon. Moreover, mowing significantly reduced surface litter and soil moisture, which have strong implications for nutrient depletion and soil drying. Our results suggest that introducing disturbances are necessary to safeguard biodiversity, and continuous mowing (5 years) belongs to over exploitation of the long-term protected grassland. Therefore, it is essential to optimize management with dual objectives of biodiversity and soil carbon sequestration in the future.