We present a model of a primary locus subject to viability selection and an unlinked locus that causes sex-specific modification of the segregation ratio at the primary locus. If there is a balanced polymorphism at the primary locus, a population undergoing Mendelian segregation can be invaded by modifier alleles that cause sex-specific biases in the segregation ratio. Even though this effect is particularly strong if reciprocal heterozygotes at the primary locus have distinct viabilities, as might occur with genomic imprinting, it also applies if reciprocal heterozygotes have equal viabilities. The expected outcome of the evolution of sex-specific segregation distorters is all-and-none segregation schemes in which one allele at the primary locus undergoes complete drive in spermatogenesis and the other allele undergoes complete drive in oogenesis. All-and-none segregation results in a population in which all individuals are maximally fit heterozygotes. Unlinked modifiers that alter the segregation ratio are unable to invade such a population. These results raise questions about the reasons for the ubiquity of Mendelian segregation.