From the moment we are born, we are exposed to a vast variety of microbes. The intestine in particular is perhaps inhabited by the largest number of microbes, consisting of both established commensals as well as sporadic pathogens. Mucosal surfaces form an important barrier against microbial invasion. Together with the physical barrier that they provide, mucosal surfaces also rely on innate immune functions to sense luminal microbes and signal accordingly to generate protective immune responses. However, since innate immune recognition is microbial specific and antigen-independent, the contact with both beneficial commensals and harmful pathogens creates the need for discrimination between the two. The mechanisms governing the ability of the mucosal immune system to discriminate between commensals and pathogens have long been unclear; however, recent discoveries have shed some light on this distinction. This review will summarize the current theories put forth to explain how the mucosal immune system maintains tolerance towards commensals while retaining the ability to mount inflammatory responses against pathogens.