Maize, commonly known as corn, was first domesticated from a grass species called teosinte, almost 9,000 years ago. Humans were able to tame corn to grow large hearty stocks, but teosinte remained a small grass because it developed a defense to breading with corn.
Sexually breeding plants, like corn and grass, reproduce when the male part of the plant releases pollen that gets caught in the female part of the plant, inside of the flower center. When the flower senses that pollen has fallen into the flower, a cascade of hormonal changes allow the pollen to form a tube to the ovary to finally fertilize the flower. It is common that closely related species of plants may pollinate one another, in a phenomenon called cross-breeding.
Source: Carnegie Science Plant Biology
A collection of genes recently characterized in teosinte suggest that when maize pollen falls into the flower of teosinte, the teosinte produces proteins which weakens the tube that the pollen forms on its way to the ovary. The tube breaks down and effectively cross-breeding between maize, and its ancestor cannot occur.
The cross-breeding ability between maize and teosinte was not always like this; thousands of years ago, it is very likely that the two were able to cross-breed like other grass species. However, after maize was domesticated and isolated, each species continued to evolve. Genetic evolution has a ‘use it or lose it’ method, so when two species are isolated, the ability to interbreed is lost.
Sources: Science Daily, Carnegie Science Plant Biology