By following their noses at night, these great-uncles of modern mammals paved the way for their descendants to evolve larger brains.
Early mammals first grew bigger brains to accommodate a fine-tuned sense of smell, according to a new study.
Their fur covering was another key part of brain evolution, providing sensory input from all over their bodies.
These two characteristics probably allowed ancient mammals to become nocturnal.
While dinosaurs ruled the daytime, the earliest mammals may have prowled the night, guided by an elaborate sense of smell that set them apart from their ancestors and paved the path to modern mammals.
By recreating the size and shape of the brains of two ancient great-uncles of modern mammals, researchers, whose work is published today in the journal Science, find that the initial boost in mammalian brain size came largely from a bigger olfactory system.
"One of the things about mammals that's unique is that they have huge brains. There are no other animals that have brains that are so big," said Timothy Rowe of the University of Texas at Austin, who led the new study. "That's been one of the questions: Why, how and when did the brain get so large and what's driving that?"
His team found the answer by scanning tiny fossilized skulls of two 190-million-year-old mammal relatives found in China using high-resolution CT scanners similar to those used in medicine, but with more intense X-rays.
The CT scans can detect the precise contours of the skull's insides without destroying them, allowing researchers to make virtual molds of the minuscule brains.
When the researchers looked at the more ancestral species, Morganocudon, they found its brain was 50 percent larger relative to its size when compared with its reptilian ancestor. Most of the size increase, they found, came from a much bigger olfactory system.
The other fossil skull, from Hadrocodium, a paperclip-sized animal that is more closely related to modern mammals, showed a further 50 percent increase in relative brain size, again with much attributable to a more refined sense of smell.
"Mammals have the best olfactory systems of anything alive today," Rowe said.
"It's been a real trick to build an olfactory sensory system," he added. "The same is true for human machinery. We have great cameras, great microphones but an artificial nose has been a real trick to turn. Mammals were unique in being able to turn this trick and being able to navigate through an olfactory landscape instead of a visual or auditory one."
While some have created casts of later, true mammals, "nobody had ever been able to study the brain morphology of the early ancestors of mammals," François Therrien, curator of Curator of Dinosaur Palaeoecology at the Royal Tyrrell Museum of Palaeontology in Drumheller, Alberta, told Discovery News.
"It came as a surprise that the size increase would involve in great part one region of the brain: the olfactory bulbs," he added.
Rowe speculates that the enhanced sense of smell allowed them to fill a nocturnal niche. Similar scans of dinosaur brains suggest nearly all dinosaurs were diurnal, or active during the day, he said.
"Mammals, by mastering the art of olfaction, were able to come out at night while the dinosaurs were sleeping and avoid competing with them, and not get eaten by them," he added.
The researchers' findings point to another contributor to mammalian brain evolution: the sensory input from body hair. Evidence suggests that both animals were furry.
"Our hair started out as part of our peripheral nervous system," Rowe explained. The sensation of something crawling on our skin comes from the shafts of the hairs on our bodies, which send a signal to an exclusive part of the mammalian brain, the neocortex.
The neocortex maintains a map of our body's surface that tells us instantly when and where something is creeping around on our bodies. This part of the brain also integrates these sensory abilities to create an unmatched motor response, Rowe said.
"You're not going to find a turtle or a lizard that can play a piano," he added. "Think of all the things that your cat can do that a turtle will not do -- all of those tools. That's all processed in the neocortex."
While many modern mammals -- opossums, raccoons, shrews, aardvarks and lemurs to name just a few -- continue to rely on the keen noses developed in these ancestral brains and continue to live the nightlife, "others have traded up," Rowe said, like humans.
"We have a remarkable set of eyes. We have color vision. We're diurnal," he said "Those things have come at the expense of our olfactory system."
Mammals all have at least 1,000 olfactory genes, he added, which came in a third wave of olfactory evolution in the mammalian brain. "You and I have all 1,000 of those, but only about 300 turn on in us," Rowe explained.
"We've moved on into a visual world and an auditory world and been spectacularly successful," he said.