Sensory organ, not brain, differentiates sexual behavior in some mammals

Rather, they say, the epicenter of sex-specific behavior in many species may be a small sensory organ found in the noses of all terrestrial vertebrates except higher primates. Their work, appearing this week in the journal Nature, indicates that defects in this organ, known as the vomeronasal organ, lead female mice to adopt male behaviors such as mounting and pelvic thrusting while abandoning female behaviors such as nesting and nursing.

"These results are flabbergasting," says Catherine Dulac, Higgins Professor of Molecular and Cellular Biology in Harvard's Faculty of Arts and Sciences and an investigator with the Howard Hughes Medical Institute. "Nobody had imagined that a simple mutation like this could induce females to behave so thoroughly like males."

The results do not apply directly to humans, which lack a vomeronasal organ, but may open new avenues of investigation for research into sex-specific human behavior.

Dulac and co-authors Tali Kimchi and Jennings Xu studied female mice mutant in TRPC2, an ion channel whose absence disables the vomeronasal organ, which works along with the nose to detect pheromones.

They found that these females, when placed in a cage with a sexually experienced male, would engage in typically male courtship activity: chasing their cagemates, lifting the males' hindquarters with their snouts, and emitting complex ultrasonic vocalizations that are part of the male mouse's mating ritual. Eventually, the female mutants would replicate male sexual behavior by mounting the hapless males and thrusting.

The male mice responded with increasing aggression toward the mutant females, eventually impregnating all of them. Once these females had given birth, Dulac and her colleagues observed a striking lack of maternal behavior. After giving birth, wild-type female mice spend about 80 percent of their time in their nest nursing their newborns, but the mutant females would readily wander away after about two days of motherhood, eventually abandoning the nest altogether. While lactating mice will ordinarily attack male intruders and reject their courtship behaviors, the mutant females were docile toward males and appeared highly receptive to their overtures.

"There are two possible interpretations," Dulac says. "Either the vomeronasal organ may be needed to grow a female-specific neural circuit during development, or the mature female mouse brain may require vomeronasal activity to repress male behavior."

To test these two alternatives, Dulac and her colleagues excised vomeronasal organs from the nasal septa of normal adult females. These mice began behaving like males, despite the fact that they -- like mutant females in the study -- showed testosterone levels, estrogen levels, and estrus cycles indistinguishable from those found in normal females.

"It had previously been thought that entirely different neural circuits, modulated by these hormones, controlled sex-specific behavior," Dulac says. "Remarkably, our work suggests that neuronal circuits underlying male-specific behaviors develop and persist in the female mouse brain, but are repressed by the normal activity of the vomeronasal organ."

"In fact, our research suggests a new model where exactly the same neural circuitry exists in males and females," Dulac says. "In this model, only the vomeronasal pathway itself -- which serves as a switch that represses male behavior while promoting female behavior -- is dimorphic. While male and female bodies are strikingly different physiologically, it appears the same cannot be said for the brain."

Dulac and colleagues are now studying the behavior of male mice mutant for TRPC2 to determine whether they display female-like traits.

Source: Harvard University