The research suggests a pivotal role for estrogen - as well as the enzyme aromatase that is responsible for estrogen synthesis - in male territorial behavior, according to findings published in the October 2, 2009 issue of the journal Cell.

Estrogen's role in the mating behaviors of these mice, however, was less clear, which indicates that territorial and sexual behaviors are likely influenced by distinct and separate connections in the brain, according to Nirao Shah, MD, PhD, an assistant professor in the UCSF Department of Anatomy and senior author of the paper.

"This really changes the way we view male and female behaviors," said Shah, who also is affiliated with the UCSF programs in neuroscience and genetics and who last week received the 2009 Pioneer Award from the National Institutes of Health for his research. "What we previously looked upon as a single unit of gender-related behavior, we now see as a collection of separate behaviors controlled at least in part by distinct neural pathways."

Males and females across all sexually reproducing species display gender-specific behavior in many areas, including mating, territorial marking, aggression and parental care, Shah explained. Collections of cells form circuits in the brain, referred to as neural pathways, that control these and other behaviors. Shah said that both estrogen and the male hormone testosterone are known to be essential in developing these circuits and in sex-specific behavior. But the precise role of these hormones and how they may interact genetically to control these behaviors has been unclear.

The current study fills in at least one piece of the puzzle, he said. The study suggests that the conversion of testosterone in the brain to estrogen by the enzyme aromatase is critical to developing and activating brain circuits that control male territorial behavior.

The researchers first used a gene-targeting strategy in which they attached highly sensitive genetic "reporters" to cells to examine the circuitry of the mouse brain at the cellular level. These "reporter" genes allowed researchers to visualize or track where testosterone was being converted to estrogen in the brain.

Melody Wu, a graduate student in the UCSF Neuroscience Program who led the five-year research effort in Shah's lab, said the team discovered extensive differences between adult male and female mice in the number and connections of nerve cells expressing aromatase. For example, they found more aromatase-positive cells in the males in two regions of the brain known to regulate sexual and aggressive behaviors.

Wu said the team then honed in on testosterone. Testosterone activates the androgen receptor, and male mice lacking the receptor do not display sexual behavior or aggression. The study found that adult males that were mutant for the androgen receptor still had patterns of aromatase-positive cells that were typical of the normal male brain. Somehow, she said, testosterone was creating these male patterns without activating the androgen receptor.

Next, the team examined specifically whether estrogen was causing these differences in male and female brains. When female mice were given estrogen supplements as newborns, they developed brain patterns of aromatase that were indistinguishable from males. These females now exhibited male territorial behavior and showed aggression toward male intruders; by comparison, untreated female mice rarely, if ever, attack males.

"Clearly, estrogen was causing this male-pattern increase of aromatase-expressing cells," Shah said. "This suggests that aromatase, which converts testosterone to estrogen in the brain, plays a critical role in the neural pathways responsible for these gender differences."

The potential conclusions of this research are intriguing, he said. "We show that exposure to estrogen neonatally can alter adult sex-specific behaviors in mice."

Those findings did not appear to apply to sexual behavior, however. While male mice reliably mate frequently with females, the estrogen-treated females showed no difference from untreated females when exposed to normal females that were sexually receptive, or in heat. However, the estrogen-treated females did not display typical female sexual behavior; they mated much less frequently with males and even attacked and chased them.

The researchers proposed that these aromatase-expressing regions of the brain could form an interconnected network that regulates sex-specific behaviors, but Shah cautioned that much more research needs to be done on the role of the development of sex-specific neural pathways, and that many additional factors, including genetics and socialization, could contribute to sexual differentiation.

More information: The paper, based on research at UCSF and Fujita Health University, appears online at www.cell.com.

Source: University of California - San Francisco