This is the conclusion of a two-part article. Part 1 was published October 3.
The evidence bearing on the question of the Neanderthal/modern human relationship falls into several categories, each giving only a partial and biased view of reality. Two of these categories are biological--the fossil record and, more recently, DNA analysis. A third major source of data is archaeology--the artifacts and other material traces left by the two populations. In this review I will focus on the biological data, since that is the origin of the recent announcements.
As indicated, each source of data has its own limitations and biases. It is a truism in paleontology that "fossils do not reproduce." One key observation to understanding evolutionary relationships among living creatures is genetic isolation--can members of two populations successfully mate with each other and produce fertile offspring? The answer is a fairly straightforward matter with respect to currently living animals. In the example given previously, horses and donkeys can mate and produce mules, but the mules cannot reproduce. There can be no self-perpetuating population of mules. Over time, horses and donkeys will tend to become increasingly different since there can be no natural gene flow between them. On the other hand, domestic dogs, coyotes, and wolves appear to be able to interbreed and create ongoing, hybrid populations. If true, despite notable differences in physical appearance and behavior, these canids all are part of a single, but markedly diverse species.
Unfortunately for science, there are no living Neanderthals, at least none that can be identified as such. Neanderthals were first identified by distinctive physical characteristics in fossil bones that differ from those seen among modern humans, either living humans or fossils dating back several tens of thousands of years. In order to make judgments regarding whether fossil specimens belong to the same or different species, paleontologists make careful observations on the available fossils and then compare the degree and consistency of differences in both the physical forms and the implied behaviors to estimate whether the differences are great enough to indicate genetic separation. These interpretations are made by analogy with the known degrees of differences between currently living species. While fairly effective in dealing with markedly different organisms, the distinctions become more difficult when closely related species are under study.
Neanderthal bones are, in general, more massive than modern human bones. Neanderthals had pronounced ridges over their eye sockets (brow ridges) as well as other protrusions of the skull not generally seen in modern humans. They were more muscular than modern humans. They had flatter, broader noses than modern humans. They had receding chins. Their brains were somewhat larger than those of modern humans. (Note: this does not necessarily mean they were smarter; brain size and intelligence are not directly correlated.) Do any or all of these differences indicate underlying genetic differences so great that Neanderthals and modern humans could not produce reproductively viable offspring?
To add more uncertainty, there is a range of variation in fossil Neanderthals. The above listed characteristics of Neanderthals are most strongly displayed in specimens from Europe, so-called Classic Neanderthals. These characteristics were less pronounced in the Neanderthals of the Middle East, known as Progressive Neanderthals. Does this mean that Progressive Neanderthals were less different from modern humans than were the ‘Classics'? If so, could there have been gene flow between modern humans and Classic Neanderthals through populations of Progressive Neanderthals?
The more massive build of Neanderthals, and other features such as flatter noses, especially of the Classic form, has been interpreted as an adaptation to the harsh glacial climates of Ice Age Europe. The more limited Neanderthal technology, compared to that of modern humans, combined with the Neanderthal's stockier build are thought by some to indicate a very physical method of hunting, involving running down and spearing or even grappling prey, which consisted of large mammals. Research by Donald Grayson of the University of Washington and Francoise Delpech of the University of Bordeaux (Knight, September 2003, NewScientist.com) suggests that Neanderthals and modern humans took the same range of animal prey when they were both present in Europe. However, much more research would be needed to demonstrate that their hunting methods were equally productive.
It may be, however, that the ability of Neanderthals to cope with cold conditions has been overestimated. Research comparing climatic and environmental changes during the last and apparently most severe cold period of the Pleistocene (there were multiple fluctuations between cold and relatively milder climates) indicates that both the Neanderthals and the earliest population of modern humans in Europe, known archaeologically as the Aurignacians, were pushed southward by the advancing ice and colder conditions that preceded it (van Andel, January 2002, Quaternary Research and Olszewski 2006 PaleoAnthropology). Under the extreme stress of this climatic change, large prey species retreated from northern Europe. Since the economies and technologies of both the Aurignacians and the Neanderthals were focused on large mammal hunting, this change in the available food supply is likely to have driven both groups to the limit of survival. However, according to this interpretation, the modern humans' ability to adapt culturally was sufficiently more flexible than that of Neanderthals so that the former survived while the latter did not.
The adaptation to changing climatic conditions by modern humans is apparently reflected archaeologically in the shift from the Aurignacian to the Gravettian cultural pattern. Gravettian technology was characterized by a more varied tool kit, including such innovations as throwing spears and nets, designed to exploit a wider range of food sources, as well as sewn clothing and woven textiles. The available archaeological evidence indicates that Neanderthals had only very limited, if any, success in adopting new technology from modern humans.
Taken together, both the physical and cultural differences (e.g., in artistic capabilities) would seem to indicate substantial divergence between Neanderthals and modern humans, presumably the result of different evolutionary trajectories driven by natural selection. However, there are notable physical differences between populations of living humans who have adapted to differing climates. For example, the need to control heat gain and loss has apparently led to the development of short and stocky physiques among native inhabitants of the Arctic as compared to taller, thinner peoples in hot and dry portions of Africa, reflecting the differing thermodynamic balances achieved by varying surface to volume ratios. Nevertheless, all living humans are members of the same species and can successfully mate and produce reproductively viable offspring. The relatively less massive build of modern humans as compared to Neanderthals may simply indicate that the former first appeared in Africa and, therefore, were physically adapted to warmer climates, without that difference necessarily indicating a separation at the species level. Indeed, there is some research which suggests that nutritional and other developmental stresses, as shown in the growth of teeth, were no greater for Neanderthals and those experienced by modern Inuit (Eskimos) (Guatelli-Steinberg, August 2003, Journal of Human Evolution).
Another study of growth patterns in Neanderthal teeth concludes that they matured at a faster rate than modern humans, reaching full adulthood by 15 years of age. If Neanderthals had physically stressful lives, with higher mortality rates and a shorter life span than those of modern humans, earlier maturation would have provided an advantage due to an earlier achievement of sexual maturity and quicker reproduction (Ramirez Rozzi, April 2004, Nature). Differences in maturation rates between Neanderthals and modern humans could have resulted in developmental incompatibilities for hybrid offspring, limiting or even preventing reproductive success between the two groups.
In sum, then, recent studies of fossil evidence have yielded much new information about Neanderthals. However, this information appears to yield contradictory interpretations that have not, as of yet, been brought within a single interpretive framework. It may be that studies of fossil morphology alone are not sufficient to reach a definitive conclusion regarding whether these closely related groups belonged to the same or separate species.
Over the past decade or so, a new source of data has been developed that can be brought to bear on this problem in general and the question of the Neanderthal/modern human relationship in particular. The analysis of DNA to compare the genetic makeup of living organisms and to judge degrees of similarity in order to develop models of evolutionary relationships has been practiced for several decades. However, more recently techniques for the extraction of DNA from fossils have been developed. Since DNA can crystallize, it may, under favorable circumstances, survive in fossils. If sufficient DNA can be recovered, the genetic makeup of long-dead organisms can be mapped (though we are nowhere near able to clone extinct animals, as in Jurassic Park, at least not yet). Consequently, it is at least potentially feasible to make genetic comparisons between extinct populations in the same way as has been done with living populations.
In order to judge the significance of recently announced results in the genetic analysis of Neanderthals using fossil DNA, it is necessary to consider some of the limitations of this source of data. Firstly, although DNA carries an organism's genetic code, scientific research is still very far from being able to decipher much of the code and understand the function of the great majority of the genetic instructions it contains. Therefore, comparisons can for the most part only be made between sequences of code, not what that code represents in a living organism. So, as with fossil bone, fossil DNA, at least currently, provides information regarding relative degrees of similarity and difference. It is not possible to determine with certainty whether the organisms represented by different DNA samples could mate and create reproductively viable offspring.
There are also practical problems with the study of fossil DNA, especially that of humans, that may call into question the reliability of particular results. Prominent among these is the issue of contamination (Caramelli et al, 16 July 2008, PLoS ONE). Since humans are responsible for excavating and processing the fossil bone specimens which are the source of fossil DNA, and humans are constantly shedding materials from their bodies that contain their DNA (e.g., skin, hair, saliva), unless extremely strict procedures are maintained throughout the process of recovery and analysis of the samples to be studied it is quite possible that contemporary human DNA may be introduced. If that happens, since it is difficult to confidently distinguish between ancient and modern DNA once mixed, the results of any analysis are rendered suspect.
Despite these limitations, the study of fossil DNA does provide a distinct source of information from that of the study of fossil bone morphology. As is the case with many forms of research, the more separate lines of evidence can be brought to bear on a particular problem, the more reliable the ultimate interpretation is likely to be.
One final problem with the study of both fossil bone morphology and fossil DNA must be mentioned, the issues of sample size and representativeness. When studying biological evolution that involves timescales of tens or hundreds of thousands or even millions of years, the numbers of generations of organisms that lived and died is quite large. Furthermore, at any given time the numbers of living members of a given species would have been in the tens of thousands or more. Any single individual represents only a small part of the range of genetic and morphological variation that existed within a species at that time. Therefore, in order to achieve a reasonable degree of confidence regarding the range of variation in any particular attribute (e.g., shape of the skull, thickness of the bones) that existed at a given time, let alone over thousands of years, it would be necessary to obtain a large and statistically random sample from the population.
Unfortunately, in comparison to the numbers of individuals that are likely to have existed during the time span of a particular species, the numbers of fossil specimens of Neanderthals and of early modern humans that currently are available is only a tiny fraction. Furthermore, the recovered specimens represent a haphazard rather than a statistically controlled sample. Therefore, it is difficult to judge with confidence whether the available sample of fossils is representative of the past populations that we wish to study. The bottom line is that when more specimens are discovered and analyzed current interpretations may have to be revised as a more complete picture emerges.
With the foregoing caveats in mind, some examples of the recent research results concerning the degree of difference between Neanderthals and modern humans using fossil and/or DNA data can be reviewed.
A leading proponent of the single-species hypothesis, Erik Trinkaus of Washington University in St. Louis, Missouri, has, over the years, described a number of fossil specimens which he claims demonstrate Neanderthal/modern human interbreeding.
One example is that of the fossil skeleton of a boy, found in Portugal, dating to 24,500 years ago (Duarte et al. 1999, Proceedings of the National Academy of Sciences). The specimen bears a number of ‘Neanderthal' characteristics thousands of years after the apparent disappearance of Neanderthals, along with other, distinctly ‘modern' attributes. The supposed ‘Neanderthal' characteristics are in the post-cranial skeleton--stocky body and short legs, as compared to a modern-looking skull. The boy was buried with strings of marine shells and painted with red ocher. These burial practices are distinctive of modern humans. If Trinkaus's interpretation is correct, the boy was a member of a hybrid population, descended from a mixed Neanderthal/modern human group. Critics have argued that the boy's supposedly Neanderthal characteristics could have existed within the range of variation of modern humans at that time and do not necessarily indicate any interbreeding (Tattersall and Schwartz 1999, Proceedings of the National Academy of Sciences).
More recently, Trinkaus and colleagues analyzed early modern human bones dating to 30,000 years ago from a cave in Romania (August 2007, Current Anthropology). The Romanian fossils betray "considerable" interbreeding, according to these researchers, exhibiting a mixture of modern human and Neanderthal characteristics in the skull. Neanderthal-like characteristics include an occipital bun (a swelling at the back of the skull), a muscle attachment at the back of the skull, and certain muscle attachments at the back of the jaw. These characteristics are rarely or never seen in modern humans.
In another line of investigation (Martinez et al., 22 June 2004, Proceedings of the National Academy of Sciences), Spanish researchers using portions of skulls from five individuals attributed to Homo heidelbergensis, thought to be the common ancestor of modern humans and Neanderthals, created a CT image of the skull. The reconstruction shows anatomical characteristics indicating hearing in approximately the same range as modern humans, but clearly different from chimpanzees. The interpretation is that members of the species Homo heidelbergensis appear to have been adapted to hear the range of sound typical of modern human speech. Presumably, therefore, their descendants, including Neanderthals, would have had that capacity as well.
Separate research at the Max Planck Institute for Evolutionary Anthropology has found that Neanderthals had the same version of a gene, designated FOXP2, that occurs in modern humans and is associated with language and speech.
Robert McCarthy, an anthropologist at Florida Atlantic University in Boca Raton, reconstructed Neanderthal vocal tracts and concluded that their vocalizations would have been somewhat different from those of modern humans, but suitable for verbal communication nevertheless (Callaway, 15 April 2008, NewScientist.com).
While the foregoing evidence indicates similarities in auditory and vocal capabilities between Neanderthals and modern humans, if these attributes already existed in the common ancestor of the two lineages, Homo heidelbergensis, then they do not shed light on whether a species-level separation existed between the two.
Genetic studies, along with some morphological studies such as that by Weaver, Roseman, and Stringer described previously, tend to support the view of a distinct separation between modern humans and Neanderthals. Among the most comprehensive investigations, two teams, one at the Lawrence Berkeley National Laboratory and the other at the Max Planck Institute for Evolutionary Anthropology in Germany, using different techniques, separately sequenced large chunks of DNA extracted from the femur of a 38,000-year-old Neanderthal specimen found in a cave in Croatia. This study involves nuclear DNA and is distinct from the mtDNA study at the Max Planck Institute described previously. The results confirm a close relationship between Neanderthals and modern humans, with a genetic overlap of more than 99.5 percent. Despite this degree of similarity, the researchers found no evidence to support the idea of modern human/Neanderthal interbreeding, though it cannot be entirely ruled out using this data. The genetic overlap is interpreted as due to common ancestry, as in the example of horses and donkeys cited previously, rather than to continued or renewed genetic flow.
In evaluating the results of genetic studies, it is important to understand that the differences between species cannot be judged purely by a numerical comparison of the similarity between DNA sequences. Put in simple terms, there are ‘architecture genes' and there are ‘control genes.' The control genes affect how the architecture genes are expressed. A small change in a control gene can have a substantial effect on the expression of one or more architecture genes. This could result in a significantly different organism despite the fact that the change in the genetic sequence was quite small. Therefore, modern humans could differ from Neanderthals in only a relative few, but very critical control genes.
Other research, this studying the mitochondrial DNA of modern humans from fossils dating to roughly the time of the overlap with Neanderthals in Europe, finds that similar mtDNA is still present in Europe today. However, this mtDNA has substantial differences with that of Neanderthals (Caramelli et al., 16 July 2008, PLoS ONE). This again indicates a definitive genetic separation between Neanderthals and modern humans.
Finally, recent revisions in the calibration of radiocarbon dates have led to a shortening of the apparent duration of overlap between Neanderthals and modern humans in Europe to as little as a thousand years (Mellars, 23 February 2006, Nature). This would indicate that instead of a relatively long period of coexistence and, perhaps, interaction, both culturally and genetically, the encounter swiftly led to the extinction of Neanderthals due to their inability to compete for resources with modern humans and/or direct extermination of the former by the latter.
In sum, genetic studies tend to support the interpretation that Neanderthal and modern humans were separated at the species level. This stands in contrast to the seemingly more contradictory results coming from studies of fossil bone. Nevertheless, overall, the trend in research seems to be toward the species-separation side of the argument.
It is important to consider that genetic research has so far dealt only with segments of Neanderthal DNA. A much more comprehensive view may come when the full DNA sequence of Neanderthals has been worked out, which should happen by the end of this year. This can then be compared with the modern human DNA sequence that was determined by the Human Genome Project.
As indicated previously, research into the nature of the relationship between Neanderthals and modern humans has implications for understanding the course of human evolution more generally. Of special interest, perhaps, is the light this research will shed on the origin and development of the human capacity for abstract thought, the single most definitive human characteristic. Did this ability appear all at once, fully formed? Or, was it gradually constructed over millions of years, species by species, with many different, perhaps more limited versions of this faculty having existed in the past? If so, our current mental capabilities are only the latest in an array of evolutionary experiments in this mode of adaptation. Seemingly, our version is the most sophisticated yet achieved, but it may not necessarily be the final version.