Before Shawn Carlson ran his experiment, the astrologers who helped him design it wrote down a number. They said they would pick the correct personality profile at least half the time. Chance alone predicts one time in three. When the results appeared in Nature on 5 December 1985, the astrologers had chosen correctly in 40 of 116 trials, a hit rate of 34 percent.
That gap between a stated prediction and a measured outcome is why the Carlson study still circulates forty years later. Skeptics cite it as the cleanest refutation of natal astrology. Astrologers cite its critics as proof that the refutation was premature. Both camps usually quote it from memory. This article reads it the slow way: the design, the arithmetic, the objections, and the limits of what any single experiment can establish. It is the first in a weekly series of research articles on this site, and it sets the evidential baseline for everything else we publish.
A note on method before we begin. We read Carlson’s own account, the 2009 critique by Suitbert Ertel, and the summaries that astrologers and skeptics have written about both. Where a figure comes from the paper, we say so. Where critics and defenders disagree, we report the disagreement and do not resolve it by fiat.
Abstract: The Short Version
- The question: can an astrologer, working only from a person’s birth chart, describe that person’s personality well enough to beat chance?
- The design: 28 astrologers, nominated by the astrologers’ own research council (NCGR), matched natal charts to personality profiles from the California Psychological Inventory (CPI) in a double-blind procedure they had approved in advance as a fair test.
- The result: the astrologers chose the correct profile first in 40 of 116 trials (34 percent). Chance predicts 33 percent. They had predicted at least 50 percent.
- The objection: critics, most fully Ertel in 2009, argue that the design lacked statistical power and that a combined first-and-second-choice analysis reaches marginal significance (p = .054). That reanalysis finds a small effect at best, nowhere near the effect the astrologers predicted.
- The limit: the study tests one claim, that a natal chart identifies a personality profile. It does not test every practice that goes by the name of astrology, and one study cannot settle a question this old.
The Question the Experiment Asked
Astrology is a family of practices. Horary astrology answers a question from the chart of the moment someone asks it. Electional astrology picks a favourable date. Mundane astrology reads the fortunes of nations. The practice most readers meet, and the one the tradition treats as foundational, is natal astrology: the claim that the positions of the Sun, Moon and planets at a person’s birth, as seen from the birthplace, describe that person’s character and circumstances.
Carlson tested only the first half of that claim, the part about character. He stated the thesis he meant to test in terms the participating astrologers accepted: that a natal chart can describe a person’s personality traits accurately. He chose the narrowest version of the claim because a narrow claim can fail cleanly. If astrologers could not manage this task, they would be unlikely to manage a broader one. If they could, the experiment would have found something worth a great deal of further work.
The experimenter was not a veteran debunker. Carlson was a Berkeley undergraduate in physics at the time, and he recruited the astrologers through the NCGR, which nominated participants and served as the astrological advisory panel. The advisers were Tony Joseph, Michael Caveney and Chris Nelson. Their role was to make the test one that astrologers would accept before the data came in.
Why a Fair Test Is Hard to Build
The philosopher Karl Popper gave the standard test of a scientific claim: it must forbid something. A claim that survives every possible observation tells us nothing about the world. Astrological readings often fail this test in practice, because astrologers write them in language flexible enough to fit almost anyone. Psychologists call the effect that follows the Barnum effect, after Bertram Forer’s 1949 classroom demonstration in which students rated an identical, vague personality sketch as highly accurate for themselves. We cover it in detail in our article on why readings feel accurate.
The Barnum effect explains why the obvious experiment fails. If you give a person a reading and ask whether it fits, nearly everyone says yes. The test has no way to lose. Carlson therefore used a matching design, which forces the reading to compete against alternatives. Take a person’s real profile, add two profiles that belong to strangers, and ask an astrologer to pick the real one from the chart. A guessing astrologer picks correctly one time in three. An astrologer who reads charts well picks correctly more often.
Matching designs have a longer history than Carlson’s. He modelled his procedure on an earlier double-blind matching test by Vernon Clark in 1961. What distinguished the 1985 version was the degree of cooperation between experimenter and practitioners. The astrologers chose the personality instrument. They reviewed the protocol. They stated in advance what result they expected. After that, nobody could honestly blame a failure on a design imposed from outside.
The Design in Detail
| Element | What Carlson reports |
|---|---|
| Astrologers | The NCGR advisers listed about 90 astrologers as qualified, experienced and familiar with the CPI. Carlson invited them all, and 28 accepted. The advisers added two participants after they reviewed their qualifications. |
| Subjects | 256 volunteers at the start, split into a test group of 128 and a control group of 128. Attrition left 177 for Part One (83 test, 94 control) and 116 usable subjects for Part Two. |
| Recruitment | Bay Area newspaper advertisements, classroom announcements and postings. About 70 percent were college students. Subjects had to be at least 17 and had to document a birth time accurate to within 15 minutes. Carlson excluded people who strongly disbelieved in astrology, and anyone who had previously had a natal chart drawn. |
| Instrument | The California Psychological Inventory, a standard personality questionnaire. The astrologers themselves identified it as the instrument best matched to what they believed charts could show. |
| Blinding | Neither the astrologers nor the experimenter knew which code number belonged to which subject while the astrologers matched. |
| Prediction | The astrologers predicted a correct choice at least half the time. They and the experimenter agreed that the protocol was a fair test. |
| Chance level | One in three for a first choice, because each trial offered three profiles. |
The attrition deserves a sentence of its own. Carlson began with 256 subjects and finished Part Two with 116. He called the loss unanticipated and said he did not believe it biased the results, but it matters for a reason we return to under statistical power.
Part Two: The Astrologers’ Matching Test
In Part Two, each astrologer received a natal chart for one subject and three CPI profiles: that subject’s own and two drawn at random from the other subjects. The astrologer ranked the profiles from the one that best fitted the chart to the one that fitted worst. Across 116 trials the results were as follows.
| Choice | Correct | Rate | Chance rate | Astrologers predicted |
|---|---|---|---|---|
| First choice | 40 of 116 | 34 percent | 33 percent | 50 percent or more |
| Second choice | 46 of 114 | 40 percent | 33 percent | No specific figure |
| Third choice | 28 of 114 | 25 percent | 33 percent | No specific figure |
The first-choice figure is the headline. At a chance rate of one in three, 116 trials would produce about 38.7 correct choices on average. The astrologers produced 40, which sits about a quarter of a standard deviation above the average and well within the ordinary scatter of guessing. Carlson reported that the result fell 3.3 standard deviations below the 50 percent the astrologers had predicted. In plain language, the prediction was not close.
The second and third choices behave as the first does. A person who guesses will land in each rank about equally often, and the figures here wander around those levels without a clear pattern. Carlson used a threshold of 2.5 standard deviations to decide whether any departure from chance deserved attention. None of the three ranks crossed it in favour of astrology.
Part One: Why Carlson Drew No Conclusion
Part One reversed the roles. Instead of the astrologers matching charts to profiles, the subjects tried to pick their own chart interpretation from a set of three. In the test group, 28 of 83 chose correctly on their first pick, a rate of 34 percent. A matched control group of 94 people made the same kind of choice and picked at 45 percent, a fluctuation Carlson attributed to chance.
Carlson nonetheless drew no conclusion from this result, and his reason is instructive. The same subjects also tried to pick their own CPI profile, the standard psychological measure, from a set of three. They succeeded at about the same modest rate, 25 of 56 in the test group (45 percent). If people cannot reliably recognise a questionnaire profile built from their own answers, their failure to recognise a chart interpretation proves little about the chart. He called Part One a poor test of astrology for exactly this reason, and he set the rating data aside because subjects had not rated the sections independently.
Readers sometimes take this as evidence of bias. It is closer to the opposite. Carlson reported a result that looked bad for astrology, then declined to lean on it because the method was weak. That is the behaviour of an experimenter who is trying not to overreach.
How to Read the Statistics
Three ideas carry the whole argument, and none needs mathematics beyond arithmetic.
Chance. If an astrologer cannot read charts, the choice among three profiles is a guess, and a guess succeeds one time in three. Any claim of skill has to beat that line by more than luck can explain.
Standard deviation. Even pure guessing does not land exactly on one third. Across 116 trials the number of correct choices scatters around 38.7 with a standard deviation of about 5.1. A result within a deviation or two of the average counts as ordinary. The further out it lies, the harder luck becomes to defend. Carlson used 2.5 standard deviations as his line.
Power. An experiment can fail to find a real effect if it is too small to see one. Statisticians call the ability to detect an effect of a given size the power of the test. Carlson’s design aimed at an effect large enough to show up at four standard deviations with 128 subjects per group. Attrition reduced the discrimination to 3.2 standard deviations for Part One and 3.9 for Part Two. The astrologers had predicted a large effect, 50 percent against 33, and a study this size could detect an effect of that magnitude. It could not reliably detect a small one.
Watch for: the difference between “no evidence of a large effect” and “evidence of no effect”. The Carlson study supports the first statement strongly. It supports the second only weakly, as Carlson’s critics point out.
The Reply From the Other Side
The fullest published critique comes from Suitbert Ertel, a psychologist at the University of Göttingen, in the Journal of Scientific Exploration in 2009. His paper argues that the design violated the demands of fairness and that the analysis ignored common statistical norms. His objections cluster in four groups.
- Piecemeal analysis. Ertel notes that the stated success criterion combined the first and second choices, yet Carlson analysed first, second and third choices separately. Kenneth McRitchie, writing from the astrological side, makes the same point.
- Power and effect size. Ertel argues that Carlson’s samples were too small to detect small effects, and he compares them with the Gauquelin studies of planetary positions and professions, which needed samples of 500 to 1,000 or more.
- Three-choice format. He holds that choosing among three profiles discriminates less well than comparing pairs, and that the profiles offered were too similar to one another.
- Control comparison. For Part One, he argues that comparing each group against chance, instead of comparing the test group directly with the control, ignores the logic of a controlled design.
Ertel then reanalysed the data. Adding first and second choices gives 86 correct against about 77.3 expected by chance, a binomial Z of 1.61 and p = .054. By the usual convention of p below .05, that falls just short. He also reports a small positive correlation between the astrologers’ confidence ratings and whether they were right (tau = .088, p = .037, one-tailed, effect size .10) and treats it as a replication of the matching result by a different method. His conclusion is that the evidence is too weak to call astrology verified, and that Carlson’s “devastating verdict” is untenable.
Our reading of the dispute
Ertel’s technical points have merit. A three-choice design is a blunt instrument, and a combined first-and-second analysis is a reasonable reading of the original success criterion. His reanalysis is also honest about its own result, which is marginal.
Three observations temper it. First, the figure is small. Eighty-six of 116 is 74 percent, against 67 percent expected by chance. That is the largest effect one can wring from the data, and it is a long way from the 50 percent first-choice rate the astrologers predicted, which would have put the first-choice count near 58. Second, a marginal result that appears only after choosing among several defensible analyses is the situation statisticians call the garden of forking paths: with enough reasonable ways to cut the data, one of them will look promising by luck. Third, the astrologers who designed the test did not predict “slightly better than chance”. They predicted a large, obvious effect, and the data show no sign of it.
Where we land: the Carlson study refutes the specific, strong claim that the astrologers made about their own skill. It does not prove that no weak relationship of any kind could exist, and a fair reader will say so.
What the Experiment Cannot Show
- It tests one task. Matching charts to CPI profiles is a narrow test. Practising astrologers say that a real consultation draws on conversation with the client, and the study did not allow for that.
- Its sample is local. The subjects came from the San Francisco Bay Area and were mostly college students. Results could differ elsewhere, though nothing in the design suggests that they would.
- The astrologers were self-selected. 28 of about 90 invitees took part. We cannot know whether the other 62 would have done better.
- One instrument carried the test. The astrologers chose the CPI, which makes the choice defensible, but every measurement tool has limits.
- One study is one study. A single experiment, even a careful one, does not close a question. Replication does, and we discuss the replications we know of next.
Where the Evidence Stands After Carlson
Several later studies tested similar claims. McGrew and McFall published a matching study in 1990 in the Journal of Scientific Exploration. According to the summaries we have read, they reported that astrologers’ success was unrelated to their stated confidence, and that astrologers failed to agree with one another’s matches. Nanninga (1996 to 1997) and Wyman and Vyse (2008) followed with further tests. We have not read those two papers closely enough to summarise them, and we list them under further reading for that reason. A reader who wants the astrological defence of Carlson in full can consult Currey’s and McRitchie’s published reviews, which argue that the data are more supportive than the original verdict suggests.
Meanwhile the psychology of why readings feel true has stood on firmer ground. Researchers have repeated Forer’s 1949 demonstration many times, and the pattern holds: people accept vague, flattering descriptions as personal. That is the strongest scientific reason to separate how a reading feels from whether it predicts.
A Test You Can Run at Home
The matching design is simple enough to copy, and doing it yourself teaches more than reading about it. This protocol uses a horoscope, a card reading or any reading that claims to describe a person.
- Write three short readings. One is the real reading for your friend. The other two belong to other people. Number them at random.
- Let your friend choose the one that fits best, without knowing which is real.
- Repeat with different people and different sets. A single trial means nothing.
- Count. At chance, one trial in three succeeds. Over 30 trials you expect about 10 correct, with a standard deviation near 2.6. Getting 15 or more right would happen by luck about 4 times in 100. Getting 17 or more would happen by luck about once in 140 tries.
- Record every trial, including the failures. A log you only update when it works proves nothing.
Most people who run this honestly find results close to one in three. A few find more. If you find more, run it again with a fresh set of readings. That is how the Carlson study earned the attention it has.
The Tradition and the Evidence: How This Site Uses Both
This site publishes astrology, numerology and tarot as traditions. When we write that the tradition says Mars governs courage or that a number carries a theme, we are reporting what practitioners have held, with dates and sources where we can find them. We are not reporting that the claim predicts anything. The Carlson study is one reason for that distinction. It is the best-known attempt to hold a predictive claim to account, and the claim did not survive in the form its defenders offered.
None of this drains the traditions of value. A symbolic language can structure reflection, supply a shared vocabulary, and connect a reader to centuries of human attention to the sky. Those functions do not require prediction. They do require honesty about the difference, and a reader who understands the evidence is better placed to use the tradition well than one who does not.
Watch for: any article, ours included, that moves from “the tradition says” to “this will happen” without a source. The jump is where the weight of evidence stops.
FAQ
Did the Carlson study prove astrology false?
It showed that the astrologers in it could not match natal charts to personality profiles at the level they themselves predicted. Proving a whole field false is a larger task than any one experiment can perform, and Carlson’s critics argue that his design could miss small effects.
Did the astrologers agree to the test beforehand?
Yes. The NCGR advisers reviewed the design and judged it a fair test, and they named the 50 percent success rate they expected. That pre-agreement is why the study is hard to dismiss on the grounds that Carlson imposed the format.
Why did Carlson use the California Psychological Inventory?
The astrologers identified it as the instrument best matched to the information they believed a chart could supply. It is a standard, well-studied personality questionnaire with scales for traits such as dominance and sociability.
Is Ertel’s reanalysis a replication?
No. It reanalyses Carlson’s own data. It finds a marginal result (p = .054) that does not reach the usual .05 threshold, and an independent replication would need new astrologers and new subjects.
Does this mean horoscopes are worthless?
The study tests whether a natal chart identifies a personality profile. It does not test whether a horoscope can be entertaining, reflective or culturally interesting, which are different questions.
Sources and Further Reading
- Carlson, S. (1985). A double-blind test of astrology. Nature, 318, 419 to 425. DOI: 10.1038/318419a0. Abstract also at NASA ADS.
- Carlson’s longer report with the full tables, held by the University of California: escholarship.org.
- Ertel, S. (2009). Appraisal of Shawn Carlson’s renowned astrology tests. Journal of Scientific Exploration, 23(2), 125 to 137. Read the paper.
- McGrew, J. H., and McFall, R. M. (1990). A scientific inquiry into the validity of astrology. Journal of Scientific Exploration.
- Forer, B. R. (1949). The fallacy of personal validation: a classroom demonstration of gullibility. Journal of Abnormal and Social Psychology, 44, 118 to 123.
- Further tests cited in the astrological literature, not summarised here: Nanninga (1996 to 1997); Wyman and Vyse (2008).
Plain note: this article describes research and tradition. It does not predict anything, and it does not tell you what to believe. It tells you what a carefully designed test found, and where reasonable people still disagree.
Last reviewed: 4 November 2026.
Discover more from Divinity Digits
Subscribe to get the latest posts sent to your email.
