It’s no surprise, human nature being what it is, that our early detections of possible life on other worlds through ‘biosignatures’ are immediately controversial. We have to separate signs of biology from processes that may operate completely outside of our conception of life, abiotic ways to produce the same results. My suspicion is that this situation will persist for decades, claim vs. counter-claim, with heated conference sessions and warring papers. But as Alex Tolley explains in today’s essay, even a null result can be valuable. Alex takes us into the realm of Bayesian statistics, where prior beliefs are gradually adjusted as new data come in. We’re still dealing with probabilities, but in a fascinating way, uncertainties are gradually being decreased though never eliminated. We’re going to be hearing a lot more about these analytical tools as the hunt continues with next generation telescopes.

by Alex Tolley

Introduction

The venerable Drake equation’s early parameters are increasingly constrained as our exoplanet observations continue. We now have a good sample of thousands of exoplanets to estimate the fraction of planets in the habitable zone that could support life. This last firms up the term n_e, the mean number of planets that could support life per star with planets.

This is now a shift to focus on the fraction of habitable planets with life (f_l). The first to confirm a planet with life will likely make the history books.

However, as with the failure of SETI to receive a signal from extraterrestrial intelligence (ETI) since the 1960s, there will be disappointments in detecting extraterrestrial life. The early expectation of Martian vegetation proved incorrect, as did the controversial Martian microbes thought to have been detected by the Viking lander life detection experiments in 1976. More recently, the phosphine biosignature in the Venusian atmosphere has not been confirmed, and now the claimed dimethyl sulfide (DMS) biosignature on K2-18b is also questioned.

While we hope that an unambiguous biosignature is detected, are null results just disappointments that have no value in determining whether life is present in the cosmos, or do they add some value in determining a frequency of habitable planets with life?

Before diving into a recent paper that attempts to answer this question, I want to give a quick introduction to statistics. The most common type of statistics is Fisher statistics, where collected sample data is used to calculate the distribution parameters for the population from which the sample is taken. This approach is used when the sample size is greater than 1 or 2, and is most often deployed in calculating the accuracy of a mean value and 95% range of values as part of a test of significance. This approach works well when the sample contains sufficient examples to represent the population. For binary events, such as heads in a coin test, the Binomial distribution will provide the expected frequencies of unbiased and small biases in coin tosses.

However, a problem arises when the frequency of a binary event is extremely low, so that the sample of events detects no positive events, such as heads, at all. In the pharmaceutical industry, while efficacy of a new drug needs a large sample size for validity, the much larger phase 4 marketing period is used to monitor for rare side effects that are not discoverable in the clinical trials. There have been a number of well known drugs that were withdrawn from the market during this period, perhaps the most famous being thalidomide and its effects on fetal development. In such circumstances, Fisherian statistics are unhelpful in determining probabilities of rare events with sample sizes inadequate to catch these rare events. As we have seen with SETI, the lack of any detected signal provides no value for the probability that ETI exists, only that it is either rare, or that ETI is not signaling. All SETI scientists can do is keep searching with the hope that eventually a signal will be detected.

Bayesian statistics are a different approach that can help overcome the problem of determining the probability of rare events, one that has gained in popularity over the last few decades. It assumes a prior belief, perhaps no more than a guess, of the probability of an event, and then adjusts it with new observed data as they are acquired. For example, one assumes a coin toss is 50:50 heads or tails. If the succeeding tosses show only tails, then the coin toss is biased, and each new resulting tail decreases the probability of a head resulting on the next toss. For our astrobiological example, if life is very infrequent on habitable worlds, Bayesian statistics can be informative to estimate the probability of detection success.

In essence, the Bayesian method updates beliefs in the probability of events, given the new observations of the event. With a large enough number of observations, the true probability of an event value will emerge that will either converge or diverge from the initial expected probability.

I hope it is clear that this Bayesian approach is well-suited to the announcement of detecting a biosignature on a planet, where detections to date have either been absent or controversial. Each detection or lack of detection in a survey will update our expectations of the frequency of life. At this time, the probability of life on a potentially habitable planet ranges from 0 (life is unique to Earth) to 1.0 (some form of life appears wherever it is possible) Beliefs that the abiogenesis of life is extremely hard due to its complexity push the probability of life being detected as close to 0. Conversely, the increasing evidence that life emerges quickly on a new planet, such as within 100 million years on Earth [6], implies that the probability of a habitable planet having life is close to 1.0.

The Angerhausen et al paper I am looking at today (citation below) considers a number of probability distributions depending on beliefs about the probability of life, rather than a single value for each belief. These are shown in Figure 1 and explained in Box 2. I would in particular note the Kerman and Jeffreys distributions that are bimodal with the highest likelihoods for the distributions as the extremes, and reflect the “fine tuning” argument for life by Kipping et al [2] explained in the Centauri Dreams post [3] i.e., either life will be almost absent, or ubiquitous, and not some intermediate probability of appearing on a habitable planet, In other words, the probability is either very close to 0 or close to 1.0, but unlikely to be some intermediate probability. The paper relies on the Beta function [Box 3] that uses the probability of positive and negative events defined by 2 parameters for the binary state of the event, e.g. life detected or not detected. This function can approximate the Binomial distribution, but can handle the different probability distributions.

Figure 1. The five different prior distributions as probability density functions (PDF) used in the paper and explained in Box 2. Note the Kerman and Jeffreys distributions that bias the probabilities at the extremes, compared to the “biased optimist” that has 3 habitable worlds around the sun (Venus, Earth, and Mars), but with only the Earth having life.

The Beta function is adjusted by the number of observations or positive and negative detections of biosignatures. At this point, the positive and negative observations are based on the believed prior distributions which can take any values, from guesses to preliminary observational results, which at this time are relatively few. After all, we are still arguing over whether we have even detected biosignature molecules, let alone confirmed their detection. We then adjust those expectations by the new observations.

What happens when we start a survey and gain events of biosignature detection? Using the Jeffreys prior distribution, let us see the effect of observing no biosignature detections for up to 100 negative biosignature observations.

Figure 2a. The effect of increasing the null observations on a skewed distribution that shows the increasing certainty of the low probability frequencies. While apparently the high probabilities also rise, the increase in null detections implies that the relative frequency of positives declines.

Figure 2b. The increasing certainty that the frequency of life on habitable planets tends towards 0 as the number of null biosignature detections increases. The starting value of 0.5 is taken from the Jeffreys prior distribution. The implied frequency is the new frequency of positives as the null detections reduce the frequency observed and push the PDF towards the lower bound of 0 (see figure 1)

So far, so good. If we can be sure that the biosignature detection is unambiguous and that the inference that life is present or absent can be inferred with certainty based on the observations, then the sampling of up to 100 habitable worlds will indicate whether life is rare or ubiquitous and can be determined with high confidence. If every star system had at least 1 habitable world, this sample would include most stars within 20 ly of Earth. In reality, if we limit our stars to spectral types F, G & K, which represent 5-10% of all stars, and half of these have at least 1 habitable world, then we need to search 2000-4000 star systems, which are well within 100 ly, a tiny fraction of the galaxy.

The informed reader should now balk at the status of this analysis. Biosignatures are not unambiguous [4]. Firstly, detecting a faint trace of a presumed biosignature gas is not certain, as the phosphine on Venus and the DMS/DMDS on TOI-270d detections make clear. They are both controversial. In the case of Venus, we are neither certain that the phosphine signal is present and that the correct identification has been made, nor that there is no abiogenic mechanism to create phosphine in Venus’ very different environment. As discussed in my post on the ambiguity of biosignatures, prior assumptions about biosignatures as unambiguous were reexamined, with the response that astrobiologists built a scale of certainties for assessing whether a planet is inhabited based on the contextual interpretation of biosignature data.[4].

The authors of the paper allow for this by modifying the formula to allow for both false-positive and false-negative biosignature detection rates, and also for interpretation uncertainty of the detected biosignature. The authors also calculate the upper bound at about 3 sigma (99.9%) of the frequency of observations. Figure 3 shows the effect of these uncertainties on the location and size of the maximal probability density function for the Jeffrey’s Bayesian priors.

Figure 3. The effects of sample and interpretation, best fit, and 99.9% uncertainties for null detections. As both sample and interpretation uncertainty increase, the expected number of positive detections increases. The Jeffrey prior’s distribution is used.

Figure 3 implies that with interpretation uncertainty of just 10%, even 100 null observations, the calculated frequency of life increases 2 orders of magnitude from 0.1% to 10%. The upper bound increases from less than 10% to between 20 and 30%. Therefore, even if 100 new observations of habitable planets with no detected biosignatures, the frequency of inhabited planets is between ⅕ and ⅓ of habitable planets at this level of certainty. As one can see from the asymptotes, no amount of further observations will increase the certainty that life is absent in the population of stars in the galaxy. Uncertainty is the gift that allows astrobiologists to maintain hope that there are living worlds to discover.

Lastly, the authors apply their methodology to 2 projects to discover habitable worlds; the Habitable Worlds Observatory [7] and the Large Interferometer for Exoplanets (LIFE} concepts. The analyses are shown in figure 4. The vertical lines indicate the expected number of positive detections by the conceptual methods and the expected frequencies of detections with their associated upper bounds due to uncertainty.

Figure 4. Given the uncertainties, the authors calculate the 99.9% ( > 3 sigma) upper limit on the null hypothesis of no life and matched against data obtained by 2 surveys by Morgan with The Habitable Worlds Observatory (HWO) and 2 by Kammerer with The Large Interferometer for Exoplanets (LIFE) [7, 8].

The authors note that it may be incorrect to use the term “habitable” if water is detected, or “living” if a biosignature[s] is detected. They suggest it would be better to just use the calculation for the detection method, rather than the implication of the detection, that is, that the sample uncertainty, but not the interpretation uncertainty, is calculated. As we see in the popular press, if a planet in the habitable zone (HZ) has about an Earth-size mass and density, this planet is sometimes referred to as “Earth 2.0” with all the implications of the comparison to our planet. However, we know that our current global biosphere and climate are relatively recent in Earth’s history. The Earth has experienced different states from anoxic atmosphere, to extremely hot, and conversely extremely cold periods in the past. It is even possible the world may be a dry desert, like Venus, or conversely a hycean world with no land for terrestrial organisms to evolve.

However, even if life and intelligence prove rare and very sparsely distributed, a single, unambiguous signature, whether of a living world or a signal with information, is detected, the authors state:

Last but not least we want to remind the reader here that, even if this paper is about null results, a single positive detection would be a watershed moment in humankind’s history.

In summary, Bayesian analysis of null detections against prior expectations of frequencies can provide some estimate of the upper limit frequency of living worlds, with many null detections reducing the frequencies and their upper limits. Using Fisherian statistics, many null detections would provide no such estimates, as all the data values would be 0 (null detections). The statistics would be uninformative other than that as the number of null detections increased, the expectation of the frequency of living worlds would qualitatively decrease.

While planetologists and astrobiologists would hope that they would observationally detect habitable and inhabited exoplanets, as the uncertainties are decreased and the number of observations continues to show null results, how long before such activities become a fringe, uneconomic activity that results in lost opportunity costs for other uses of expensive telescope time?

The paper is Angerhausen, D., Balbi, A., Kovačević, A. B., Garvin, E. O., & Quanz, S. P. (2025). “What if we Find Nothing? Bayesian Analysis of the Statistical Information of Null Results in Future Exoplanet Habitability and Biosignature Surveys”. The Astronomical Journal, 169(5), 238. https://doi.org/10.3847/1538-3881/adb96d

References

1. Wikipedia “Drake equation” https://en.wikipedia.org/wiki/Drake_equation. Accessed 04/12/2025

2. Kipping & Lewis, “Do SETI Optimists Have a Fine-Tuning Problem?” submitted to International Journal of Astrobiology (preprint). https://arxiv.org/abs/2407.07097

3. Gilster P. “The Odds on an Empty Cosmos“ Centauri Dreams, Aug 16, 2024 https://www.centauri-dreams.org/2024/08/16/the-odds-on-an-empty-cosmos/

4. Tolley A. “The Ambiguity of Exoplanet Biosignatures“ Centauri Dreams Jun 21, 2024
https://www.centauri-dreams.org/2024/06/21/the-ambiguity-of-exoplanet-biosignatures/

5. Foote, Searra, Walker, Sara, et al. “False Positives and the Challenge of Testing the Alien Hypothesis.” Astrobiology, vol. 23, no. 11, Nov. 2023, pp. 1189–201. https://doi.org/10.1089/ast.2023.0005.

6. Tolley, A. Our Earliest Ancestor Appeared Soon After Earth Formed. Centauri Dreams, Aug 28, 2024 https://www.centauri-dreams.org/2024/08/28/our-earliest-ancestor-appeared-soon-after-earth-formed/

7. Wikipedia “Habitable Worlds Observatory” https://en.wikipedia.org/wiki/Habitable_Worlds_Observatory. Accessed 05/02/2025

8. Kammerer, J. et al (2022) “Large Interferometer For Exoplanets (LIFE) – VI. Detecting rocky exoplanets in the habitable zones of Sun-like stars. A&A, 668 (2022) A52
DOI: https://doi.org/10.1051/0004-6361/202243846