The state of nested sampling research in 2026: I) Scaling nested sampling to high-dimensions

Since I wrote a systematic literature review (arxiv, scholar) in 2023, and written or contributed to several nested sampling packages, I’m someone who people turn to and ask, “what is next in nested sampling research”?

In this first part, I will write about scaling nested sampling to high-dimensions.

Contrary to conventional wisdom, nested sampling does not have poor scaling with dimensionality at all. If the prior equals the posterior, i.e., no information gain, the algorithm terminates successfully immediately, with high effective sample size.

Two difficulties occur with the high information common with large datasets and hierarchical Bayesian models, where we learn an enormous amount of information. This can be easily seen with prior predictive checks – every prior realisation is plausible, but the observations are each highly peculiar in their own way, requiring a lot of fine tuning of the parameter space to find the posterior. This implies many nested sampling iterations, because in nested sampling, the number of iterations times the number of live points is proportional to the information gain: sqrt(N*K)=H

The second difficulty is that the likelihood-restricted prior sampling used may scale poorly with dimensionality. The best technique so far is hit-and-run Monte Carlo (HARM) – miscalled slice sampling – first proposed used T Jasa & N Xiang – but introduced in detail by Will Handley’s PolyChord. Slice sampling is somewhat of a misnomer because we are not sampling the slice height, and in classical slice sampling one samples one dimension at a time. In any case, such MCMC approaches scale with dimensionality in a theoretically simple but practically difficult-to-understand way. One needs to tune M, the number of MCMC steps until the next point is decorrelated from the starting location (a random live point). Realistically the minimum M may be different throughout the run, but the worst-case will dominate and thus sets M which sets the compute budget. Annoying.

Here are the current approaches:

1. Make the run code faster – this is engineering: Some hacks here include vectorization, analytic marginalisation of variables not explicitly needed, and changing programming language. 2 research groups work on jax-based nested sampling implementations and parallelising. From an algorithm point of view, there is nothing here. Good marketing though …
2. Focus on convex likelihoods: proximal nested sampling, ~1 researcher focused on imaging. This is an interesting approach not taken up by other research groups, yet.
3. Give up on doing it right the first time and anneal towards the answer.
   • Diffusive nested sampling frames NS sampling as a MCMC process, with 1 active researcher (Brendon Brewer). The range of proposals within Diffusive Nested Sampling has not yet been fully explored, and a jax-based implementation would be good. I’ve had no luck getting the algorithm to behave though.
   • Snowballing nested sampling: Instead of expanding the number of steps until you get a nicely behaved run, expand the number of live points with a fixed number of steps.

In my opinion, one of the most powerful techniques for making nested sampling run fast with many parameters is prior predictive checks. Often, the prior is misspecified, and information gain is unnecessarily high. Start from a reasonable prior.

In high dimensions, you should also ask yourself whether the parameters have a meaning. If you are dealing with millions of pixel parameters and morphology, what are you doing? Do you really want to do Bayesian model comparison or do you just not know any other model comparison methods that may be a better fit?

But what can you do. “High-dimensional inference” sounds cool and important, but I believe other inference axes are also worth pursuing, such as high information gain inference, phase transition inference, multi-modal inference.

Because of the change in nature of the parameters, I don’t think nested sampling is the ideal algorithm for high-dimensional inference. My go-to-methods are

a) divide and conquer importance resampling (as for example in PosteriorStacker). This is suitable for hierarchical models with low-dimensional per-object parameter spaces with medium to high information-gain per object. The parent distribution parameter space can be high dimensional.

b) dynamic HMC. I’ve seen several people die on the hill of getting Stan to fit realistic astronomy models though. Differentiable programming with instrument responses is tough.

The aversion of statisticians to nested sampling because they prefer SMC and the aversion of astrophysicists to SMC is a bit funny, given how close they are among the family of Monte Carlo algorithms. SMC requires some tuning of the tempering schedule, but has the advantage that one can potentially skip ahead to the posterior bulk. This is desirable for impatient researchers, such as those working on Gravitational Wave detections. A recent interesting paper was SMC+NUTS by Demasi+26.

see also part II.