Archive for likelihood-free

to the early Universe and back

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , on November 16, 2025 by xi'an

On 28 October, I spent the day at Institut d’Astrophysique de Paris (where I used to work on PMC for cosmology between 2005 and 2009), as a committee member for the habilitation defence of Florent Leclercq. Not only it was nice to be back in this unique institution (with vestiges from Laplace’s era), but this was a fantastic habilitation, with a superb thesis that beautifully gathered the different fields mastered by the candidate in a highly coherent discourse. And could serve as an introduction to cosmostatistics for many.

And provided the background to ten years (post-PhD) of research on forward modelling in cosmology and resulting Bayesian statistical analysis either by implicit likelihood (or likelihood-free) inference or by field-level inference. He describes the Simbelmynë software he developed to produce maps of the density field and analyse dark matter dynamics. Ẁhose name is borrowed from Tolkien (along with a quote from Guy Gavriel Kay!):

“How fair are the bright eyes in the grass! Evermind they are called, simbelmynë in this land of Men, for they blossom in all the seasons of the year, and grow where dead men rest.” — J.R.R. Tolkien, The Lord of the Ring

And the Bayesian computational and modelling tools he elaborated, like SELFI (Simulator expansion for likelihood-free inference, Leclercq et al., 2019), that relates to Michael Gutmann’s and Juka Corander’s BOLFI. (Obviously, I did not get every aspect right from just reading the thesis and attending the lecture, in particular the remarks on using SELFI to assess model misspecification. But I remain impressed by the scope of the work and its likely impact on the field!)

Leave a comment »

approximate maximum likelihood estimation using data-cloning ABC

Posted in Books, Statistics, University life with tags , , , , , , , , on June 2, 2015 by xi'an

“By accepting of having obtained a poor approximation to the posterior, except for the location of its main mode, we switch to maximum likelihood estimation.”

Presumably the first paper ever quoting from the ‘Og! Indeed, Umberto Picchini arXived a paper about a technique merging ABC with prior feedback (rechristened data cloning by S. Lele), where a maximum likelihood estimate is produced by an ABC-MCMC algorithm. For state-space models. This relates to an earlier paper by Fabio Rubio and Adam Johansen (Warwick), who also suggested using ABC to approximate the maximum likelihood estimate. Here, the idea is to use an increasing number of replicates of the latent variables, as in our SAME algorithm, to spike the posterior around the maximum of the (observed) likelihood. An ABC version of this posterior returns a mean value as an approximate maximum likelihood estimate.

“This is a so-called “likelihood-free” approach [Sisson and Fan, 2011], meaning that knowledge of the complete expression for the likelihood function is not required.”

The above remark is sort of inappropriate in that it applies to a non-ABC setting where the latent variables are simulated from the exact marginal distributions, that is, unconditional on the data, and hence their density cancels in the Metropolis-Hastings ratio. This pre-dates ABC by a few years, since this was an early version of particle filter.

“In this work we are explicitly avoiding the most typical usage of ABC, where the posterior is conditional on summary statistics of data S(y), rather than y.”

Another point I find rather negative in that, for state-space models, using the entire time-series as a “summary statistic” is unlikely to produce a good approximation.

The discussion on the respective choices of the ABC tolerance δ and on the prior feedback number of copies K is quite interesting, in that Umberto Picchini suggests setting δ first before increasing the number of copies. However, since the posterior gets more and more peaked as K increases, the consequences on the acceptance rate of the related ABC algorithm are unclear. Another interesting feature is that the underlying MCMC proposal on the parameter θ is an independent proposal, tuned during the warm-up stage of the algorithm. Since the tuning is repeated at each temperature, there are some loose ends as to whether or not it is a genuine Markov chain method. The same question arises when considering that additional past replicas need to be simulated when K increases. (Although they can be considered as virtual components of a vector made of an infinite number of replicas, to be used when needed.)

The simulation study involves a regular regression with 101 observations, a stochastic Gompertz model studied by Sophie Donnet, Jean-Louis Foulley, and Adeline Samson in 2010. With 12 points. And a simple Markov model. Again with 12 points. While the ABC-DC solutions are close enough to the true MLEs whenever available, a comparison with the cheaper ABC Bayes estimates would have been of interest as well.

3 Comments »