Archive for David MacKay

David MacKay  remembrance day [27 March, Cambridge]

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , on February 8, 2026 by xi'an


[Re-posted:] Professor Sir David MacKay (1967-2016) made fundamental contributions to both public and theoretical understandings of energy and of information. He served as Chief Scientific Adviser to the Department of Energy and Climate Change and was Professor of Natural Philosophy in the Cavendish Laboratory before being appointed as the inaugural Regius Professor of Engineering. He was a Fellow of Darwin College.

This one-day meeting of the Cambridge Philosophical Society, dedicated to his memory, considers both the urgent challenges of sustainable energy resources and the global opportunities arising from information technologies. We will be addressing the two main themes of his work: machine learning, information theory and Bayesian inference, together with sustainable energy. The meeting marks the tenth anniversary of David’s death, with speakers who worked with David, build on his contributions in the field of energy and information, and share his values on the importance of clear and accessible communication.

The meeting in Cambridge University Engineering Department is open to all to attend, without charge. The lectures will be live-streamed; edited recordings will later be made available through the Cambridge Philosophical Society website. Registration for both in-person and virtual attendance is recommended.

David MacKay is remembered by many with great affection and respect as colleague, mentor and friend, and it will be good to share these memories. We have set up a webform for you to provide a personal tribute if you wish.

Leave a comment »

back to Ockham’s razor

Posted in Statistics with tags , , , , , , , , , on July 31, 2019 by xi'an

“All in all, the Bayesian argument for selecting the MAP model as the single ‘best’ model is suggestive but not compelling.”

Last month, Jonty Rougier and Carey Priebe arXived a paper on Ockham’s factor, with a generalisation of a prior distribution acting as a regulariser, R(θ). Calling on the late David MacKay to argue that the evidence involves the correct penalising factor although they acknowledge that his central argument is not absolutely convincing, being based on a first-order Laplace approximation to the posterior distribution and hence “dubious”. The current approach stems from the candidate’s formula that is already at the core of Sid Chib’s method. The log evidence then decomposes as the sum of the maximum log-likelihood minus the log of the posterior-to-prior ratio at the MAP estimator. Called the flexibility.

“Defining model complexity as flexibility unifies the Bayesian and Frequentist justifications for selecting a single model by maximizing the evidence.”

While they bring forward rational arguments to consider this as a measure model complexity, it remains at an informal level in that other functions of this ratio could be used as well. This is especially hard to accept by non-Bayesians in that it (seriously) depends on the choice of the prior distribution, as all transforms of the evidence would. I am thus skeptical about the reception of the argument by frequentists…

3 Comments »

convergence of MCMC

Posted in Statistics with tags , , , , , , , , , on June 16, 2017 by xi'an

Michael Betancourt just posted on arXiv an historical  review piece on the convergence of MCMC, with a physical perspective.

“The success of these of Markov chain Monte Carlo, however, contributed to its own demise.”

The discourse proceeds through augmented [reality!] versions of MCMC algorithms taking advantage of the shape and nature of the target distribution, like Langevin diffusions [which cannot be simulated directly and exactly at the same time] in statistics and molecular dynamics in physics. (Which reminded me of the two parallel threads at the ICMS workshop we had a few years ago.) Merging into hybrid Monte Carlo, morphing into Hamiltonian Monte Carlo under the quills of Radford Neal and David MacKay in the 1990’s. It is a short entry (and so is this post), with some background already well-known to the community, but it nonetheless provides a perspective and references rarely mentioned in statistics.

Leave a comment »