Recently I’ve been thinking about the LogSumExp trick since it is used in the integration step of nested sampling. I won’t go over too much of the math here, but the reason this trick exists is to greatly increase the numerical stability of the operation
$$ \log \sum_i \exp x_i $$
via the identity
$$ a + \log \sum_i \exp\left(x_i - a\right) $$
Naive implementations In Julia we can implement a naive logsumexp with
logsumexp_naive(X) = log(sum(exp, X)) let’s test the numerical accuracy against Julia’s BigFloat for some very large numbers

2021-04-19

3 min read