Math

Developing a Single-Pass Weighted LogSumExp Function

Recently I’ve been thinking about the LogSumExp trick since it is used in the integration step of nested sampling. I won’t go over too much of the math here, but the reason this trick exists is to greatly increase the numerical stability of the operation $$ \log \sum_i \exp x_i $$ via the identity $$ a + \log \sum_i \exp\left(x_i - a\right) $$ Naive implementations In Julia we can implement a naive logsumexp with logsumexp_naive(X) = log(sum(exp, X)) let’s test the numerical accuracy against Julia’s BigFloat for some very large numbers