statistics

Recently I&rsquo;ve been thinking about the LogSumExp trick since it is used in the integration step of nested sampling. I won&rsquo;t go over too much of the math here, but the reason this trick exists is to greatly increase the numerical stability of the operation $$\log \sum_i \exp x_i$$ via the identity $$a + \log \sum_i \exp\left(x_i - a\right)$$ Naive implementations In Julia we can implement a naive logsumexp with logsumexp_naive(X) = log(sum(exp, X)) let&rsquo;s test the numerical accuracy against Julia&rsquo;s BigFloat for some very large numbers
2021-04-19