4F13Equations

The probability of a test document using the Bayesian model is:
\begin{equation*}
    p(\Tilde{D}|D,\alpha)
\end{equation*}
As we work through the document observing words $\Tilde{x}_{1}, \Tilde{x}_{2}, \dots, \Tilde{x}_{N^{obs}}$ we need to include the word in our data:
\begin{equation*}
\begin{split}
    p(\Tilde{D}|D,\alpha) &= p(\Tilde{x}_{1:N^{obs}}|D)\\
    &= p(\Tilde{x}_{1}|D)p(\Tilde{x}_{2}|D,\Tilde{x}_{1})\dots p(\Tilde{x}_{N^{obs}}|D,\Tilde{x}_{1}, \Tilde{x}_{2}, \dots, \Tilde{x}_{N^{obs} - 1})
\end{split}
\end{equation*}
We already have:
\begin{equation*}
\begin{split}
    p(X=m|D,\alpha) = \frac{\alpha_m + c_m}{ \alpha  + c}
\end{split}
\end{equation*}
When we substitute this into the above to update the counts  and then write as $\Gamma$ functions we get:
\begin{equation*}
\begin{split}
    p(\Tilde{D}|D,\alpha) &= \prod_{m=1}^M \prod_{i=0}^{c_m^{obs}-1}\frac{\alpha_m + c_m^{old} + i}{\alpha + c^{old} + i} \\
    &= \frac{\Gamma(\alpha + c^{old})}{\Gamma(\alpha + c^{old} + c^{obs})} \prod_{m=1}^{M}\frac{\Gamma(\alpha_m + c_m^{old} + c_m^{obs})}{\Gamma(\alpha_m + c_m^{old})}
\end{split}
\end{equation*}
The log probability is:
\begin{equation*}
\begin{split}
    \log p(\Tilde{D}|D,\alpha) = &\log (\Gamma(\alpha + c^{old})) - \log (\Gamma(\alpha + c^{old} + c^{obs})) \\ & + \sum _{m=1}^{M} (\log (\Gamma(\alpha_m + c_m^{old} + c_m^{obs})) - \log (\Gamma(\alpha_m + c_m^{old})))
\end{split}
\end{equation*}