I was reading the paper written by Taesup Kim and Yoshua Bengio] (2016) named Deep Directed Generative Models with Energy-Based Probability Estimation. arXiv:1606.03439

Introduction: Training energy-based probabilistic models is confronted with apparently intractable sums, whose Monte Carlo estimation requires sampling from the estimated probability distribution in the inner loop of training. This can be approximately achieved by Markov chain Monte Carlo methods, but may still face a formidable obstacle that is the difficulty of mixing between modes with sharp concentrations of probability. Whereas an MCMC process is usually derived from a given energy function based on mathematical considerations and requires an arbitrarily long time to obtain good and varied samples, we propose to train a deep directed generative model (not a Markov chain) so that its sampling distribution approximately matches the energy function that is being trained. Inspired by generative adversarial networks, the proposed framework involves training of two models that represent dual views of the estimated probability distribution: the energy function (mapping an input configuration to a scalar energy value) and the generator (mapping a noise vector to a generated configuration), both represented by deep neural networks.

The details of getting equation (7) of the original paper.

$$\begin{align*} P_{\mathrm{\Theta}}(\mathrm{x}) & = \frac{e^{-E_{\mathrm{\Theta}}(\mathrm{x})}}{Z_{\mathrm{\Theta}}} \quad Z_{\mathrm{\Theta}} =\sum_{\mathrm{x}}e^{-E_{\mathrm{\Theta}}(\mathrm{x})} \tag{1}\\ \frac{\partial\mathcal{L}(\mathrm{\Theta}, \mathcal{D}')}{\partial \mathrm{\Theta}} & = -\frac{1}{N}\sum_{i=1}^{N}\frac{\partial\log P_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}}\\ &=-\frac{1}{N}\sum_{i=1}^{N}\frac{\partial\log \frac{e^{-E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}}{Z_{\mathrm{\Theta}}}}{\partial\mathrm{\Theta}}\\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} + \frac{1}{N}\sum_{i=1}^{N}\frac{\partial \log Z_{\mathrm{\Theta}}}{\partial \mathrm{\Theta}} \\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} + \frac{\partial \log Z_{\mathrm{\Theta}}}{\partial \mathrm{\Theta}}\\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} + \frac{\partial \log\sum_{i=1}^{N}e^{-E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}}{\partial \mathrm{\Theta}} \\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} + \frac{1}{\sum_{i=1}^{N}e^{-E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}}\frac{\partial \sum_{i=1}^{N}e^{-E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}}{\partial \mathrm{\Theta}} \\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} + \frac{\sum_{i=1}^{N}e^{-E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}\frac{-\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}}}{\sum_{i=1}^{N}e^{-E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}} \\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} - \sum_{i=1}^{N}\frac{e^{-E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}}{Z_{\mathrm{\Theta}}}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial \mathrm{\Theta}}\\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} - \sum_{i=1}^{N}P_{\mathrm{\Theta}}(\mathrm{x}^{(i)})\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial \mathrm{\Theta}}\\ &=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x}^{(i)})}{\partial\mathrm{\Theta}} - E_{\mathrm{x}\sim P_{\mathrm{\Theta}(\mathrm{x})}}\left[\frac{\partial E_{\mathrm{\Theta}}(\mathrm{x})}{\partial \mathrm{\Theta}} \right]\tag{7} \end{align*}$$

Some middle steps in equation (8).

$$\begin{align*} P_{\psi}(y=1|x) & = \sigma (-E_{\psi}'(x))=\frac{1}{1+e^{E_{\psi}'(x)}} \\ \log P_{\psi}(y=1|x) & = 1- \log(1+e^{E_{\psi}'(x)})\\ \frac{\partial\log P_{\psi}(y=1|\mathrm{x}^{+})}{\partial \psi} & = - \frac{\partial \log(1+e^{E_{\psi}'(\mathrm{x}^{+})})}{\partial \psi} \\ &= -\frac{1}{1+e^{E_{\psi}'(x)}} \frac{\partial e^{E_{\psi}'(\mathrm{x}^{+})}}{\partial \psi}\\ &= -\frac{e^{E_{\psi}'(x)}}{1+e^{E_{\psi}'(x)}}\frac{\partial E_{\psi}'(\mathrm{x}^{+})}{\partial \psi}\\ & = -(1- \frac{1}{1+e^{E_{\psi}'(x)}})\frac{\partial E_{\psi}'(\mathrm{x}^{+})}{\partial \psi} \\ & = -(1- P_{\psi}(y=1|\mathrm{x}^{+}) )\frac{\partial E_{\psi}'(\mathrm{x}^{+})}{\partial \psi} \\ & = -P_{\psi}(y=0|\mathrm{x}^{+}) \frac{\partial E_{\psi}'(\mathrm{x}^{+})}{\partial \psi} \end{align*}$$