• API
• FAQ
• Tools
• Archive
SHARE
TWEET # Untitled a guest Mar 14th, 2018 74 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
1. \\
2. Feed \ forwarding \\
3. net_1=xw_{1}+b \\
4. h=\sigma (net_1) \\
5. net_2=hw_{2}+b \\
6. {y}'=\sigma (net_2) \\ \\
7.
8. Loss \ function \\
9. L=\frac{1}{2}\sum(y-{y}')^{2} \\ \\
10.
11. Gradient \ calculation \ (Refer \ mattmazur's \ and \ DattA's tutorial)\\ \\
12. \frac{\partial L}{\partial w_{2}}=\frac{\partial net_2}{\partial w_2}\frac{\partial {y}' }{\partial net_2}\frac{\partial L }{\partial {y}'} \\ \\
13.
14. \frac{\partial L}{\partial w_{1}}= \frac{\partial net_1}{\partial w_{1}} \frac{\partial h}{\partial net_1}\frac{\partial net_2}{\partial h}\frac{\partial {y}' }{\partial net_2}\frac{\partial L }{\partial {y}'} \\ \\ \\
15.
16. Where: \\ \\
17. \frac{\partial L }{\partial {y}'}=\frac{\partial (\frac{1}{2}\sum(y-{y}')^{2})}{\partial {y}'}=({y}'-y) \\ \\
18. \frac{\partial {y}' }{\partial net_2}={y}'(1-{y}')\\ \\
19. \frac{\partial net_2}{\partial w_2}= \frac{\partial(hw_{2}+b) }{\partial w_2}=h \\ \\
20. \frac{\partial net_2}{\partial h}=\frac{\partial (hw_{2}+b) }{\partial h}=w_2 \\
21. \frac{\partial h}{\partial net_1}=h(1-h) \\ \\
22. \frac{\partial net_1}{\partial w_{1}}= \frac{\partial(xw_{1}+b) }{\partial w_1}=x \\ \\
23.
24. \\
25. The \ gradients \ can \ be \ rewritten \ as: \\ \\
26. \frac{\partial L }{\partial w_2 }=h\times {y}'(1-{y}')\times ({y}'-y) \\ \\
27. \frac{\partial L}{\partial w_{1}}=x\times h(1-h)\times  w_2 \times {y}'(1-{y}')\times ({y}'-y) \\ \\
28.
29. Weight \ update \\
30. w_{i}^{t+1} \leftarrow w_{i}^{t}-\alpha \frac{\partial L}{\partial w_{i}}
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.

Top