SHARE
TWEET

Untitled

a guest Mar 14th, 2018 74 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. \\
  2. Feed \ forwarding \\
  3. net_1=xw_{1}+b \\
  4. h=\sigma (net_1) \\
  5. net_2=hw_{2}+b \\
  6. {y}'=\sigma (net_2) \\ \\
  7.  
  8. Loss \ function \\
  9. L=\frac{1}{2}\sum(y-{y}')^{2} \\ \\
  10.  
  11. Gradient \ calculation \ (Refer \ mattmazur's \ and \ DattA's tutorial)\\ \\
  12. \frac{\partial L}{\partial w_{2}}=\frac{\partial net_2}{\partial w_2}\frac{\partial {y}' }{\partial net_2}\frac{\partial L }{\partial {y}'} \\ \\
  13.  
  14. \frac{\partial L}{\partial w_{1}}= \frac{\partial net_1}{\partial w_{1}} \frac{\partial h}{\partial net_1}\frac{\partial net_2}{\partial h}\frac{\partial {y}' }{\partial net_2}\frac{\partial L }{\partial {y}'} \\ \\ \\
  15.  
  16. Where: \\ \\
  17. \frac{\partial L }{\partial {y}'}=\frac{\partial (\frac{1}{2}\sum(y-{y}')^{2})}{\partial {y}'}=({y}'-y) \\ \\
  18. \frac{\partial {y}' }{\partial net_2}={y}'(1-{y}')\\ \\
  19. \frac{\partial net_2}{\partial w_2}= \frac{\partial(hw_{2}+b) }{\partial w_2}=h \\ \\
  20. \frac{\partial net_2}{\partial h}=\frac{\partial (hw_{2}+b) }{\partial h}=w_2 \\
  21. \frac{\partial h}{\partial net_1}=h(1-h) \\ \\
  22. \frac{\partial net_1}{\partial w_{1}}= \frac{\partial(xw_{1}+b) }{\partial w_1}=x \\ \\
  23.  
  24. \\
  25. The \ gradients \ can \ be \ rewritten \ as: \\ \\
  26. \frac{\partial L }{\partial w_2 }=h\times {y}'(1-{y}')\times ({y}'-y) \\ \\
  27. \frac{\partial L}{\partial w_{1}}=x\times h(1-h)\times  w_2 \times {y}'(1-{y}')\times ({y}'-y) \\ \\
  28.  
  29. Weight \ update \\
  30. w_{i}^{t+1} \leftarrow w_{i}^{t}-\alpha \frac{\partial L}{\partial w_{i}}
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top