We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent e452465 commit c1f3eabCopy full SHA for c1f3eab
notes/residual-connections.md
@@ -58,7 +58,6 @@ Gradient = 0.1 * 0.1 * 0.1 = 0.001
58
59
With a skip connection this becomes:
60
```console
61
-```console
62
x_1 = x_0 + F(x_0)
63
x_2 = x_1 + G(x_1)
64
x_3 = x_2 + H(x_2)
@@ -86,6 +85,7 @@ Total Gradient = (1 + 0.1) * (1 + 0.1) * (1 + 0.1)
86
85
Total Gradient = 1.1 * 1.1 * 1.1 = 1.331
87
```
88
Instead of shrinking to 0.001, the signal stayed strong at 1.331.
+```
89
90
(1 + H'(x_2)) * (1 + G'(x_1)) * (1 + F'(x_0))
91
↓
0 commit comments