1.一层logistic regression

其中\(\iota \left ( a,y \right )\)定义的损失函数\(\iota \left ( a,y \right )= -yloga-(1-y)log(1-a)\)

则\(\frac{d}{da}\iota \left ( a,y \right )=-\frac{y}{a}+\frac{1-y}{1-a}\)

因为\(\sigma (z)=\frac{1}{1+e^{-z}}\),所以\(\frac{da}{dz}=\frac{1}{1+e^{-z}}(1-\frac{1}{1+e^{-z}})=a(1-a)\)

所以\(\frac{d}{dz}\iota \left ( a,y \right )=a(1-a)\cdot (-\frac{y}{a}+\frac{1-y}{1-a})=a-y\)

因为\(\frac{dz}{dw}=x\),所以\(\frac{dy}{dw}=\frac{dy}{dz}\cdot \frac{dz}{dw}=(a-y)\cdot x\)

同理可得\(\frac{dy}{db}=\frac{dy}{dz}\cdot \frac{dz}{db}=(a-y)\)

2.二层logistic regression

由一层logistic regression可得

\(\frac{d}{dz^{[2]}}\iota \left ( a^{[2]},y \right )=a^{[2]}(1-a^{[2]})\cdot (-\frac{y}{a^{[2]}}+\frac{1-y}{1-a^{[2]}})=a^{[2]}-y\)

\(\frac{dy}{dw^{[2]}}=\frac{dy}{dz^{[2]}}\cdot \frac{dz^{[2]}}{dw^{[2]}}=(a^{[2]}-y)\cdot a^{[1]T}\)

\(\frac{dy}{db^{[2]}}=\frac{dy}{dz^{[2]}}\cdot \frac{dz^{[2]}}{db^{[2]}}=(a^{[2]}-y)\)

则\(da^{[1]}=W^{[2]}\cdot dz^{[2]}\),则\(dz^{[1]}=\frac{dy}{da^{[1]}}\cdot \frac{da^{[1]}}{dz^{[1]}}==W^{[2]}\cdot dz^{[2]}\cdot \frac{da^{[1]}}{dz^{[1]}}\)

\(dw^{[1]}=\frac{dy}{dz^{[1]}}\cdot x^{T}\)

\(db^{[1]}=\frac{dy}{dz^{[1]}}\)