Implemented dropout, starting convnets

[gstvtrp]

My Deeplearning skills are growing every day.

 

[idkwattodowithlife]

Nice. Do you prefer deep learning over machine learning?

 

[gstvtrp]

Nice. Do you prefer deep learning over machine learning?

kinda yeah even though DL is a subset of ML. I believe in connectionist approaches overall. I don't think there is any other path to AGI.

 

[nausea]

My Deeplearning skills are growing every day.

so u claim

Nice.

blob

 

[gstvtrp]

so u claim

blob

oh no, you too? Did the lowiq bug get you as well?

 

[nausea]

oh no, you too? Did the lowiq bug get you as well?

u didnt execute my plan

 

[Deleted member 3291]

I spent like 3 hours last week trying to understand back propagation lol

 

[gstvtrp]

I spent like 3 hours last week trying to understand back propagation lol

You get used to it. Try watching the 3blue1brown videos on neural networks.

Basically, we have a loss function right? We want to minimize the loss. If we imagine the loss as a hilly terrain, then we want to find the low points in the terrain (local minima). To find the direction we should move in, we need to know the gradient of the loss with respect to the parameters. This is where the chain rule comes in. As we backprop we get the gradients for each layer.

Then with SGD, you simply update the parameters W of a layer by W := W - learning_rate * dW.

 

[Deleted member 3291]

You get used to it. Try watching the 3blue1brown videos on neural networks.

Basically, we have a loss function right? We want to minimize the loss. If we imagine the loss as a hilly terrain, then we want to find the low points in the terrain (local minima). To find the direction we should move in, we need to know the gradient of the loss with respect to the parameters. This is where the chain rule comes in. As we backprop we get the gradients for each layer.

Then with SGD, you simply update the parameters W of a layer by W := W - learning_rate * dW.

I used to watch everything from 3blue1brown, he puts a lot of effort.

I understood it a little better after his videos and a few from udacity, I guess I had a few problems with the backprop calc

 

[gstvtrp]

I used to watch everything from 3blue1brown, he puts a lot of effort.

I understood it a little better after his videos and a few from udacity, I guess I had a few problems with the backprop calc

Keep at it man. When I first tried learning backprop, I was like wtf? Now it makes complete sense to me but it took a while. You should try the Coursera course by Andrew Ng. Even in that course Ng says sometimes he felt like he didn't understand backprop. Of course by now he probably does understand it completely. I think this is a very common thing among beginner ML practitioners.

 

[gstvtrp]

LOL I just read the title, I meant to say implemented DROPOUT lol. Of course I've implemented backprop before.

First time I've done Dropout. JFL.

 

[Deleted member 3291]

Keep at it man. When I first tried learning backprop, I was like wtf? Now it makes complete sense to me but it took a while. You should try the Coursera course by Andrew Ng. Even in that course Ng says sometimes he felt like he didn't understand backprop. Of course by now he probably does understand it completely. I think this is a very common thing among beginner ML practitioners.

yup, was planning to after I refreshed on linear algebra

 

[Indari]

tfw not high iq programmer. tfw rarted laborer

 

[incels.REEEE]

I spent like 3 hours last week trying to understand back propagation lol

My professor made me memorize the entire equation. I did, but now I've forgotten all of it.

Nice. Do you prefer deep learning over machine learning?

Deep learning is the best approach as a whole but sucks to program/work with. Machine learning makes for more enjoyable programming since you are the one doing the thinking and finding patterns, not your computer. I think deep learning will be what most programs use in the future which is unfortunate imo.

 

[gstvtrp]

fuarkk, I implemented forward and backward passes for a convolutional layer. That was pretty tough. I think that's one of the tougher things in the whole class.

 

[gstvtrp]

fuarrr, I'm getting even further. Implemented a 3-layer convnet and also spatial batchnorm. Almost done with assignment 2!

 

[PM_ME_STRIPPERS]

dropout = ?

 

[gstvtrp]

dropout = ?

A technique where some neurons are probabilistically turned off during training time. This has the effect of helping a neural network generalize better.

 

[PM_ME_STRIPPERS]

A technique where some neurons are probabilistically turned off during training time. This has the effect of helping a neural network generalize better.

i still dont understand kek