In deep learning, the optimization techniques are for the most part dependent on gradient descent methods, such as SGD, ADAM which adopt the leading place in the area of optimization methods.