It is completely forbidden to punish a toddler. The punishment does not help from an educational standpoint. On the contrary, it causes damage. A child of that age does not possess the emotional or intellectual ability to transform a punishment into a positive lesson.
Even the best, non-Jewish educators reached the conclusion that punishments cannot fix everything. This is a great advancement when compared to the state which existed in Europe a hundred years ago where children would be hit in a cruel manner and called horrible names.
We find teachings against child punishment in the writings our Sages from France from more than 500 years ago. See, for example, Rabbi Eliyahu De Vidas’ famous work "Reishit Chochma", in the chapter on raising children. Similarly, Rabbi Moshe ben Machir in his book "Seder Ha-Yom" on the Mishnah "A five-year old begins learning Chumash," where he explains that before that age one should not cause distress to a child since he is incapable of learning proper lessons from it. He does not say that a child of that age does not possess intellect, but that he is not able to understand cause and effect. Therefore, do not punish a toddler, but help to build him through his own intellect.