4 weeks is waaaaaaaayyyyy too young to be away from mum and siblings. He should be a minimum of 8 weeks old. There is a lot of vital early socialization from the dam and siblings that happen in those last 4 weeks with mum and siblings. If he is indeed only 4 weeks, he's better off going back to the breeder if the breeder will take him back, that would absolutely be the best option for this pup - at least for the next 4 weeks.
That said, it sounds like you're suffering from what we call the Puppy Blues. It will pass. Just take each day at a time. Your life has been turned upside down by a little land crocodile who doesn't know where the toilet is or how to play with humans. It happens to the best of us.