Hi,
What did your vet do? Did he take any x-rays? Palpate his spine? Check his legs, range of motion? If so, did your dog react at all? What meds did he put your dog on?
It does sound like your dog could have hurt his spine. Usually when a dog cries that much and that long, with being uncomfortable, it is an emergency!
My first thought was torn cruciate in his knee. Then further reading of your post, I thought maybe a ruptured disk. If he fell hard on the ice, he could have bruised his shoulder as well.
If you don't keep anything on hand for pain, I would get Benedryl (you can give your boy 2 pills, every 8 hours.) and definitely keep buffered aspirin on hand. The Benedryl is not only good for allergies, but will help him relax. Also to keep on hand, Pepto Bismal tablets and Tagamet (sp?).
You may want to look for a doggy accupunturist as that will help with pain a great deal. I've done this with some of my dogs and it helped tremendeously!
When older large dogs injure themselves and get this worried another concern is Gastric Tortion. So you will need to watch him for any signs of this as it's life threatening. It's also why I suggested that you keep Tagamet on hand. Ask your vet if you can give him Tagamet with the meds that he is on now.
I really hope that your boy is ok. And that it was just a muscle pull! It's so hard to see your dog(s) in pain.
Sorry for such a long post!
Good luck! And let us know what happens.