Your problem is that you are not just training him to go outside, you are also having to re-train him not to go inside; unfortunately that's what he has learned was acceptable until now.
If he is toileting within a couple of hours of being outside, take him out every hour. He may not have developed the muscle control to hold for longer (overnight, everything slows down and its different).
Make sure your reward is immediate and generous. It has to be immediate to be clear that it's for toileting and not for anything else and it has to be generous to make it worth his while to try to hold his toilet until he is outside.
If he is toileting within a couple of hours of being outside, take him out every hour. He may not have developed the muscle control to hold for longer (overnight, everything slows down and its different).
Make sure your reward is immediate and generous. It has to be immediate to be clear that it's for toileting and not for anything else and it has to be generous to make it worth his while to try to hold his toilet until he is outside.
