Last few weeks i spent a lot of time searching on internet to watch Youtube HD videos on PS3, there were a few solutions available, but the following worked for me and is the simplest.
1. Open browser on ps3
2. Goto Ps3Youtube.com
3. There is a search box on top, enter your keywords (you may add word HD in your search criteria)
4. From the search list click on a video and play it, thats it !!
I know there are some other instructions on website in green colored text but playing the video on same site worked for me.
I hope this helps, searching through internet i found lot of people feeling this pain of unable to watch youtube HD videos and low quality that is provided by Youtube XL.
Good luck !