Some Mac OSX packages come with uninstall scripts, some dont, others you just drag the app out the application folder and into the trash. Xcode most likely dumps a lot of stuff all over the file system, to do a clean uninstall just run this…

sudo /Developer/Library/uninstall-devtools –mode=all

Took about 6-7 minutes on my 2.4GHz iMac