[To save time I will not be giving exact mareh mekomos. Anyone who wants can always email me. I will be drawing heavily on the kuntres on Yichud in Tzitz Eliezer 6/40]
WEEEEE BEGGGGIIIIINN!!!!!!!!!!!!!!
A simple reading of the gemara in avoda zara [36b] reveals that it is a TORAH prohibition. This is in fact the opinion of almost all the rishonim [Rabbeinu Chananel Rashi Tosfos and many others]. However the Rambam holds that it is "מפי קבלה" which means either a post biblical decree or a tradition from Sinai.
Some people view yichud as a type of fence that people with self control don't need. The truth is that according to all opinions it is absolutely forbidden regardless of one's spiritual stature.