However, we cannot say that the ENTIRE idea of Orlah is that one may not eat the fruits for the first three years so that he can eat them in Yerushalayim with kedusha in the fourth, because there are Amoraim [Brachos 35a] who hold that רבעי [eating in the fourth year] applies only to a כרם [vineyard] while everyone holds that Orlah applies to all trees. So we see that there is more to Orlah than just not eating for three years in order to bring the first fruits to Hashem in the fourth [as the Ramban said and compared it to Bikkurim] because most fruits don't have a din of רבעי and Orlah stands alone.
In addition the Rambam [Maachalos Asuros 10-15] holds that נטע רבעי [eating in the 4th year] doesn't apply to fruits in חוץ לארץ and only Orlah applies. But according to the Ramban there should be no איסור ערלה since there is no רבעי?! So again we see that ערלה and רבעי are not necessarily linked.
Rashi [סוטה מ"ג: ד"ה קלא אית להו] writes that according to the opinion that רבעי only applies only to a כרם, there is no ערלה if there is only one sapling. There must be a complete כרם. The Rishonim [Baal Hamaor Brachos 35, Ramban Rosh Hashana 10a, R' Akiva Eiger Gilyonei Hashas, Chasam Sofer Yo"d 285] wonder where Rashi gets this from. The Sefer Hashlam explains that Rashi certainly holds that ערלה applies even to one tree and Rashi means that נטע רבעי applies only to a complete כרם [I hope to explain why he writes "ערלה" and not "נטע רבעי"]. If so, how does one tree have a din of ערלה if there is no din of נטע רבעי [according to the Ramban who links the two].
We see again from this that ערלה applies to fruits independent of the obligation to being the fruits fruits to Hashem in Yerushalayim. So the rationale for the איסור can be as the Ramban writes further that it is unhealthy to eat fruits in the first three years or as the Rambam says in the Moreh Nevuchim [quoted by the Ramban] that the idea is to prevent witchcraft or for other possible reasons. However, the first reason mentioned by the Ramban still applies [as implied by the words of the pasuk, as we explained earlier]. Therefore, a tree planted for a fence which by definition can't be brought to eat before Hashem, because it is not a tree for eating and thus precluded from the mitzva of נטע רבעי, is not subject to the איסור of ערלה [even though the other reasons for the איסור still apply]. Just in חוץ לארץ or when it is not a כרם [according to the opinion that only a כרם is obligated in רבעי] even though there is no din to bring it to Yerushalayim before Hashem, since there is no הפקעה [preclusion] in its essence from being brought before Hashem, the איסור of ערלה applies.
