The pasuk in Mishlei [18/1] says לתאוה יבקש נפרד - He who is separated seeks lust.
Explains the Gra:
"כלומר, תיכף כאשר האדם נפרד מן התורה הוא רודף אחר התאווה ומבקש אותה, כי התורה הוא משבר את התאווה כמ"ש למעלה, וכשהוא נפרד מן התורה הוא רודף אחר התאוה".
Immediately when one separates from Torah he runs after illicit desire, because Torah breaks desire and when he separates from Torah, he pursues desire.
He also says on the pasuk נפש שבעה תבוס נפת ונפש רעבה כל מר מתוק:
A sated soul tramples honeycomb, but to a hungry soul all bitter is sweet.
"נפש שבעה תבוס נפת - ועוד, נפש שבעה - כאשר ימלא האדם תאוות נפשו, אז היא תבוס נפת - שאינו חפץ בתורה, אבל נפש רעבה - שאינו ממלא תאוותו, אלא מסגף עצמו ומרעיב את עצמו. כל מר - כל היגיעות והמרירות יהיה לו מתוק וחביב מאוד, כי יגיע וישיג מתיקות התורה".
When a person fulfills his lusts and desires, then he won't desire Torah, he will "trample the honeycomb", the sweetness of Torah. But "a hungry soul", that doesn't fulfill his desires but afflicts himself, "all bitter" - all of the toiling in Torah, "will be sweet", because he learned and comprehended Torah.
The reason is as Rav Aharon Kotler explains in Mishnas Rebbe Aharon [vol. 1 p.43] as the Chovos Halevavos says, if a person is in love with this worldly pleasures he blocks himself off from Torah.
A POWERFUL Sfas Emes:
The untershte shura, the bottom line [for our purposes] of this shtickel Sfas Emes is that to the extent that we separate ourselves from the pleasures of this world, we will be able to enjoy the taste of Torah and mitzvos. The more we are immersed in this worldly pleasures, the less we will taste kedusha.
I don't think that this means that we have to be ascetics. We should eat and drink etc. whatever our body requires - but not more. It is bad for the body to overeat and bad for the soul as well.
Dovid Hamelech said לעשות רצונך א-להי חפצתי ותורתך בתוך מעי - I desired to do your will Hashem and your Torah is in my stomach.
If a person has a sincere desire to fulfill the will of Hashem and therefore focuses on ruchniyus and not on filling his belly, then the belly will indeed be filled with Torah.
As the medrash says [cited by Tosfos in Ksubos]:
"עד שאתה מתפלל שיכנסו ד"ת בתוך מעיך, התפלל שלא יכנסו מעדנים לתוך גופך"
Before you daven that Torah should enter your belly, daven that delicacies DON'T enter your body!