One of the common weapons used against the Olam HaTorah is the Rambam.
Such as...
"The Rambam says that Yeshiva boys have to serve in the army."
"The Rambam says that one has to receive a thorough secular education".
"The Rambam says that Kollel is אסור".
As if the Gedolei HaDor today and in the past never learned the Rambam or just don't care and the only truly pious Jews are those who truly follow the Rambam.
---
Response [Note: This writer is a super Rambam enthusiast who spends much of his time plumbing the depths of his wisdom].
1] The opinion of the Rambam is often far more nuanced than it seems at first glance. One has to see what he says in different places and go delve deeply into his intention.
2] The Rambam in certain cases was talking about a different reality than we are living in today.
3] The Rambam is not the only opinion. There have been other חכמי ישראל whom we often follow against the Rambam.
4] There are plenty of Rambam's that the Rambam-wavers themselves don't keep [see Rambam Teshuva 10-3 and De'os 6-3 and many others] or even believe in [such as Yesodei HaTorah 3-9].
Part of the מדה of עין טובה is to attribute sincere intentions to others and particularly to people whose entire lives revolve around Torah.
In general, it is clear that much of what people believe didn't *start* with the Rambam. First people believed something. Then they saw that the Rambam also [seemingly] says the same thing. It is like the idea of "Modern Orthodoxy". First people were Modern Orthodox [which used to mean - and still does in some places - that people only keep only some of the Torah] and then a philosophy was built around it trying to show how it is all rooted in tradition.