Entering The Holy Of Holies

The Rambam [based on Chazal] says that the staff of Aharon was placed in the Kodesh Hakodashim [בית הבחירה ד-א]. 

How was he allowed in there outside of Yom Kippur??