Isn't circling the Kaaba and kissing the Black Stone a form of idolatry?

No. Muslims worship neither the Kaaba nor the Black Stone. The Kaaba is a simple house of stone and a sign of the Oneness of God and the unity of mankind.

Idolatry means worshipping something to which one ascribes divine power. That is exactly what Muslims do not do with the Kaaba. It is a house made of stone, nothing more. No one asks it for help, and no one believes that God lives inside it. The second verse above says it plainly: whichever way you turn, there is God.

Why then a shared direction? Islam teaches the Oneness of God, and from it the unity of people should grow. When Muslims all over the world pray facing the same way, this unity becomes visible. Some direction had to be chosen, and the Kaaba has a special claim. According to the Holy Qur’an it is the oldest house built for the worship of the One God and for all people, not for one nation or one religion alone.

The Black Stone is not worshipped either. Kissing it is an expression of love and remembrance. Think of a letter written by someone you love. You might kiss it, yet no one considers the paper holy or expects help from it. The hadith above shows that the Companions of the Holy Prophet (saw) thought exactly this way.

So the Kaaba is not an object of worship. It is a sign. It reminds us that there is only one God, and that all people are one family before Him.