Send him one that says: Announce yourself if you want something rather than just “stopping by". Not all of my work happens in an office or a classroom.

-or-

Classroom time and mandatory meeting times are not open office times. That would be inappropriate.