Observing the observer

Some months ago, I got into a habit of taking a couple of scienticfic papers and go to a nice little cafe here in Oxford on Sundays. It almost don't feel like work when you sit with cappucino on a cafe and it allowed me to read up on topics that were only borderline work-related. However, there was this middle-aged man who would almost always sit and draw people at the café. He only drew young women and only their faces. He didn't try to hide that he was drawing, in fact, when he had finished the drawing and he would come up to girl he had been drawing and show her the drawing. I didn't particularly fancy been drawn while I was reading, so I usually sat myself with my back against him. Then one day he came later than me, sat down at the table in front of me and began drawing. He sat there staring fixedly at me while drawing; it was extremely uncomfortable. I tried to hide behind my hair and otherwise express my discomfort. He nevertheless continued, for perhaps 20 minutes and then came over to me to show the drawing. It was a cartoony colourpencil drawing and not very good. I didn't know what to say.

As a kind of revenge, I made a sketch of him, and since then I have made a couple of other drawings of guys I have caught drawing people on cafés. They are not very good drawings; they were quickly and secretly made and you would never be able to recognize the people from the drawings. Still, I think they make a funny set of drawings. Because I drew the men while they were focusing on their drawings, they appear withdrawn and unaware of their surrondings while they are actually observing other people.

I must admit that I have stopped coming at this particular café. On the other hand I also like to draw people in public places (like these old drawings on flickr). What do you think? Is it okay to draw people in public places? Should you ask permission? Or is it okay, as long as you do it discretely?