Portraits of Strangers

Does anybody know how hard it is to walk up to a total stranger and ask if you can take their picture. I did not realize that until I started doing that myself. What's so scary about strangers, or is it just me:-)