some fun facts about this overblown Google Profile Privacy problem:
Your Google Profile only lists the Names of people you follow, that have Public Profiles. If the person does not have a Public Profile, then they won't be listed. The Followers list says "And X other people who do not have public profiles".
You have to specifically create a Google Profile. And, as far as I know, unless you created it through Buzz(which requires a Public Profile to use), the Profile is Private, by default. You have to 1) Opt to make your profile public, and 2) add a sufficient amount of information (Normally, a Picture and a Link and a short Bio is good enough) before your profile is even seeable.
so if your GMail Contact(Mommy, Daddy, Grandma, Mistress, whomever) that you don't want listed does not have a Public Google Profile......they won't be listed. By default.
Now of course, if everyone on your GMail Contacts already has a Public Google Profile, then you are screwed. I don't have an answer for that one.