There's that whole set of people that don't use Facebook ... also, creeper factor much? "hey.. I was searching for you on facebook, and... wanted to know if we could hang out" insert heavy breathing where most awkward to fit my mental image of someone stalking me on facebook
Maybe that's the key to making a good product in this space. Making a group connect web page for each get-together putting an optional amount of contact info in it after the get together. Also add an IM system.
Looks like you need to have Facebook to sign up, which I assume is how they screen people. Unfortunate, as I dropped my account a few years ago and I would have tried this, looks like fun.
You are probably right if you are adding someone with any mutual friends. If you can frame it more as "I saw you comment on x's status so I added you" that is fine.