This is a general lesson: whenever someone criticizes something you've built, be extremely polite and you'll deflate their ill-conceived anger at you. Several times I've received angry emails about a (free) web site I run, like "your site is sh*t", or "I give you a D", and I look for whatever's legitimate about their complaints. If there's nothing worth doing based on their complaints, then I just thank them for their feedback and say I'm sorry it didn't do what they wanted. None of those people has ever written me a second time.
So bravo to mbs348 for ignoring mikemarotti's use of "bullshit" and "lazy" and just getting out of his comments that you could improve your captions.
"Just" being able to freely place them, choosing alignment (left, center, right, justify?), various fonts, selectable font color and maybe even an outline color and width (you can simulate that with a bunch of text-shadows) would go a long way :)
The fridge thingy could also do with several fonts, but I actually quite like it as is. Though here a selectable background and font color would also be nice, or the option to put the caption above the image.
How do good defaults and options exclude each other?
Also, we're talking about creativity here: you cannot be very creative without a toolbox and some options. And frankly, I think you're grossly underestimating "real people" (and good user interfaces).