[equark]

It sounds like you're just saying that if you want to know the frequentist properties of an estimator you have to be frequentist. That's a tautology.

The harder question is whether there are any decisions you'd prefer to make using a non-Bayesian procedure. That's basically a tautology in the other direction though.

[antics]

As I said, my Bayesian theory is rusty, but there are no "frequentist properties" of an estimator. Frequentist inference is inference -- it doesn't make guarantees about the underlying thing it's approximating, it provides guarantees about its approximation.

The key here is that Bayesian and frequentist procedures provide different sorts of guarantees. Frequentists optimize for \theta, the possible set of things that could describe all of the data X, while Bayesians will assume a single describing function \theta (this might come from an "expert") and simply optimize the expectation conditioned on the data. Neither is "wrong" but in the case of the bootstrap, the result is calibrated in a way that Bayesian inference simply never will be (if it were, it would be frequentist).

EDIT: As per your second question, actually I think it's not more interesting. A classifier is a type of estimator, so all of the general frequentist guarantees actually still apply to decisionmaking.

[equark]

Frequentist statistics is about determining the repeated sampling properties of a procedure/statistic/estimator. It's about evaluation not estimation. "Optimizing \theta" or whatever you're envisioning is just one possible procedure you might be interested in evaluating. You can use the repeated sampling properties of your procedure to do frequentist inference or to evaluate other properties like unbiasedness, consistency, risk, etc. Typically the goal is to find procedures that have "good" frequentist (repeated sampling) properties. Most Bayesian-inclined statisticians would tend to argue that many frequentist properties are not important to applied data analysis or optimal decision making.

[taliesinb]

A possible example given in the lecture is figuring out if two sets of numeric data were sampled i.i.d from the same distribution (being hand-wavy about what this precisely means).

I don't see how you can sensibly approach this kind of problem in a Bayesian way. From a frequentist perspective you're sort of spoiled for choice about how to approach this problem.