The problem with benchmarks is that they don't really correspond to reality. Your app is not a simple test app. Your site will not be running on my MacBook Air. Your users won't have the usage patterns my script does.
IMO you shouldn't do the benchmark. Let this project be a starting point for a Rails api project and/or a how to for people who want to customize Rails for their needs.
"We are building an API. Rails has extra components which aren't used but might cause overhead. Here is how to remove it."
Of course. It's not about showing some absolute numbers that users can expect. It's just about showing the potential overhead of a full Rails app vs Rails::API. I'd think this would be a reason why people would chose Rails::API over just Rails.
The README itself says Rails::API is faster, so not sure why a simple benchmark to back that up isn't a good idea.
edit: It really boils down to another way to visualize the overhead vs just listing what's been removed.
"We are building an API. Rails has extra components which aren't used but might cause overhead. Here is how to remove it."
I think that's a valid enough use-case.