That's why it's suggested you using Jammit (for Rails), or similar asset packager. Django has django-assetpackager. Or you can write your own pretty easily. Scoped selectors makes concatenating style-sheets safe.
If you have to sacrifice organization to gain performance, I think that's a sign you're doing something wrong. If number of requests is a concern, you should probably be doing asset packaging.
It's silly not to use a bundler with Rails (both for JS and CSS). That way you don't incur the performance hit from organizing project files by controller name.