5 digits would be easy. It would take a little over an hour on average. However, passcodes are not limited to digits. Use upper and lower case letters and digits, and then a 5 character passcode would on average a little over a year. Make it 6 characters, and that's 72 years.
I'm guessing based on my observations of 40-50 people entering Passcode, that north of 90% of people who bother to use a Passcode (or care to) - use a 4 digit pin.
Even those who are super security conscious tend to just use numeric PINs. It's the very, very rare individual who enters an alphanumeric passcode.