You wouldn't want to remove the styling completely, but definitely get rid of the browser default and use a custom focus style that looks better.