There was an article recently about John Ogbu, either on HN or reddit. Can't find the same source, but from a bit of googling: http://www.ogbu.com/. Main point: black children tend to be behind at school because the home and peer culture is at least not supportive, and at worst against education.
Back in the 80s, I very clearly remember being the only person in my grade school library during recess (granted, it was a very small school), and one of very few people who carried a book around with them in high school.