[kzvezdarov]

Not sure which book the parent was referring to, but that's the premise of Charles Stross' Laundry Files series: too many humans and too many computers doing too much complex mathematics attract tentacled horrors from beyond space-time. Edit: I also like how in it, mathematics is the language of sorcery, and computing devices of any form - the implements.