[harty65]

I work for one of the ISPs in question as an Analyst and work is in progress to stop the storing of certain data after a certain time period.

The main reason for holding call data is for billing purposes, no e-mail information has ever been stored (other than in customer's own inboxes and sent items in their e-mail accounts). All browsing data is anonymised and only used for optimisation purposes.

Hundreds of thousands of pounds are spent annually on data storage, it’s welcoming that there is now a reason to reduce the amount of useless data we hold.

[omh]

Can you clarify what you mean by "browsing data"? I understand why some data could be useful for debugging, but it's the DPI-style data that I'm most concerned about. I'd assumed that it would be too expensive for ISPs to bother collecting it if they weren't legally obliged to.

[harty65]

So information is collected regarding what IP addresses are accessed from our network, it isn't traceable back to customer accounts (in its usual form anyway, I’m sure if some data forensics people got hold of it they could maybe do something, but that’s outside my scope of knowledge) this information is used to improve customer experience and reduce network traffic.

For example, in 2012-2013 almost 40% of traffic on our Network was Youtube related, not just streaming, Youtube specifically! Obviously millions of these requests are going to be for the exact same video, say for example Gangnam Style. So instead of us pulling the data from Youtube every time a user requests that particular video we store that video on our Network, so when a user requests it (even for the first time), it actually comes from us. All my company needs to do is check for any changes to the specific page. The result is a better customer experience (faster streaming) but also far less network traffic (this literally saves millions).

Without the storing of IP addresses accessed we wouldn't have the information needed to determine what sites/data we should be cashing and what we don’t need to bother with and this sort of thing would be considerably harder to pull off.