| Hi, this is Ben, CEO and Co-Founder of DigitalOcean, we have received the document and will discuss the matter publicly. ----- All times are UTC. Our monitoring picked up a malicious UDP traffic pattern on 2013-09-08 00:58:23. A ticket was then opened with the customer at : 2013-09-08 01:05:55 roughly 7 minutes later. The customer informed us that it was a script that was crawling in the background. We informed the customer that it may be a good idea to check through the virtual server to see if there were any signs of a compromise just in case. The droplet was unlocked at this time. A second UDP pattern was detected on 2013-09-24 12:27:09 and a ticket was opened 2013-09-24 12:27:14 to request more information from the customer. Because this was already a second occurrence we had to do a more thorough follow up. Discussing the matter with the customer, he informed us that it was a mysql db dump script that was pushing data to dropbox. He provided us a link to a github project that he wrote, we asked further questions. Specifically if you are writing a mysql dump remotely why are the packets being sent as UDP? Additionally if the final destination is dropbox that would be an SSL encrypted connection and again why would that transfer go over UDP? We reviewed the code of the dump-to-cloud project and it was using the dropbox sdk, here is where the file transfer is initiated: def upload_file(file_name)
client = DropboxClient.new(@access_token)
file = open(file_name)
puts 'Uploading file!! Please wait.'
response = client.put_file("/#{file_name}", file)
puts "uploaded:", response.inspect
end
From the dropbox SDK here is where it sets the destination for the file transfer: def build_url(url, params=nil, content_server=false) # :nodoc:
port = 443
host = content_server ? Dropbox::API_CONTENT_SERVER : Dropbox::API_SERVER
versioned_url = "/#{Dropbox::API_VERSION}#{url}"
target = URI::Generic.new("https", nil, host, port, nil, versioned_url, nil, nil, nil)
#add a locale param if we have one
#initialize a params object is we don't have one
if @locale
(params ||= {})['locale']=@locale
end
if params
target.query = params.collect {|k,v|
CGI.escape(k) + "=" + CGI.escape(v)
}.join("&")
end
target.to_s
end
The code that actually transfers the file from the dropbox sdk: def do_put(url, headers=nil, body=nil) # :nodoc:
assert_authorized
uri = URI.parse(url)
do_http_with_body(uri, Net::HTTP::Put.new(uri.request_uri, headers), body)
end
The file is transferred via HTTPS since it is going to a secure service and HTTPS would rely on TCP for the data transfer, again to ensure that all packets are delivered.Given that it was the second incident that a UDP traffic pattern was observed in less than 30 days and that the information the customer provided regarding the traffic did not match up, we made a determination that in fact it couldn't be this script that was generating the traffic. All of this information was relayed to the customer that we did not believe that the traffic in question was related to this script because it would not rely on UDP, an insecure protocol to deliver files to a secure endpoint where data integrity was of the utmost importance. Unfortunately, we could not unlock the account at this time because the information we received was not clear and we already had two incidents of outbound UDP traffic that appeared to be disruptive and abusive in nature totaling 1Gbps as if it were a denial of service attack, typically associated with UDP packets. |
I think I have to agree with a comment someone else made in the previous thread about this, with the rise of cheap VPS services we're seeing an influx of people unqualified and unprepared to run their own internet routable servers and things like this are the outcome of that. When you choose to stand up a VPS with a service like DO you take on the responsibility of keeping it secure and preventing it from engaging in malicious activity. If you fail at that task, the consequence is your servers will be shutdown for the good of the internet as a whole. If you are either unprepared or incapable of dealing with that responsibility you should be paying for a hosting service that's prepared to offer those services for you.
I say this as someone that currently has accounts with a number of VPS providers where I do take the responsibility of managing my servers seriously, as well as previously helping to administer a server that was compromised and taken offline until such a time as we had performed a full audit and verified our code on a new instance.
You should be prepared to treat a compromise of your servers the same as any other form of disaster. Treat it the same as if a flood happened and took out the facility you were hosted out of. You should have a backup plan in place so you can roll over to your backup until such time as you can fix the "broken" server, or else accept the downtime.