Ask HN: My Linux server was hacked

[yuashizuki]

I dont understand how my linux server got hacked? the ssh key was also compromised. I have switched the server off and plan to reinstall a new linux distribution. I also changed my profile password of linode. Do I need to do anything else to protect my self again? Thanks for reading.

[jfaucett]

Linode has a great guide for locking down your server, if you don't do anything else at a bare minimum do this (https://www.linode.com/docs/security/securing-your-server).

I can remember many many moons ago I got hacked too so I know the feeling. If you don't have mission critical stuff, this can be a good thing though, makes you learn how to lock down your servers.

Besides locking down your SSH config, Fail2Ban is nice, and be sure to set your iptables so your only accepting packets you want. Also here's a list of other things I usually do:

1. Depending on your distro there's usually a way to install automatic security updates, in debian its called unattended upgrades (https://wiki.debian.org/UnattendedUpgrades)

2. Install logwatch to keep yourself periodically updated with whats going on.

3. Control your packages. If you don't need services make sure your box is clean of them. Basically, for every service you have on the machine thats one more possible attack point, so just use the ones you know you need.

4. In your firewall ignore icmp_echo and icmp_echo_ignore_broadcasts if you can. There's also other stuff you can do in sysctl and with SELinux but I've never really had the need.

[mobiplayer]

Why #4?

I know sysadmins that love to disable ICMP, hence why I consider it a terrible way to test or benchmark anything, but why do you think it should be disabled?

When I scan a network or a host my muscle memory types -P0 just after nmap, so I thought ignoring pings was more a hassle than a security advantage these days... But I might be wrong :)

[Corrspt]

Hey, that's a bummer. I've been in the same situation about a year ago (I thought I had done a reasonable job of securing my instalation but they hacked my web application through a vulnerability in jboss)

I blogged about it and posted on reddit. Lot's of people gave me useful feedback (checkout this link http://www.reddit.com/r/programming/comments/1vo7zv/kids_thi...)

I'd recommend (as others have said here) disabling password login via SSH (only keys), disable root login, installing fail2ban, update the system regularly, setup firewall to close ports that don't need to be open.

[penguinlinux]

did your server had any type of website running ? such as open source projects installed ? any services or ports running and available to the outside world. Did you have a website running with code you wrote?

Out of the box a fresh ubuntu server is pretty secure so you had to install something that exposed some type exploitable code and that's how they got access to your machine.

[yuashizuki]

I was running nodejs on port 80 a older version but still quite recent. and the usual sshd deamon on port 22. Even my password was pretty strong. How did they do this ? I am really surprised.

[mindcrash]

If you have a somewhat older version of nodejs or a somewhat older version of sshd which was compiled against a somewhat older version of openssl then your box was quite possibly (actually quite definitely) pwned via heartbleed or poodle. No need to know any passwords, just a matter of pointing a tool checking and abusing heartbleed or poodle at your box and a few minutes later: access to a fresh rootshell and pwned box.

Anyway, before reinstalling you should definitely quarantine your box and figure out how they got in before reinstalling. Because if and when you don't know, and the specific vulnerability is inside the current version of your Linux distro the chance is almost 100% they will discover a fresh target once they scan for vulnerable servers and they will hack your box again.

[pipu]

How did you secure your server?

How do you know it got hacked?

[yuashizuki]

every min they were brute forcing my sshd which I found from /var/log/auth.log also I saw a few unusual entries in my db.

[hakanderyal]

Disable password access, use keys instead. And install fail2ban to prevent brute force attempts.

Here is a good read about basic security: http://plusbryan.com/my-first-5-minutes-on-a-server-or-essen...