Posted by Steve on Thu 21 Dec 2006 at 09:30
There are times when people make mistakes, and manage to do crazy things to working systems. It wasn't so long ago that a hasty deletion caused me all kinds of problems. Recently I read of an unfortunate sysadmin who managed to recursively change permissions on their root filesystem - and here is my attempted solution.
The problematic command was:
(none):~# chmod -R 777 /
Here the entire filesystem, and all mounted partitions, were modified to be world-writable. One immediate consequence of this is that ssh logins fail:
skx@lappy:~$ ssh 192.168.1.199 -l root ssh_exchange_identification: Connection closed by remote host
Looking in /var/log/auth.log upon the host we can see the cause of this failure:
error: Could not load host key: /etc/ssh/ssh_host_rsa_key error: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ error: @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ error: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ error: Permissions 0777 for '/etc/ssh/ssh_host_dsa_key' are too open. error: It is recommended that your private key files are NOT accessible by others. error: This private key will be ignored. error: bad permissions: ignore key: /etc/ssh/ssh_host_dsa_key error: Could not load host key: /etc/ssh/ssh_host_dsa_key fatal: /var/run/sshd must be owned by root and not group or world-writable.
These errors are fixable fairly easily since we still have an open root session:
(none):~# chmod 600 /var/run/sshd (none):~# chmod 600 /etc/ssh/ssh_host_*
Since we have root access we can now fix things up, and the availability of remote ssh access is a nice bonus. However these remote logins cannot become root themselves just yet - because su and sudo have lost their setuid flags:
skx@(none):/$ su Password: setgid: Operation not permitted
We can fix this fairly easily, again using our existing root login:
(none):~# chmod 4755 /bin/su /usr/bin/sudo
Now we can login remotely and become root. So the system is 90% fixed!
But what if we didn't have root access? What if we could only login via a normal user account, via the console perhaps?
Lets try that now.
(none):~# chmod -R 777 / (none):~# exit
Remote logins via ssh will have become broken again - how can we fix things now? Suddenly we can't fixup permissions as we're not root, and we can't become root since the binaries are no longer setuid.
It seems that our first job is to gain root access. Whilst there are many things we could try (such as emptying the password fields in /etc/passwd - possible now all files are world-writable) first solution to occur to me was to abuse a daemon which is already running and which has permissions to do root-type jobs. In a word: cron.
Since the system has various php4 packages installed we can modify the cronjob which has been installed:
echo "*/1 * * * * root cp /bin/sh /tmp/sh && chmod 4755 /tmp/sh" > /etc/cron.d/php4
Waiting a minute finds us a setuid shell in /tmp and we can use that to become root and fix things as we did previously:
skx@uml:~$ /tmp/sh sh-3.1# id uid=1000(skx) gid=1000(skx) euid=0(root) groups=1000(skx)
Having writable files will allow a lot of different attacks against a system, cron is just the most obvious. The important thing is to not panic, and do anything hasty such as rebooting or exiting your root shell!
I guess it did just tail off a little, but the hard parts were done. Once you can login as root remotely then you're easily able to do the site-specific fixups from the comfort of your desk.
If it were me I'd just restore from backups which would incidentally fix the permissions!
Still if you have another working system I guess the obvious thing to do is to write a simple script to output a file containing the path + permissions of every file on the system. Then another to read in that file and apply the fixups.
I remember doing that once in the past. Something like this simple save.pl script would do the job:
#!/usr/bin/perl -w
use strict;
use File::Find;
find(\&wanted, "/");
sub wanted
{
my $file = $File::Find::name;
my $mode = (stat($file))[2];
printf "%04o, %s\n", $mode & 077777, $file;
}
This will give output such as this:
40755, / 0600, /.bash_history 0600, /.viminfo 0600, /.lesshst 0600, /initrd.img.old 0644, /vmlinuz.old 40755, /lost+found 40755, /home
The script to restore those permissions is left as an excercise ;)
[ Parent ]
[ Parent ]
[ Parent ]
Not directly; although you could run "apt-get install --reinstall [package1] [package2] .." to do it.
Failing that you could install the acl package and use getfacl + setfacl to create/restore from a backup as suggested by mvanbaak already.
In the case of missing files then the "--reinstall" trick might work, but the obvious answer is "restore from backups". If you don't currently have backups you'll have learnt how important they are for the future ;)
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
[ Parent ]
Kind regards
Wolfgang Karall
--
Debian GNU/Linux on an IBM Thinkpad T43p
[ Parent ]