This site is now 100% read-only, and retired.

Scheduling backups, log rotations and the like

Posted by simonw on Sun 10 Jul 2005 at 12:23

Tags: none.

One of the unwritten rules of good system administration is never to schedule unattended jobs for times when you won't be there to pick up the pieces (or more crucially not for a time when everyone is asleep).

Conventional system administration wisdom says schedule these tasks for quiet times, and indeed for many system admins that meant outside of 9 to 5 Monday to Friday. But conventional system administration dealt with a small number of tightly controlled servers, providing services for the working week, and arose in a time when computing resources were scarce.

For many these assumptions are no longer valid. Servers run 24x7, and ways and means exist to avoid the substantial down time backups use to mean. Hardware is cheap, and even when busy the OS can schedule its work sensibly so as not to impact response dreadfully when running a backup.

Thus the thing to do is schedule the tasks when it suits the humans managing them, not when it suits the machines. For most of us this doesn't mean starting backups at midnight, or running cron jobs before 07:00 in the morning.

This rule I understand well, so why did my pager go off at 06:35 this morning, when the Apache log file rotation messed up? It is not enough to be a good system administrator, one must also dot the i's cross the t's, and "$EDITOR /etc/crontab".

 

 


Re: Scheduling backups, log rotations and the like
Posted by Steve (82.41.xx.xx) on Mon 11 Jul 2005 at 11:51
[ View Weblogs ]

On the one hand I agree because it's nice to be around to fix things as they break, rather than waiting until the next day - when a job has failed at around midnight.

On the other hand some jobs cannot realistically be scheduled to occur during the working day. I've seen some backup systems bring the network to it's knees just by virtue of the amount of network traffic being moved.

Something like wouldn't be appreciated by the regular network users, no matter how useful it might be to me.

Steve
-- Steve.org.uk

[ Parent ]

Re: Scheduling backups, log rotations and the like
Posted by Anonymous (66.54.xx.xx) on Mon 11 Jul 2005 at 22:02
One old-school trick I always use is to schedule cronjobs for prime number minutes. less likely to collide with other periodic jobs. and easier to pick out of perf logs.

[ Parent ]

Re: Scheduling backups, log rotations and the like
Posted by Anonymous (64.5.xx.xx) on Thu 14 Jul 2005 at 18:58
Tip: change the cron.daily, weekly and monthly times (and days, if appropriate) if you're running a stack of servers in a SMB environment off one or two UPSen. If you have (to pick a random example) 15 servers, all with at least 2 drives, running at least RAID1, all running mandb, updatedb, etc at the same time, you can draw enough current to kill the UPS. Or so I hear, anyway...

[ Parent ]