This site is now 100% read-only, and retired.

Question: local repository for production servers?

Posted by jeward on Wed 19 Jul 2006 at 10:26

We have currently 184 production debian servers in various states of out-of-date. We also maintain a local mirror of the i386 distribution. My goal is to develop an easy way to keep these 184 servers up to date.

One idea I'm considering is a second local respository that I update manually once a package has been tested in our production environment and deemed safe to deploy on the 184.

If the repository only contains packages we have tested, then in theory we can automatically update the 184 from this repository.

The other idea I've had is to keep the 184 pointed at a current mirror, but only update them after all packages have been tested on a production server.

Thanks in advance for any suggestions,

James

 

 


Re: Question: local repository for production servers?
Posted by sphaero (83.160.xx.xx) on Wed 19 Jul 2006 at 11:26
[ View Weblogs ]
what I'm currently implementing is setting up my own repository on a central server. I fill this repository with security updates from security.debian.org which I test for automatic deployment before adding to the repository. All servers just cron-apt upgrade from this repository every night. It's not operational just yet but hopefully it will be any time soon.

[ Parent ]

Re: Question: local repository for production servers?
Posted by simonw (84.45.xx.xx) on Wed 19 Jul 2006 at 18:45
[ View Weblogs ]
Building your own Debian mirror is about a five minute job, plus a lot of waiting for the mirroring to complete (and in one case finding a bug in rsync).

It is well documented.

In such an environment it is probably well worth it.

Some ISPs in the situation make their mirror public, but that can be a big donation of bandwidth to the Debian project. Still you can view it as other people testing your mirror ;)

I've always wondered whether in such environments it makes sense to have staged mirrored, so that you update some test servers from one mirror and when happy copy the updated state to the main mirror. Thus allowing you to fully automate the update on the bulk of servers (as you'll have tested the updated inhouse before allowing them to update with them). Be interested how others handle this, not that we have that many servers. I just get them to email me when updates have been downloaded, probably doesn't scale to 200 very well.

[ Parent ]

Re: Question: local repository for production servers?
Posted by Anonymous (217.77.xx.xx) on Mon 24 Jul 2006 at 12:30
There's apt-proxy as well, which caches the repository locally, on a need to have basis:
http://apt-proxy.sourceforge.net/

But one of the developers dies, and it hasn't been updated in quite a while. I don't know if it's still functional.

[ Parent ]

Re: Question: local repository for production servers?
Posted by DaveV (24.8.xx.xx) on Wed 19 Jul 2006 at 19:12
Install Apt-Cacher on one of your firewalls or servers.

Then update each /etc/apt/sources.list so that the other systems point at it.

Like so:

deb http://<ipaddress_of_cache>:3142/mirrors.kernel.org/debian sid main contrib non-free

Then every machine that uses the same repository will grab the cached packages.

Nice part of this is you dont waste bandwidth downloading and maintaining updates of the packages you dont use.

Its simple and efficient.

[ Parent ]

Re: Question: local repository for production servers?
Posted by Anonymous (195.252.xx.xx) on Thu 20 Jul 2006 at 02:04
Sounds like you should look into FAI.
It's based on some scripts runing based on computers "class" and different scripts names (in bash, cfengine, perl etc).
It scales good. There are examples on this on FAI wiki page.

[ Parent ]

Re: Question: local repository for production servers?
Posted by daveseff (72.229.xx.xx) on Thu 20 Jul 2006 at 03:14
Also You may want to take a look at a project I am currently working on. It's called sysmgr (System Manager)

A while ago I was in the middle of trying to figure out how to keep track of package updates on all of my servers. I have around 300 servers of various platforms (Debian, Redhat, Solaris) and periodically I'd have to shell around and figure out which machines needed updates and which didn't.

I then decided that a neat thing to have was a way to do this from one central location. So created a web-based System Package Manager, "sysmgr", where I can see all of my servers and an indication as to which ones need updates. This project is life-like. Meaning it is slowly evolving with new features frequently.

http://www.solar1.net/index.php?pid=1

Let me know if it works for you and let me know how I could improve it. It's GPL so if you want to improve it, Feel free. Send patches if you do.

-Dave

[ Parent ]

Re: Question: local repository for production servers?
Posted by Anonymous (63.97.xx.xx) on Thu 3 Aug 2006 at 21:54
I would highly recommend reading up on the infrastructures.org website and investigating systemimager.org. I use it to manage ~100 machines and have seen it scale to managing 1,500 machines. Automation! Trust in the system to take care of your hosts (just watch the system manager closely.

[ Parent ]