This site is now 100% read-only, and retired.

Setting up multiple Subversion repositories

Posted by hruske on Tue 9 Aug 2005 at 11:20

With subversion (and svn-buildpackage) becoming popular hosting multiple repositories with different access controls is something that is commonly required. Here we'll show how you can set this up in a very flexible manner using PostgreSQL

This article was inspired after reading the guide on LinuxJournal written by Cristiano Paris. However this introduction describes a more flexible setup.

Since this guide is for Debian Sarge, it may use different versions of the software.


We will be using PostgreSQL with Apache 2 WebDAV and mod-auth-pgsql. Packages you will need to install are:

  • postgresql
  • postgresql-contrib
  • subversion
  • apache2
  • libapache2-mod-auth-pgsql
  • libapache2-svn
PostgreSQL and database setup

Installing PostgreSQL is pretty easy. Be sure to select the proper encoding settings, ideally this would be the same as your system locale.

For storing password in PostgreSQL table we'd like some encryption, so we need to install pgcrypto too. Do this by switching to user postgres and running pgcrypto.sql:

root @ box # su postgres
postgres @ box $ psql template1 < /usr/share/postgresql/contrib/pgcrypto.sql  
SET
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION

This creates some new functions, including crpyt, which we will use to store encrypted passwords in database. First we need to create the database and new user:

postgres @ box $ createdb subversion
CREATE DATABASE
postgres @ box $ createuser subversion
Shall the new user be allowed to create databases? (y/n) n
Shall the new user be allowed to create more new users? (y/n) n
CREATE USER

Now we can already map the database structure, which is mostly defined by mod_auth_pgsql. For the SQL queries to work, you need to connect to the database:

postgres @ box $ psql subversion
subversion=#

Now we need a table for users ...

CREATE TABLE users (username char(32),pass text,email varchar(80));
GRANT SELECT ON users to subversion;

... and a table for groups ...

CREATE TABLE groups (username char(32),memberof char(64));
GRANT SELECT ON groups to subversion;

... and a table for logging authorizations. One can never know.

CREATE TABLE log (uname char(20), time timestamp(8), uri varchar(512),ip inet);
GRANT INSERT ON log TO subversion;

We also need to change the subversion's password.

ALTER USER subversion WITH ENCRYPTED PASSWORD 'secretpass';
Apache 2 and WebDAV

After getting Apache2 running make sure you enable mod_dav, mod_dav_svn and mod_auth_pgsql. On using the Debian tools for Apache2:

root @ box # a2enmod dav
root @ box # a2enmod dav_svn
root @ box # a2enmod auth_pgsql

When this is set up, we need to make the layout for repositories. Default Debian location is /var/lib/svn, and I'm going to use this default. In /var/lib/svn create this layout:

/var/lib/svn/
/var/lib/svn/conf
/var/lib/svn/conf/policies
/var/lib/svn/repository

We need some config files for Apache, so create /var/lib/svn/conf/mod_dav_svn.conf with following content:

Include /var/lib/svn/conf/default_policy.conf
Include /var/lib/svn/conf/policies/*
DavLockDB /var/lib/svn/repository/DavLock

Link location /etc/apache2/mods-available/dav_svn.conf to this file. Also create new file /var/lib/svn/conf/default_policy.conf:

<Location /svn/>
        Dav svn 
        SVNParentPath /var/lib/svn/repository
        <LimitExcept GET PROPFIND OPTIONS REPORT>
            Order deny,allow
            Deny from all
            # for access from 'safe' ip, uncomment following line
            # allow from 192.168.0.0/24
        </LimitExcept>
</Location>


The policies for each project are then stored in folder /var/lib/svn/conf/policies/ and are included by Apache. This means Apache needs to be restarted when new project is added. The repository needs to be Apache writable, therefore we need to adjust permissions.

find /var/lib/svn/repository -type f -exec chmod 660 {} \;
find /var/lib/svn/repository -type d -exec chmod 2770 {} \;
chown -R root.www-data /var/lib/svn/repository

Apache needs one more setting set up properly. Add this stanza to /var/lib/svn/conf/mod_dav_svn.conf:

<Location /svn>
        Order deny,allow
        Allow from all
</Location>

Viewing files with browsers should now work.

Setting up the authentication with mod_auth_pgsql

First we need to allow connections for user subversion to PostgreSQL database subversion. Add this line to /etc/postgresql/pg_hba.conf:

host    subversion  subversion  127.0.0.1         255.255.255.255   md5

Be sure, to put it before line

host    all         all         127.0.0.1         255.255.255.255   ident sameuser

You may need to add your real IP too, not only 127.0.0.1, so either add it now or check logs later.

We will now create a test project to make sure this works. Create a new file /var/lib/svn/conf/default_auth.conf with common parameters for mod_auth_pgsql:

AuthType Basic
Auth_PG_host localhost
Auth_PG_port 5432
Auth_PG_database subversion
Auth_PG_user subversion
Auth_PG_pwd secretpass
Auth_PG_pwd_table users
Auth_PG_uid_field username
Auth_PG_pwd_field pass
Auth_PG_grp_table groups
Auth_PG_grp_group_field memberof
Auth_PG_grp_user_field username
Auth_PG_cache_passwords on
Auth_PG_connection_reuse on
Auth_PG_log_table log
Auth_PG_log_uname_field uname
Auth_PG_log_date_field time
Auth_PG_log_uri_field uri
Auth_PG_log_addrs_field ip

Create a new file /var/lib/svn/conf/policies/blah intended for access control to a project, having following content:

<Location /svn/blah/>
        <LimitExcept GET PROPFIND OPTIONS REPORT>
                AuthName "Subversion repository for project blah"
                require group blah
        </LimitExcept>
</Location>

Next we need to create subversion repository in /var/lib/svn/repository/blah:

root @ box # mkdir /var/lib/svn/repository/blah
root @ box # svnadmin create /var/lib/svn/repository/blah

You can not authenticate user, if you don't have one, therefore we will create a 'test' user to see if we've got authentication set up properly:

root @ box # su postgres
postgres @ box $ psql subversion
subversion=# INSERT INTO users VALUES ('test',crypt('pass','seed'),'test@localhost');
subversion=# INSERT INTO groups VALUES ('test','blah');

We can now try and hopefully succeed in importing Revision 1 of our project.


If you are having trouble setting up, be sure to check Apache and PostgreSQL logs to see what is going on.

 

 


Re: Setting up multiple Subversion repositories
Posted by hruske (193.2.xx.xx) on Wed 10 Aug 2005 at 16:44
uhm ... Apache directives with lessthan and greaterthan have been stripped?

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Steve (82.41.xx.xx) on Wed 10 Aug 2005 at 16:47
[ View Weblogs ]

Ugh.

I just checked my mailbox and they were never present - I don't know how I didn't notice.

If you give me updated bits I can fix it.

There are times when I really hate having to handle stuff like this :S

Steve
-- Steve.org.uk

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by hruske (193.2.xx.xx) on Wed 10 Aug 2005 at 23:31
I've put a copy at http://www.kiberpipa.org/~hruske/subversion.html

Oh, and it would be smart to change the ip field from char to inet. ;) I forgot that earlier.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (193.170.xx.xx) on Fri 12 Aug 2005 at 14:40
I also want to point you to mod_authz_svn, which allows you to use directory based access control within repositories:

http://svnbook.red-bean.com/en/1.1/ch06s04.html#svn-ch-6-sect-4.4 .2

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (148.136.xx.xx) on Fri 16 Sep 2005 at 07:36
Am I the only one who cannot make this work?

I have the default_auth.conf, but I havn't told apache anywhere to use it, and the guide doesn't tell me to. Is there a step missing? Or am I just not finding it...

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (193.2.xx.xx) on Fri 16 Sep 2005 at 08:25
How on earth did I miss that?

Yes, you have to add it to /var/lib/svn/conf/policies/blah, into the LimitExcept directive.

thanks for pointing this out.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (148.136.xx.xx) on Fri 16 Sep 2005 at 11:18
Thanks for the reply, and all that...

But..

Exactly how do I type? I tried a few different things, but I have so far failed to figure out the magic word to use before the filename (yes, I've looked in the mod_auth_pgsql documentation, as well as som general apache auth documentation)

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (4.154.xx.xx) on Sat 15 Oct 2005 at 03:36
I too would like to know the answer to this question. Two months since publishing and there's no revision? These directions are really shady. They should have been checked before being posted. I appreciate the effort, but a half-written article almost does more harm than good.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (81.152.xx.xx) on Sun 9 Sep 2007 at 11:35
Just add "Include /var/lib/svn/conf/default_auth.conf" as the first line in the <LimitExcept> bit.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by hackeron (212.36.xx.xx) on Mon 24 Oct 2005 at 14:25
If I were to write such a howto, I would go with pam (i.e. pam-mysql or pam-ldap) or even htauth for simplicity instead of auth-pgsql. Many WebDAV users dont want a separate password database.

I.E. register on a forum, and get read-only access to svn/webdav.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (193.2.xx.xx) on Tue 25 Oct 2005 at 07:14
You could still use mod_auth_pam, this is basically just an example.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by kaybenleroll (66.250.xx.xx) on Tue 30 Oct 2007 at 21:22
I keep getting a Server 500 error. There appears to be a problem with mod_auth_pgsql it seems.

I've looked at the error log, and the following error line get logged:

[Mon Oct 29 13:42:53 2007] [error] [client xxx.xxx.xxx.xxx] (9)Bad file descriptor: Could not open password file: (null)

The pgsql logs have not registered any kind of connection attempt.

Anyone got any idea what I am doing wrong?

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (213.165.xx.xx) on Sun 11 Nov 2007 at 21:49
I'm having the same problem with mod_auth_pgsql.

Apache/2.2.6 (Debian) mod_auth_pgsql/2.0.3 DAV/2 SVN/1.4.4 mod_python/3.3.1 Python/2.4.4 mod_ssl/2.2.6 OpenSSL/0.9.8g

I've isolated the problem to the mod_auth_pgsql module, I think, unless it is another auth module which is looking for an htaccess file or something similar.

Rory

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (194.204.xx.xx) on Fri 7 Dec 2007 at 14:33
This usually stands when, on that or another cause - your auth_pgsql cannot connect to the Postgres server.

No entry in pg_hba.conf ? Server down ? A typing mistake ?

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by kaybenleroll (87.192.xx.xx) on Wed 13 Feb 2008 at 13:44
Yes, for some reason no connection is been made to the database.

I've made sure the module is enabled, and my pg_hba.conf is fine. The server is up and I have made no typos. I get the same error all the time (the bad password one, as before).

I even ran tcpdump on the network interface at the time I tried to view the page, and I noticed that no connection attempt appears to be made.

Anyone have any idea what I might be doing wrong, or suggest things I could look at to try to figure out what is happening?

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (68.42.xx.xx) on Sun 6 Apr 2008 at 20:50
I ran into the same problem and found that setting
AuthBasicAuthoritative Off

Corrects the issue with recent apache versions.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (193.34.xx.xx) on Mon 8 Sep 2008 at 21:15
To prevent Apache from filling error.log with messages about being unable to open password file add this directive too:
AuthUserFile /dev/null

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (209.206.xx.xx) on Wed 7 May 2008 at 17:04
I wanted to say that these instructions are just fine. They are offered as a baseline by someone who knows what they are doing and how to get there. There's a bit of assumption that you, the user, are skilled enough to understand the steps he's omitted for brevity. I was able to get this system up and running - from building a scratch sys on XEN to logging with my firefox client to see Revision 0, in around 1.5 hours. Not super fast, but suitable. I thank you for the effort you put into this, and consider the instructions here nothing less than perfect.

[ Parent ]

Re: Setting up multiple Subversion repositories
Posted by Anonymous (84.10.xx.xx) on Tue 10 Mar 2009 at 19:59
Is this right to access svn repo via http in browser and apache doesn't require password? Cause when importing via svn it has asked for password. And i think it also should ask for password when accessing repo via http in browser...

[ Parent ]