Do you use let's encrypt?





6369 votes ~ 21 comments

 

Playing with SVN

Posted by DineshPremalal on Mon 20 Mar 2006 at 10:15

Today I started to set up a SVN repository for our final year project. I tried to setup a SVN server using Apache2 so that the SVN repository is available to the client through the WebDAV/DeltaV protocol. Read on for a trial-and-error introduction.

The Version Control with Subversion book (by Ben Collins-Sussman, Brian W. Fitzpatrick & C. Michael Pilato) was very useful to me when I struggled with SVN. The e-version of the book also available for free.

After installing subversion I found it inside the directory /usr/share/doc/subversion/book.

First I installed apache2 using apt-get

#apt-get install apache2

Then subversion:

#apt-get install subversion

In order to setup subversion with apache2, we need to use the mod_dav_svn module. Here I tried to use the pre-compiled version from the Debian repositories because otherwise the process get more complicated.

/**
 * If you really need to compile apache2 and subversion your self the following
 * link might help you. But I *didn't get dirty with it. :)
 * http://www.linuxjournal.com/article/7655
 */

Now I need to install mod_dav_svn module. I tried serveral searches using aptitude. Then I found out that libapache2-svn package provides mod_dav_svn plugin, so I installed it.

#apt-get install libapache2-svn

The most interesting part is, when we install pre-compiled package using apt-get or aptitude they edit configaration files accordingly.

When we install apache2 it installs configuration files in the /etc/apache2 directory. Modules are placed inside /usr/lib/apache2/modules.

In normal installation we found only httpd.conf for managing httpd server's configuration but in Debian we manipulate two configuration files namely "apache2.conf" and "httpd.conf".

As the book stated I edited httpd.conf as following:

LoadModule dav_module      /usr/lib/apache2/modules/mod_dav.so
LoadModule dav_svn_module /usr/lib/apache2/modules/mod_dav_svn.so

After restarted httpd server:

debian:/etc/apache2# /etc/init.d/apache2 restart
Forcing reload of web server: Apache2[Sun Mar 05 20:21:41 2006] [warn] module dav_module is already loaded, skipping
[Sun Mar 05 20:21:41 2006] [warn] module dav_svn_module is already loaded, skipping
[Sun Mar 05 20:21:42 2006] [warn] module dav_module is already loaded, skipping
[Sun Mar 05 20:21:42 2006] [warn] module dav_svn_module is already loaded, skipping

It seems that module already loaded then, so I commented out that newly added two lines (little devation from the book).

debian:/etc/apache2# /etc/init.d/apache2 restart
Forcing reload of web server: Apache2.

Since my intention was to have more repositories in the svn, I added following lines to the apache2.conf:

<Location /svn>
  DAV svn
  # any "/svn/foo" URL will map to a repository /home/dinesh/svn/foo
  SVNParentPath /home/dinesh/svn (absolute path to repository)
</Location>

Then I created repository using 'svnadmin':

$svnadmin create file:///home/dinesh/svn/repos

Then created directory in that repository:

$svn mkdir file:///home/dinesh/svn/repos/guththila

First I tried to acces repository using the http:// protocol but it didn't work :

dinesh@debian:~/tmp$ svn checkout http://127.0.0.1/svn/repos/guththila project
svn: PROPFIND request failed on '/svn/repos/guththila'
svn:
Could not open the requested SVN filesystem

Then tried to use the repository using web http://127.0.0.1/svn/repos/guththila:

<d:error>
<c:error>
<m:human-readable errcode="160029">
 Could not open the requested SVN filesystem
</m:human-readable>
</c:error>
</d:error>

The Apache error log (in /var/log/apache/error.log) said:

[error] [client 127.0.0.1] (20014)Error string not specified yet:
  Berkeley DB error while opening 'nodes' table for filesystem
   /home/dinesh/svn/repos/db:
   Permission denied

It seems that there was a problem in permissions. I set permissions of the svn repository to the 777

#chmod -R 777 svn

Then I was able to acces the repository through the web browser using the URL http://127.0.0.1/svn/repos/.

At one point I came across a database error. The Apache error log said:

[error] [client 127.0.0.1] (20014)Error string not specified yet: 
  Berkeley DB error while opening environment for filesystem 
  /home/dinesh/svn/repos/db:
  DB_RUNRECOVERY: Fatal error, run database recovery

After seaching on the web found a similar problem it said to goto the repository db folder and run db_recovery.

$ cd ~/svn/repos/db
dinesh@debian:~/svn/repos/db$ db4.2_recover

This recovered the database.

Now everything works fine but when I tried to commit into the repository:

$svn commit -m "test"
svn: Commit failed (details follow):
svn: MKACTIVITY of '/svn/repos/!svn/act/3033bdb6-400e-0410-9aed-b2743e91598e': 500 Internal Server Error (http://localhost)

This happened because of I didn't have authentication method for my svn repository.

# htpasswd -cm /etc/svn-passwd dinesh
New password:
Re-type new password:
Adding password for user dinesh

# htpasswd /etc/svn-passwd -m thilina
New password:
Re-type new password:
Adding password for user thilina

I added the following lines to the apache2.conf:

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /etc/svn-passwd
</Location>

With this method the password is transmitted as plain text over the network.

Now I'm able to acces and commit SVN repository over the network!

I still need to configure the SVN repository to use SSL. I'll post the steps that I followed to configure SVN with SSL also.

If you interested please look at my weblog for original article

 

 


Re: Playing with SVN
Posted by jwm (203.79.xx.xx) on Mon 20 Mar 2006 at 11:01

The subversion book is pretty sloppy at telling you how to manage the repository ownership, which tends to bite new admins, especially when using file or ssh+svn access. It doesn't help that the debian package doesn't step up to the plate and do the right thing.

Basically, you have to make sure that the repository files and directories are writable by all the users with commit access to it. In your case—access via Apache only—this means you should set the repository to be owned by the web server user (ie www-data), (as root):

chown -R www-data:www-data /home/dinesh/svn/repos
find /home/dinesh/svn/repos -type d chmod 2755 "{}" /;
find /home/dinesh/svn/repos -not -type d chmod 0644 "{}" /;

It's probably a smart move to shift this repository to something like /var/lib/svn/repos, too.

The svnserve case is much the same, as the repository only has one system user, however if you want to use file: and svn+ssh: access methods instead or as well, you need to do a bit more work:

  1. Set up a group for all your subversion users with write access, say svn-users. Put all the users in it, including the svnserve and apache user if you want access via those methods.
  2. Change the group ownership on the whole repository to svn-users
  3. Make sure the group write bit is set on all the repository files and directories
  4. Make sure the set group id bit is set for yourrepos/db and it's subdirectories, so that new files created will have the correct group ownership set (svnadmin does this for you automatically)
  5. When using the file: or svn+ssh: access methods, make sure the umask has been set to 0002 or similar. Debian's default umask is 0022, which filters out the group write permission. A wrapper script is probably the best way to handle this.

This applies to Subverson 1.1.x in sarge. I think these issues might have been addressed in 1.2 and greater that's in etch. I haven't really tinkered with it recently.

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by alfadir (141.58.xx.xx) on Tue 21 Mar 2006 at 07:50
[ View Weblogs ]

Good comments jwm,

I used your find/chmod lines on some other data, and I am just not sure if they work? I run bash and they might work in other shells.
Here is what works for me

 find path/to/change -type d -exec chmod 2755 \{} \;
 find path/to/change -not -type d -exec chmod 0644 \{} \;

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by jwm (222.154.xx.xx) on Tue 21 Mar 2006 at 08:32

Ah—I made a brain-o on the last slash.

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by bezell (70.88.xx.xx) on Mon 27 Mar 2006 at 21:40
Some of these were mentioned above:

-- use -fsfs

Sure the bdb package is faster, but as you show in your example it can easily be corrupted. Using -fsfs makes a tar backup simple and reliable.

-- add "SVNPathAuthz off" to your Apache2 location directive

If you don't need per-directory security, turn off the checks to improve the speed.

-- using SSL

Someone mentioned having trouble with the 'SSLRequire' directive. The easiest way to force SSL is to add a RedirectMatch to your normal, HTTP (80) virtual host:

RedirectMatch (.*) https://svn.mydomain.org$1

SSL and Apache2 work well and allow one to control access using authentication methods that might already be in use. They are slower though and can be CPU intensive. If you don't need the SSL, you might consider avoiding it in order to improve speed. mod_auth_digest offers a more secure method, but I've never tried it with Subversion clients.

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by Anonymous (84.92.xx.xx) on Mon 20 Mar 2006 at 11:21
Erm, you shouldn't make directories within SVN the way you have done. You should make them by checking out and adding the directory, or by importing a directory. The directory you made, creates a physical directory in the file system rather than in the db files, which won't work how you want it to.

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by multani (82.233.xx.xx) on Mon 20 Mar 2006 at 11:23
Hello,

you can use fsfs backend instead of the legacy Berkley-DB backend for data storing.

One of the motivation to move from BDB to fsfs is the speed at which BDB backend tends to lock, keeping admin on the edge to restore it.
There's less problem when migrating from one version of BDB to another (as Subversion did in 1.3 if I am correct).

See http://svnbook.red-bean.com/nightly/en/svn.reposadmin.html#svn.re posadmin.basics.backends for more informations

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by fugit (66.108.xx.xx) on Tue 21 Mar 2006 at 03:56
[ View Weblogs ]
Just wanted to second multani advice on using fsfs. I have had no problems with repos created with fsfs. Where as I have had to recover BDB repos which I quickly converted to fsfs.

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by alfadir (141.58.xx.xx) on Mon 20 Mar 2006 at 11:24
[ View Weblogs ]

Thanks for sharing your experiences. I am also about to go through this, although not for the first time. I intend to do a more complex setup and try to write about it.

I just want to point out a few things.

Instead of editing :
As the book stated I edited httpd.conf as following:

LoadModule dav_module      /usr/lib/apache2/modules/mod_dav.so
LoadModule dav_svn_module /usr/lib/apache2/modules/mod_dav_svn.so

You can under Debian run

a2enmod dav
a2enmod dav_svn

That will set the appropriate links between /etc/apache2/mods-available/ and /etc/apache2/mods-enabled/

I am still a bit undecided how to properly add the subversion location. I have used the conf file in /etc/apache2/mods-availible, but maybe sites-enabled can be used too. (I am thinking for multi-site apache)
Have a look at the file

/etc/apache2/mods-available/dav_svn.conf
for an example. I use the below structure but the mentioned file have more information. (Please observe that the location apache tags were stripped by the blogging software)

  DAV svn
  SVNPath /srv/svn/project

  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile /etc/apache2/dav-svn.passwd

  AuthzSVNAccessFile /etc/apache2/dav_svn.authz

  Require valid-user

For access configuration have a look in

/usr/share/doc/libapache2-svn/INSTALL.authz.gz

Lastly the permission should be handled differently. It is not good to set the repository writable by anyone. I can not find the reference right now but on earlier recomendation I have configured my running repository as

chown -R www-data.www-data /srv/svn/project

I am not sure how well that works with other access methods, but I will investigate that in the comming months.

It would be good if you can investigate these things and update the article accordingly so that the article shows as much as possible the Debian way. That is at least what I like about http://www.debian-administration.org/

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by fugit (66.108.xx.xx) on Tue 21 Mar 2006 at 04:02
[ View Weblogs ]
I built everyting from source and setup svn 1.1.3 with apache2 using webdav for AD intergration. I'm about to upgrade to 1.3 for autoversioning.

As for the perms I set everything to be writable by svn.


[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by alfadir (141.58.xx.xx) on Tue 21 Mar 2006 at 07:54
[ View Weblogs ]

Any special reason for building it from source ?

I am just curious if I am missing out on some feature that is missing in the precompiled debian package.

Or has this to do with AD (Active Directory?). Is that the reason you have to build from source ?

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by fugit (199.2.xx.xx) on Wed 22 Mar 2006 at 21:21
[ View Weblogs ]
I did the install back in 20050215. I don't recall exactly why I used source.

I tend to use packages when possible with the exception of apache. I tend to build apache from source because there are often many instances of apache running on one box (apache dev boxes).

AD(Active Directory) may have been part of the reason but the real reason(s) have been lost to time...

Should have included them in my notes and I could have gotten it from svn. :-)

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by multani (82.230.xx.xx) on Thu 23 Mar 2006 at 01:26
I had a problem with sending mail with the post-commit hook and the commit-mail.pl script (available in the subversion-tools package) with my users' log message.
It contains non-ascii characters (they are french, so the log there commits in french) : all my accentued characters where replaced by "?\195" or "?\162", or "\?....", etc., instead of "é, è, ç, ..."

For those who wants to fix the problem, consider setting the LANG environement variable to something like "fr_FR@euro" (adjusts it to your locale), as in (in the /hooks/post-commit script) :

LANG=fr_FR@euro /usr/lib/subversion/hook-scripts/commit-email.pl "$REPOS" "$REV" test@xxx.xxxx

There maybe better solutions (you need to set this fix for every hook of every repository :/ ), which I'll be glade to know :)

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by Anonymous (192.38.xx.xx) on Thu 23 Mar 2006 at 08:14
>still need to configure the SVN repository to use SSL. I'll post the steps that I >followed to configure SVN with SSL also.

You can add SSLRequireSSL


DAV svn
SVNParentPath /usr/local/svn
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /etc/svn-passwd
SSLRequireSSL


Then only ssl acces is allowed. SSL should work fine in the default apache2 instalation.


[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by Anonymous (138.131.xx.xx) on Thu 23 Mar 2006 at 12:52
I installed Subversion a few days ago following the instructions on:
http://projects.edgewall.com/trac/wiki/TracOnDebianSarge
which worked like a charm :)

(Trac btw. might be worth taking a look at as well!)

How ever, I'm new to debian and apache2 so I would like to know the debian way of apache2 configuration... i just did what's in the tutorial.

Your hint to use SSLRequireSSL didn't work for my default apache2 installation (not even after: a2enmod ssl). What else am I missing?

Would be great to have a comprehensive debian subversion setup, config and maintain articel here.

Regards

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by Anonymous (192.38.xx.xx) on Fri 24 Mar 2006 at 13:34
Sorry my bad. I properly forgot that I read this article about setting up an ssl server:
http://www.debian-administration.org/articles/349

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by Anonymous (66.209.xx.xx) on Thu 31 Jan 2013 at 14:36
Just in case anyone else runs into SSLRequireSSL doing nothing, you may need to add this right before it:

SSLOptions +StrictRequire

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by Anonymous (82.146.xx.xx) on Sun 26 Mar 2006 at 08:50
#chmod -R 777 svn

OUCH!

How about chown -R www-data:www-data svn; chmod -R 770 svn ?

With 777 you're givng permission for everything to write directly on the svn repository files, something that will most likely cause corruption, sooner or later. (the apache web daemon runs as the www-data system user by default in Debian)



Also, what are your reasons to go with the berkelydb format?
I have seen zero problems with the fsfs backend so far (9 months, 1800 commits, ~100MB).

I choose fsfs originally because it's filesystem-atomic, meaning that you can just tar up the svn dir at any time and have a consistent backup.


For SSL encryption (sometimes a must to get arround corporate firewalls/proxies, not many of them understand and/or allow svn!), just make apache2 use ssl the regular way, there are no svn implications and most clients appear to support it.


PS: it's worth noting that the latest svn version (not in stable atm) allows for true dav autoversioning.
This allows you to do stuff with a regular (non-svn) dav client, like the windows "mount remote location" thingy. Every change will be committed in svn, and is tracable & reversable with an svn client.
I haven't tried this out yet and I expect it to be slow, but ... it sure sounds great!

[ Parent | Reply to this comment ]

Re: Playing with SVN: SVNPath vs SVNParentPath, Debian Sarge
Posted by trygub (62.253.xx.xx) on Mon 12 Jun 2006 at 23:55
What goes below applies to Debian Sarge.

File /etc/apache2/mods-available/dav_svn.conf distributed as part of libapache2-svn package contains the following lines:

# Set this to the path to your repository
SVNPath /var/lib/svn

Second line should probably read

SVNParentPath /var/lib/svn

Otherwise, the error prevents from accessing the repository

[date] [error] [client x.x.x.x] Could not open the requested SVN filesystem [500, #2]

Semen Trygubenko

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by Ivan (84.45.xx.xx) on Mon 16 Oct 2006 at 11:56
Dear All

I found this 'Playing with Subversion' article very useful, so thanks to Dinesh for posting it.

I used the article to help me set up my own subversion repository, and I have documented my procedure at http://www.iau.ukfsn.org/iau/linux/subversionSetup.html. I'll update this document as my Subversion usage gets more ambitious (and as my understanding improves).

I don't know if I've got everything quite right, however. In particular I have two areas of doubt:

1. AuthzSVNAccessFile

I commented out the AuthzSVNAccessFile field in <location /svn>. I need AuthzSVNAccessFile only if I want to control access per directory, is that right? Actually I have no idea what Authz is for.

2. Permissions

The original poster gave everything 777 permissions. I can see this is a bad idea. Two corrections were offered:

(a) debian administration

# find /var/svn-repos/ -type d -exec chmod 2755 \{} \;
# find /var/svn-repos/ -type f -exec chmod 644 \{} \;

(b) http://trac.edgewall.org/wiki/TracOnDebianSarge

# find /var/svn-repos -type d -exec chmod 2770 \{} \;
# find /var/svn-repos -type f -exec chmod 660 \{} \;

My understanding of permission numbers is fairly rudimentary so I have two questions here:
- why are files and directories given different permisions?
- what is the difference between the two approaches?

I would appreciate any help on these two issues, and I'll update my subversionSetup page accordingly.

Thanks.

Best wishes

Ivan

[ Parent | Reply to this comment ]

Re: Playing with SVN
Posted by hesco (154.5.xx.xx) on Sun 7 Jan 2007 at 23:30
As I understand it, the directories have their execute bit set so that a system user can see into them. Otherwise, even though you may have the ability to read, write and execute `/path/to/file` if you know where to find it, without the x bit set on the directory, you would be unable to read the directory to find out the name of `file`.

[ Parent | Reply to this comment ]