Changes RSS

Some experimentation with git

The world has already moved from Centralized Version Control Systems (CVS, SVN et.al) to Distributed Version Control Systems, and it has for long been my intention to play catch-up on that. For me, and for quite a lot of open-source oriented people, the de-facto choice has become Git (http://git-scm.com/), so that is the DVCS I will be focusing on.

Now, starting to use Git as a single user, or even as multiple contributors when working on a shared system is very straight-forward. In these scenarios, you can live with all source-control happening in the locally available file system. The git tutorial and Carl Worths "A tour of git" will tell you hot to get started with that.

But, I keep having the desire/need to publish my work more publicly than on my local file system. And coming from a CVSC-background, I have become very used to having external contributors to my code, without having to be a “patch-broker”. In addition, I use “VCS-pushing” of code as a form of easily available backup. I have little or no desire to add local Unix-accounts to my web-server to allow for external contributions, and I would like to avoid having to run a separate Daemon process.

With that basis, I'll describe my foray into git.

NOTE: For those of you who want to know about using git, and are not interrested in setting up a “svn-to-git-migration-like” server, I have a section on Everyday use of git towards the end.

About the environment

I am implementing this using a web-server running Debian Lenny and apache2, hosting all git repositories from a separate virtualhost. My example server hostname will be “git.example.com”. My example clients will be a Debian Lenny system and an Ubuntu 9.10.11.12 FIXME.

Installing

Attempt one

As you probably get from the heading, I have ended up using a couple of tries getting my setup workable. There is one single reason for this; I started out with the old version of git supplied with Debian Lenny, git 1.5.6.5. But I realized after setting this scenario up, that this version will allow for either anonymous read/write or authenticated read/write. There is no sane way to do anonymous read, authenticated write. To get that functionality, you have to go for git 1.6 or newer, where the more recent git-http-backend is supported.

There is still an important reason to keep this part of the document. I have not been able, despite serious attempt, troubleshooting, irritation and swearing, to get a git 1.7.1 http-backend based setup working with git-push from a 1.5.6 client! So, if you need to support the Debian Lenny default git version for direct upload (push) over HTTP, use the information in this section.

On the other hand, if you can live with having to be a patch-broker for users of this older version of git, the Attempt two section is for you.

Serverside install

We need to install the git software first, and then prepare a repository for publication via web:

apt-get install git-core

mkdir /var/www/git.example.com
mkdir /var/www/git.example.com/.passdb

cd /var/www/git.example.com
mkdir test-repo.git
cd test-repo.git
git --bare init
chown -R www-data.www-data .
chmod +x hooks/post-update

Next, we create the required configuration of Apache2's VirtualHost:

cat << EOF > /etc/apache2/sites-available/git.example.com.conf
<VirtualHost *:80>
        ServerAdmin webmaster@example.com
        ServerName  git.example.com
        DocumentRoot /var/www/git.example.com

        <Directory /var/www/git.example.com>
                Options +Indexes
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        <Directory /var/www/git.example.com/.passdb>
                AllowOverride None
                Order deny,allow
                deny from all
        </Directory>

        <Location /test-repo.git>
                DAV on
                AuthType Basic
                AuthName "Git"
                AuthUserFile /var/www/git.example.com/.passdb/test.git
                Require valid-user
        </Location>
</VirtualHost>
EOF

So, this VirtualHost will require:

  • A htpasswd file for users / groups
  • The mod_dav module
  • The mod_dav_fs module
htpasswd -c /var/www/git.example.com/.passdb/test.git testuser
a2enmod dav
a2enmod dav_fs
a2ensite git.example.com
/etc/init.d/apache2 reload

Clientside install

First, we need to test that WebDAV works properly. I do this using cadaver, the command line DAV client:

sudo apt-get install cadaver
cadaver http://git.example.com/test-repo.git
Authentication required for Git test on server `git.example.com':
Username: testuser
Password: 

With that working, it is time to prepare our local git environment. Steps to do here:

  • Set up automatic authentication for curl/git-push-http
  • Verify authentication
  • Install the git program core
  • Set up your identity as represented with git.

Change this as needed, in the form I list this, I completely replace any previous content:

cat << EOF > ~/.netrc
machine git.example.com
  username testuser
  password Pa55w0rd
  
EOF
chmod 0600 ~/.netrc

To verify that the .netrc file works, this should not fail:

curl --netrc --location -v http://git.example.com/test-repo.git/HEAD

Install and configure git:

sudo apt-get install git-core
git config --global user.name "My Name"
git config --global user.email email@example.com

The setup is now tested with a sample data-set. I did quite a bulk of commands to test and troubleshoot my setup, I am listing them here just as much for my own reference as yours :)

cd ~/testcode
git init
git add .
git commit
git push http://testuser@git.example.com/test-repo.git/ master
git branch
git branch testing
git checkout testing
vim newfile
git add newfile
git commit
git push http://testuser@git.example.com/test-repo.git/ testing
git ls-remote http://git.example.com/test-repo.git
mkdir ../testout
cd ../testout
git clone http://testuser@git.example.com/test-repo.git
git branch -a
git checkout -b testing origin/testing

Note that whenever you need to add a new repository to this kind of setup, you will have to update the apache configuration, adding a new Location block.

Attempt two

As I mentioned in the Attempt one section, the older versions of git leave a bit to be desired when it comes to access options. But, looking at the results from the Git User's Survey 2020, it seems that maintaining support for these older git-clients should not really be needed. So, I tore the previous setup down, and started with more modern, recent technology. I can live with users of git 1.5.x submitting patches via email, and forcing people who need direct push access into using a newer version.

Serverside install

The serverside install starts out very similar to the previous attempt, but this time differs strongly by pulling in the new (1.7.x) version of git from Lenny Backports.

cat << EOF > /etc/apt/sources.list.d/lenny-backports.list
deb http://backports.debian.org/debian-backports lenny-backports main contrib non-free
EOF
apt-get update
apt-get install debian-backports-keyring
apt-get autoremove git-core
apt-get -t lenny-backports install git

Preparation of our initial repo is identical to attempt one:

mkdir /var/www/git.example.com
mkdir /var/www/git.example.com/.passdb

cd /var/www/git.example.com
mkdir test-repo.git
cd test-repo.git
git --bare init
chown -R www-data.www-data .
chmod +x hooks/post-update

The VirtualHost definition for Apache2 is changed somewhat. Most notably, it has become longer, and now uses a ScriptAlias to execute CGI instead of relying on WebDAV

<VirtualHost *:80>
        ServerAdmin webmaster@example.com
        ServerName  git.example.com
        DocumentRoot /var/www/git.example.com

        <Directory /var/www/git.example.com>
                Options +Indexes +FollowSymLinks +MultiViews +ExecCGI
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        <Directory /var/www/git.example.com/.passdb>
                AllowOverride None
                Order deny,allow
                deny from all
        </Directory>
        <Directory /usr/lib/git-core>
                Options None
        </Directory>
        <Files /usr/lib/git-core/git-http-backend>
                Options ExecCGI
        </Files>
        
        CustomLog /var/log/apache2/git.example.com_access.log combined
        ErrorLog /var/log/apache2/git.example.com_error.log
        LogLevel warn

        SetEnv GIT_PROJECT_ROOT /var/www/git.example.com
        SetEnv GIT_HTTP_EXPORT_ALL
        SetEnv REMOTE_USER=$REDIRECT_REMOTE_USER

        ScriptAliasMatch \
           "(?x)^/(.*/(HEAD | \
               info/refs | \
               objects/(info/[^/]+ | \
                   [0-9a-f]{2}/[0-9a-f]{38} | \
                   pack/pack-[0-9a-f]{40}\.(pack|idx)) | \
               git-(upload|receive)-pack))$" \
           "/usr/lib/git-core/git-http-backend/$1"

        <LocationMatch "^/.*/git-receive-pack$">
                AuthType Basic
                AuthName "Git Access"
                AuthUserFile /var/www/git.example.com/.passdb/test.git
                Require valid-user
        </LocationMatch>
        
</VirtualHost>

Now, create the htpasswd file and enable the configuration. Note that I disable my old WebDAV support completely here. If you need WebDAV for something unrelated, please make sure you skip those lines :)

a2dismod dav
a2dismod dav_fs

htpasswd -c /var/www/git.example.com/.passdb/test.git testuser
a2ensite git.example.com
/etc/init.d/apache2 reload

Clientside install

This is for Debial Lenny. If you have the older version of git-core installed, start by removing it:

sudo apt-get install git-core

Next, pull in the new version from Lenny Backports:

cat << EOF > /etc/apt/sources.list.d/lenny-backports.list
deb http://backports.debian.org/debian-backports lenny-backports main contrib non-free
EOF
apt-get update
apt-get install debian-backports-keyring
apt-get -t lenny-backports install git

Now, configure git global options (your identity):

sudo apt-get install git-core
git config --global user.name "My Name"
git config --global user.email email@example.com

Set up a .netrc file to avoid those pesky password prompts. Change this as needed, in the form I list this, I completely replace any previous content:

cat << EOF > ~/.netrc
machine git.example.com
  username testuser
  password Pa55w0rd
  
EOF
chmod 0600 ~/.netrc

The setup is now tested with a sample data-set. Note a difference from the 1.5-series git version, I no longer list the username in the git http URL.

cd ~/testcode
git init
git add .
git commit
git push http://git.example.com/test-repo.git/ master

mkdir ../testout
cd ../testout
git clone http://git.example.com/test-repo.git

A prettyfied web-view: gitweb

Install gitweb from Lenny Backports, and point it to the root of the GIT projects:

apt-get install -t lenny-backports gitweb

Add the following to the Apache2 VirtualHost:

        <Directory /usr/share/gitweb>
                Options FollowSymLinks +ExecCGI
                AddHandler cgi-script .cgi
        </Directory>
        Alias /gitweb /usr/share/gitweb

        ScriptAlias / /usr/share/gitweb/index.cgi/
        
        <Directory /var/www/git.example.com/.gitweb>
                AllowOverride None
                Order deny,allow
                deny from all
        </Directory>
        <Location "/private">
                AuthType Basic
                AuthName "Git Private"
                AuthUserFile /var/www/git.example.com/.passdb/private_users
                Require valid-user
        </Location>
cat << EOF > /etc/gitweb.conf
# path to git projects (<project>.git)
$projectroot = "/var/www/git.example.com";

# directory to use for temp files
$git_temp = "/tmp";

# target of the home link on top of all pages
#$home_link = $my_uri || "/";

# html text to include at home page
# $home_text = "indextext.html";
$home_text = "/var/www/git.example.com/.gitweb/index.inc";

# The text on the home link in the breadcrumbs
$home_link_str = "git.example.com";

# file with project list
$projects_list = "/var/www/git.example.com/.gitweb/list";

# Hide any repo not listed in the $projects_list
$strict_export = 1;

# stylesheet to use
$stylesheet = "/gitweb/gitweb.css";

# logo to use
$logo = "/gitweb/git-logo.png";

# the 'favicon'
$favicon = "/gitweb/git-favicon.png";
EOF

echo "Example text" > /var/www/git.example.com/.gitweb/index.inc
ls | grep -v private | awk ' { print $1 " Me" }' > /var/www/git.example.com/.gitweb/list 

Everyday use of git

References