Defcon Kaklkaklkekl (Wiki)

====== Some experimentation with git ====== The world has already moved from Centralized Version Control Systems (CVS, SVN et.al) to Distributed Version Control Systems, and it has for long been my intention to play catch-up on that. For me, and for quite a lot of open-source oriented people, the de-facto choice has become Git (http://git-scm.com/), so that is the DVCS I will be focusing on. Now, starting to use Git as a single user, or even as multiple contributors when working on a shared system is very straight-forward. In these scenarios, you can live with all source-control happening in the locally available file system. The [[http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html|git tutorial]] and [[http://cworth.org/hgbook-git/tour/|Carl Worths "A tour of git"]] will tell you hot to get started with that. But, I keep having the desire/need to publish my work more publicly than on my local file system. And coming from a CVSC-background, I have become very used to having external contributors to my code, without having to be a "patch-broker". In addition, I use "VCS-pushing" of code as a form of easily available backup. I have little or no desire to add local Unix-accounts to my web-server to allow for external contributions, and I would like to avoid having to run a separate Daemon process. With that basis, I'll describe my foray into git. NOTE: For those of you who want to know about using git, and are not interrested in setting up a "svn-to-git-migration-like" server, I have a section on [[#Everyday use of git]] towards the end. ===== About the environment ===== I am implementing this using a web-server running Debian Lenny and apache2, hosting all git repositories from a separate virtualhost. My example server hostname will be "git.example.com". My example clients will be a Debian Lenny system and an Ubuntu 9.10.11.12 FIXME. ===== Installing ===== ===== Attempt one ===== As you probably get from the heading, I have ended up using a couple of tries getting my setup workable. There is one single reason for this; I started out with the old version of git supplied with Debian Lenny, git 1.5.6.5. But I realized after setting this scenario up, that this version will allow for either anonymous read/write or authenticated read/write. There is no sane way to do anonymous read, authenticated write. To get that functionality, you have to go for git 1.6 or newer, where the more recent [[http://www.kernel.org/pub/software/scm/git/docs/git-http-backend.html|git-http-backend]] is supported. There is still an important reason to keep this part of the document. I have not been able, despite serious attempt, troubleshooting, irritation and swearing, to get a git 1.7.1 http-backend based setup working with git-push from a 1.5.6 client! So, if you need to support the Debian Lenny default git version for direct upload (push) over HTTP, use the information in this section. On the other hand, if you can live with having to be a patch-broker for users of this older version of git, the [[#Attempt two]] section is for you. ==== Serverside install ==== We need to install the git software first, and then prepare a repository for publication via web: <code> apt-get install git-core mkdir /var/www/git.example.com mkdir /var/www/git.example.com/.passdb cd /var/www/git.example.com mkdir test-repo.git cd test-repo.git git --bare init chown -R www-data.www-data . chmod +x hooks/post-update </code> Next, we create the required configuration of Apache2's VirtualHost: <code> cat << EOF > /etc/apache2/sites-available/git.example.com.conf <VirtualHost *:80> ServerAdmin webmaster@example.com ServerName git.example.com DocumentRoot /var/www/git.example.com <Directory /var/www/git.example.com> Options +Indexes AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/git.example.com/.passdb> AllowOverride None Order deny,allow deny from all </Directory> <Location /test-repo.git> DAV on AuthType Basic AuthName "Git" AuthUserFile /var/www/git.example.com/.passdb/test.git Require valid-user </Location> </VirtualHost> EOF </code> So, this VirtualHost will require: * A htpasswd file for users / groups * The mod_dav module * The mod_dav_fs module <code> htpasswd -c /var/www/git.example.com/.passdb/test.git testuser a2enmod dav a2enmod dav_fs a2ensite git.example.com /etc/init.d/apache2 reload </code> ==== Clientside install ==== First, we need to test that WebDAV works properly. I do this using cadaver, the command line DAV client: <code> sudo apt-get install cadaver cadaver http://git.example.com/test-repo.git </code> <code> Authentication required for Git test on server `git.example.com': Username: testuser Password: </code> With that working, it is time to prepare our local git environment. Steps to do here: * Set up automatic authentication for curl/git-push-http * Verify authentication * Install the git program core * Set up your identity as represented with git. Change this as needed, in the form I list this, I completely replace any previous content: <code> cat << EOF > ~/.netrc machine git.example.com username testuser password Pa55w0rd EOF chmod 0600 ~/.netrc </code> To verify that the .netrc file works, this should not fail: <code> curl --netrc --location -v http://git.example.com/test-repo.git/HEAD </code> Install and configure git: <code> sudo apt-get install git-core git config --global user.name "My Name" git config --global user.email email@example.com </code> The setup is now tested with a sample data-set. I did quite a bulk of commands to test and troubleshoot my setup, I am listing them here just as much for my own reference as yours :) <code> cd ~/testcode git init git add . git commit git push http://testuser@git.example.com/test-repo.git/ master git branch git branch testing git checkout testing vim newfile git add newfile git commit git push http://testuser@git.example.com/test-repo.git/ testing git ls-remote http://git.example.com/test-repo.git mkdir ../testout cd ../testout git clone http://testuser@git.example.com/test-repo.git git branch -a git checkout -b testing origin/testing </code> Note that whenever you need to add a new repository to this kind of setup, you will have to update the apache configuration, adding a new Location block. ===== Attempt two ===== As I mentioned in the [[#Attempt one]] section, the older versions of git leave a bit to be desired when it comes to access options. But, looking at the results from the [[https://git.wiki.kernel.org/index.php/GitSurvey2010#05._Which_Git_version.28s.29_are_you_using.3F|Git User's Survey 2020]], it seems that maintaining support for these older git-clients should not really be needed. So, I tore the previous setup down, and started with more modern, recent technology. I can live with users of git 1.5.x submitting patches via email, and forcing people who need direct push access into using a newer version. ==== Serverside install ==== The serverside install starts out very similar to the previous attempt, but this time differs strongly by pulling in the new (1.7.x) version of git from Lenny Backports. <code> cat << EOF > /etc/apt/sources.list.d/lenny-backports.list deb http://backports.debian.org/debian-backports lenny-backports main contrib non-free EOF apt-get update apt-get install debian-backports-keyring apt-get autoremove git-core apt-get -t lenny-backports install git </code> Preparation of our initial repo is identical to attempt one: <code> mkdir /var/www/git.example.com mkdir /var/www/git.example.com/.passdb cd /var/www/git.example.com mkdir test-repo.git cd test-repo.git git --bare init chown -R www-data.www-data . chmod +x hooks/post-update </code> The VirtualHost definition for Apache2 is changed somewhat. Most notably, it has become longer, and now uses a ScriptAlias to execute CGI instead of relying on WebDAV <code> <VirtualHost *:80> ServerAdmin webmaster@example.com ServerName git.example.com DocumentRoot /var/www/git.example.com <Directory /var/www/git.example.com> Options +Indexes +FollowSymLinks +MultiViews +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/git.example.com/.passdb> AllowOverride None Order deny,allow deny from all </Directory> <Directory /usr/lib/git-core> Options None </Directory> <Files /usr/lib/git-core/git-http-backend> Options ExecCGI </Files> CustomLog /var/log/apache2/git.example.com_access.log combined ErrorLog /var/log/apache2/git.example.com_error.log LogLevel warn SetEnv GIT_PROJECT_ROOT /var/www/git.example.com SetEnv GIT_HTTP_EXPORT_ALL SetEnv REMOTE_USER=$REDIRECT_REMOTE_USER ScriptAliasMatch \ "(?x)^/(.*/(HEAD | \ info/refs | \ objects/(info/[^/]+ | \ [0-9a-f]{2}/[0-9a-f]{38} | \ pack/pack-[0-9a-f]{40}\.(pack|idx)) | \ git-(upload|receive)-pack))$" \ "/usr/lib/git-core/git-http-backend/$1" <LocationMatch "^/.*/git-receive-pack$"> AuthType Basic AuthName "Git Access" AuthUserFile /var/www/git.example.com/.passdb/test.git Require valid-user </LocationMatch> </VirtualHost> </code> Now, create the htpasswd file and enable the configuration. Note that I disable my old WebDAV support completely here. If you need WebDAV for something unrelated, please make sure you skip those lines :) <code> a2dismod dav a2dismod dav_fs htpasswd -c /var/www/git.example.com/.passdb/test.git testuser a2ensite git.example.com /etc/init.d/apache2 reload </code> ==== Clientside install ==== This is for Debial Lenny. If you have the older version of git-core installed, start by removing it: <code> sudo apt-get install git-core </code> Next, pull in the new version from Lenny Backports: <code> cat << EOF > /etc/apt/sources.list.d/lenny-backports.list deb http://backports.debian.org/debian-backports lenny-backports main contrib non-free EOF apt-get update apt-get install debian-backports-keyring apt-get -t lenny-backports install git </code> Now, configure git global options (your identity): <code> sudo apt-get install git-core git config --global user.name "My Name" git config --global user.email email@example.com </code> Set up a .netrc file to avoid those pesky password prompts. Change this as needed, in the form I list this, I completely replace any previous content: <code> cat << EOF > ~/.netrc machine git.example.com username testuser password Pa55w0rd EOF chmod 0600 ~/.netrc </code> The setup is now tested with a sample data-set. Note a difference from the 1.5-series git version, I no longer list the username in the git http URL. <code> cd ~/testcode git init git add . git commit git push http://git.example.com/test-repo.git/ master mkdir ../testout cd ../testout git clone http://git.example.com/test-repo.git </code> ===== A prettyfied web-view: gitweb ===== Install gitweb from Lenny Backports, and point it to the root of the GIT projects: <code> apt-get install -t lenny-backports gitweb </code> Add the following to the Apache2 VirtualHost: <code> <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> Alias /gitweb /usr/share/gitweb ScriptAlias / /usr/share/gitweb/index.cgi/ <Directory /var/www/git.example.com/.gitweb> AllowOverride None Order deny,allow deny from all </Directory> <Location "/private"> AuthType Basic AuthName "Git Private" AuthUserFile /var/www/git.example.com/.passdb/private_users Require valid-user </Location> </code> <code> cat << EOF > /etc/gitweb.conf # path to git projects (<project>.git) $projectroot = "/var/www/git.example.com"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages #$home_link = $my_uri || "/"; # html text to include at home page # $home_text = "indextext.html"; $home_text = "/var/www/git.example.com/.gitweb/index.inc"; # The text on the home link in the breadcrumbs $home_link_str = "git.example.com"; # file with project list $projects_list = "/var/www/git.example.com/.gitweb/list"; # Hide any repo not listed in the $projects_list $strict_export = 1; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; EOF echo "Example text" > /var/www/git.example.com/.gitweb/index.inc ls | grep -v private | awk ' { print $1 " Me" }' > /var/www/git.example.com/.gitweb/list </code> ===== Everyday use of git ===== ===== References ===== http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html http://cworth.org/hgbook-git/tour/ http://www.kernel.org/pub/software/scm/git/docs/howto/setup-git-server-over-http.txt http://www.espace.com.eg/blog/2010/11/10/migrating-from-subversion-to-git/ http://www.kernel.org/pub/software/scm/git/docs/git-http-backend.html