for details on what is being mirrored.
ftp MUUG Online FTP area (more Gigs than you want to download!)
pub MUUG Online pub area (more Gigs than you want to download!)
mirror Mirror of various sites (more Gigs than you want to download!)
redhat ftp.redhat.com mirror (more Gigs than you want to download!)
redhat-contrib ftp.redhat.com mirror, contrib directory
redhat-updates updates.redhat.com mirror
fedora fedora.redhat.com mirror, top-level directory
fedora-linux-core fedora.redhat.com mirror, core directory
fedora-linux-core-updates fedora.redhat.com mirror, updates directory
fedora-linux-core-development fedora.redhat.com mirror, development directory
fedora-linux-core-test fedora.redhat.com mirror, (beta) test directory
fedora-linux-extras fedora.redhat.com mirror, extras directory
In this case, the introductory message indicates that fedora-linux-core-development contains the development tree. Use rsync again to view the contents of that directory:
$ rsync -v rsync://ftp.muug.mb.ca/fedora-linux-core-development/
Welcome to MUUG Online Network Access, courtesy of the
Manitoba Unix User Group.
...(Lines snipped)...
drwxrwsr-x 4096 2006/07/13 18:43:37 .
-rw-r--r-- 3101 2003/11/04 12:23:24 README
drwxr-xr-x 4096 2006/07/13 15:21:37 i386
drwxrwsr-x 4096 2006/07/13 15:21:33 source
sent 117 bytes received 544 bytes 440.67 bytes/sec
total size is 3101 speedup is 4.69
Don't omit the final / on the rsync URI.
The i386 directory is the one we're interested in (it looks like you'd have to use a different mirror for other architectures):
$ rsync -v rsync://ftp.muug.mb.ca/fedora-linux-core-development/i386/
Welcome to MUUG Online Network Access, courtesy of the
Manitoba Unix User Group.
...(Lines snipped)...
drwxr-xr-x 4096 2006/07/13 15:21:37 .
drwxr-xr-x 81920 2006/07/13 15:37:52 debug
drwxrwsr-x 4096 2006/07/12 08:09:49 iso
drwxrwsr-x 4096 2006/07/13 15:49:37 os
sent 123 bytes received 530 bytes 145.11 bytes/sec
total size is 0 speedup is 0.00
The presence of the debug , iso , and os subdirectories indicates that this is the directory we're looking for.
Armed with that information, create a script, /usr/local/bin/rawhide-rsync , on a stable (non-Rawhide) system with 10 GB or more free storage space:
#!/bin/bash
#
# rawhide-rsync :: script to mirror the Fedora rawhide repo locally
#
MAILTO= alert # Person/alias to receive reports
DIR= /var/www/html/rawhide # Mirror directory
URI=rsync:// mirrorhost/directory / # Rsync URI
if tty -s # If being run interactively, show progress
then
XCMD='tee /dev/tty'
else
XCMD='cat'
fi
(
cd $DIR || exit 2 # Abort if the cd fails (important!)
rsync --recursive --delete -v $URI . 2>&1
echo
)|$XCMD|mail $MAILTO -s "Rawhide Rsync Report"
Ensure that httpd and rsync are installed on the target system, and create a directory to hold the development mirror (replace user with the name of the non- root user account that you will be using to run the rawhide-repo script):
# mkdir -p /var/www/html/rawhide/
# chown user /var/www/html/rawhide
# chmod a+rx /var/www/html/rawhide
Finally, run the script:
$ rawhide-rsync
Welcome to MUUG Online Network Access, courtesy of the
Manitoba Unix User Group.
For any questions, problems, or concerns about this site,
please send e-mail to: <ftp@muug.mb.ca>.
Look under the /pub directory and subdirectories for files to download.
We are now also maintaining a mirror of selected sites (or a subset
thereof), in the /mirror directory. Look at the README file there
for details on what is being mirrored.
receiving file list ... done
debug/ElectricFence-debuginfo-2.2.2-20.2.2.i386.rpm
debug/ElectricFence-debuginfo-2.2.2-20.2.i386.rpm
debug/GConf2-debuginfo-2.14.0-2.1.i386.rpm
...(Lines snipped)...
os/repodata/repoview/zsh-html-0-4.2.5-1.2.2.html
os/repodata/repoview/zulu-support.group.html
sent 15296418 bytes received 706808440 bytes 166633.17 bytes/sec
total size is 8112656832 speedup is 11.23
The server and the local rsync program will compare notes and modify the files and directories on the local system to match the server. The first time the script is run, it will transfer the entire repository, and the speedup value will be 1.0 . In subsequent runs, the speedup value will indicate the amount of time saved over transferring the entire repository (the preceding example indicates that the transfer took 1/11.23 of the time that a full transfer would take).
There is a high rate of change in the development repository, and from time to time, most or all of the repository will be freshly rebuilt, resulting in very large transfers. If you have a transfer-limited or capped Internet account and run the rawhide-rsync script often, be careful that you don't accidentally exceed your transfer limits.
You can now automate the rsync process by adding a crontab entry. Using the non-root account that will be performing the mirroring, edit the crontab :
$ crontab -e
Modify the crontab file to start the rawhide-rsync script at a convenient time: