Skip to content

Instant update mirroring

abh edited this page Mar 6, 2011 · 17 revisions

Instant Update Mirroring

The future tier'ed CPAN mirroring will use File::Rsync::Mirror::Recent (also known as rrr). Currently for CPAN mirroring (as opposed to PAUSE-only mirroring...) a recent version from git is recommended.

This setup is just in testing for now; don't try it unless you want to help make sure it works!

Configuration

Email [email protected] for a username/password for the master mirror for testing and put it into the script below which you'll run to mirror CPAN "instantly". It will take a while to get synced up, but after that it'll get the latest changes every 20 seconds with very little overhead.

Client

#!/usr/bin/env perl
use strict;
use warnings;
use local::lib;

my $user = '';
my $pass = '';
my $CPAN = '/local/ask/CPAN';
my $tmp  = '/local/ask/tmp';

$ENV{RSYNC_PASSWORD} = $pass;

use File::Rsync::Mirror::Recent;
my $rrr = File::Rsync::Mirror::Recent->new
  (
   ignore_link_stat_errors => 1,
   localroot => $CPAN,
   tempdir   => $tmp,
   remote => $user . '@cpan-rsync-master.perl.org::CPAN/RECENT.recent',
   max_files_per_connection => 5000,
   rsync_options => {
                     compress => 1,
                     links => 1,
                     'safe-links' => 1,
                     times => 1,
                     checksum => 0,
                    },
   verbose => 1,
   _runstatusfile => "recent-rmirror-state.yml",
   _logfilefordone => "recent-rmirror-donelog.log",
  );
$rrr->rmirror ( "skip-deletes" => 0, loop => 1 );
Clone this wiki locally