the structure of the repec account at all.repec.org

Current maintainer: Thomas Krichel

Revision of 2007-04-04

Status of this document

This version is
http://openlib.org/home/krichel/work/mannheim.html

This document describes the structure and work of the repec account at all.repec.org. It does not describe everything that goes on the account, but those aspects that others will be interested in. It was written to commemorate the 13th anniversary of the creation on NetEc.

This document may be reproduced without permission.

The ~/ftp directory

Some parts of the account are accessible via anonymous ftp. They have to be kept in as separate directory. This directory is ~/ftp.

The ftp configuration is in /etc/vsftpd.conf. But the ftp home directory is fixed as the home directory of the user ftp. This home directory is found in /etc/passwd. It is ~/ftp.

~/ftp/CitEc/ contains a rsync'ed copy of the CitEc ReDIF files from citec.repec.org.

~/ftp/JelEc/ contains an AMF representation of the JEL classification data that is in all RePEc files.

~/ftp/LotEc/ contains a rsync'ed copy of the LotEc AMF files from sahure.openlib.org. This is an attempt to find full-text document files from RePEc metadata.

~/ftp/RePEc/ contains all the archives that are maintained locally and/or are available through the ftp://all.repec.org server.

The ~/RePEc directory

In ~/RePEc/remo all the remote archives are being kept.

In ~/RePEc/zzz, the local pseudo-archive is being kept. This is the directory that acts as the ReDIF-home that is seemingly a requirement of ReDIF-Perl. It contains a zzzarch.rdf files. It also contains a conf subdirectory with two files remi.raneb.conf and remi.core.conf.

The ~/obsolete directory

This directory contains data series that have been kept locally, but have since been taken over by other institutions. The "RePEc:fth" "RePEc:wop" and "RePEc:wuk" series hold such data.

When a series in made obsolete, it is saved in a tarball of the name RePEc_archive_series_obsolete.tar.gz.

To do this, a Perl script obsolete_series archive series can be used.

Maintenance

The crontab is in ~/etc/crontab.

remi is started using ~/perl/remi_round_the_clock. remi_round_the_clock just checks that there is no remi process running before it fires up remi. The remi command is ~/perl/remi -conf remi.raneb.conf. The mirrored archives are kept is ~/RePEc/remo. remi.raneb.conf is found in ~/RePEc/zzz.

In addition to this task, ~/perl/remi_round_the_clock rsyncs by directory to both nebka and sahure.

remi for the core archive is started with rm -f /var/tmp/remi.core ; perl ~/perl/remi -conf remi.core.conf >> ~/public_html/remi.core.last.txt 2>> ~/public_html/remi.core.last.txt . The configuration file remi.core.conf is found in ~/RePEc/zzz.

The script ~/perl/internal_remi provides an internal mirror for the local archives in the ~/ftp/RePEc directory. It finds the archives in ~/ftp/RePEc and rsyncs each archive with rsync -qa ~/ftp/RePEc/archive/ ~/RePEc/remo/archive. Then in second stage, it rsyncs the ???arch.rdf and ???seri.rdf from each internal archive into the RePEc:all archive with rsync -aq ~/ftp/RePEc/archive/archivearch.rdf ~/ftp/RePEc/all/archivearch.rdf and with rsync -aq ~/ftp/RePEc/archive/archiveseri.rdf ~/ftp/RePEc/all/archiveseri.rdf