0. This document is the Skaneateles protocol. Its initial version was written on 2002–05–05 by José Manuel Barrueco Cruz and Thomas Krichel. It replaces the Tokyo protocol for the maintenance of NetEc machines.
1. There are three NetEc machines. These sites are called
"midas", "coe" and "wustl". Each one has a site name and a host
name that are defined in /home/adnetec/etc/local on each
site. One site is called the main site. At the time of writing the
main site is midas.
2. The home directory must be /home/adnetec.
3. In order to avoid that we have to work with different shell
commands, we link, on each account ~/.bash_profile with
~/global/shell/bash_profile and ~/.bashrc with
~/global/shell/bashrc. In addition, if there are local
settings, they are included in ~/etc/local_bashrc and
~/etc/local_bash_profile.
4. All general purpose unix programs we need are placed in
~adnetec/bin or are soft links. Any compiled packages will be
set the "prefix" as ~adnetec. All perl scripts are called as
/home/adnetec/bin/perl. analog is an exceptional program, it
requires files that are in ~/analog4.01. JMBC to
investigate.
5. ~/perl stores all perl scripts that are needed to
manage a things that run on a particular site. This directory is not
mirrored.
6. There is a file name ~/global/etc/sync that list the
directories that need to be mirrored. In the main site we use
~/global/perl/sync to push the files to the peripheral
sites. And in the peripheral site we use this same script to pull the
files from the main site.
7. A file ~global/perl/repecsync may be used to sync
the RePEc file structure. It is based on rsyncing the
~RePEc/remo from a machine and then copying any
local ~/RePEc/??? directory to ~/RePEc/???.
The script takes the site to mirror from as the argument.
If this script is activated, it is done in the local
crontab.
8. The web pages of BibEc and WoPEc are created using the script
~/global/perl/update_bibec_wopec. Basically this script
launches two processes: rewe to create the pages and
index to generate the index of both projects.
9. ~/etc/crontab contains the crontab for each machine. A
change in the crontab must be recorded there before it is implemented.
10. ~/global/perl/nightly will be run every day on each
site. This script launches remi.
update_bibec_wopec will be launched
by the script on Wednesday and Sunday.
Nightly calls the ~/global/perl/makecentral script.
The last process launched by this script is localpages.
11. ~/global/perl/weekly will be run once a week on each
site. It runs the index of WebEc, EDIRC, JokEc, and EconFAQ.
12. ~/global/perl/monthly will be run once a month on
each site.
13. On the main site, EconFAQ is updated manually when a new
version is announce by Bill Goffe. Thomas places it in ~/EconFAQ
and runs ~/perl/ineconfaq on it. The pages are placed in
~/WWW/EconFAQ, from where they are mirrored to the peripheral
sites. Weekly indexes them.
14. remi is invoqued with ~/global/perl/remi -conf
remi.site.conf. The configuration
files live in ~/RePEc/zzz/conf, they are centrally
managed on the main site. When installing a new remi, care
must be taken to verify that it works.
15. On the main site, JokEc is updated by a cron call
to the script ~/perl/getjokec. This script mirrors JokEc files
from Finland into ~/JokEc. If there are changes in the files as
indicated in the mirror log, the JokEc files are copied into
~/WWW/JokEc. New individual joke files for indexing are created at
~/WWW/JokEc/jokes, and indexing is performed.
16. WebEc is updated by Lauri Saarinen, he transfers the
files with ftp into ~/WWW/WebEc on the main site.
From they are mirrored to the other site.
17. The update of EDIRC is done through remi. ~/WWW/EDIRC
is a soft link to the web pages exported by RePEc:edi. It is
indexed every week in weekly.
18. The script ~/perl/wlogger is called on each site in the
nightly script. It makes calls to ~/global/perl/wlog. This script
is a machine-independent log accumulation script. The accumulation
script accumulates parameterized files into a stock of partial logs,
who live in ~/partial/log.
19. A single script ~/global/perl/makecentral is called by
nightly to centralize log from the ~/partial/log directory to
~/central/log directories. This scripts works in a bidirectional
mode. It rysncs from the partial directory on the site that
it runs on to the other sites to the ~/central directory,
and it rsyncs from its own partial directory to the central directory
on the two other sites' global directory. However, no central
log is kept in midas, for reasons of space.
20. A script ~/global/perl/makecurrentlogec
makes the LogEc deposit of
logs. It is a customized call to Sune Karlsson's rstat script. It
is called by monthly on every machine, but does only do something one
one site, and exit gracefully on all other sites.
~/global/perl/makecurrentlogec is called by
~/global/perl/monthly.
21. A script ~/global/perl/projectmonthlog
makes the logs of EDIRC for the previous month. It
is called by monthly on every machine, but does only do something one
one site, which currently is coe. It deposits logs into
~/log/EDIRC.
22. In addition we use a site as the log center. The log center
gets all the files that are on the directory ~/partial on
each site to merge them in a directory ~/central. Several
logs centers may be created. but only one exports to LogEc.
23. The comprehensive usage statistics page of NetEc is updated in the
log center once a month. makestat.new is the script in charged of
create the new statistic pages for each project. It is fire up using
the crontab of the log center machine. The page
~/WWW/central/NetEc_Stats.html must be updated manually once we
have checked that the process has ended correctly.
24. NEP is based in an account admnep at the wustl site.
midas rsyncs the NEP documentation from ~admnep/public_html at
wustl to
~adnetec/WWW/NEP at midas. A reverse rsync job is carried
an by admnep at the wustl site. From midas, NEP is mirrored
to the other sites.
25. NEP reports are created using two scripts: nere and
sere. Both scripts are distributed in the wop archive.
26. New additions to RePEc are being discovered by
nere. This script is executed once a week using a cron call of
the admnep account in wustl. nere produces a
file ~admnep/var/stock/WoPEc.new-0.3 with the new
templates. The NEP general editor is in charged of going through the
file removing the old stuff. Once he has finished the work will
execute sere to send the customized reports for each list.
27. Every night, ~admnep/perl/nepmailparse parses NEP data
and issues dumps of the mail data ~admnep/var/nep_papers and
an AMF representation of NEP in
~admnep/public_html/amf/nep. This representation is rsynced to
adnetec@irwell:RePEc/wop/conf for export to RePEc services.
28. A technical guide to NEP is published at durl(http://nep.repec.org/technical.html).
29. To open a new list, su to list, and then do
/var/lib/mailman/bin/newlist. Finnally make the changes that
list admin can post only, that the subscriber list is only visible to
the list admin.
30. On midas, on ~/ftp/RePEc/help, there is a README file
about a RePEc checking service that used to live at mimas.
31. Each night on the main site, a script
~/shell/makerepecalltargz produces a new compressed archive file
for the RePEc all archive. It is stored in
~/RePEc/all/all.tar.gz.
32. Each night on the main site, an internal mirror
~/etc/mirror/internal_all is performed that mirrors
~/RePEc/all to ~/RePEc/remo/all. This is something
that is not performed by remi. In addition, there are calls to other
mirrors, in ~/etc/mirror/nst-fth and ~/etc/mirror/nst_wop,
for data for the RePEc:fth and RePec:wop archive, and mirrors
~/RePEc/all/root from openlib.org.
33. CitEc and HoPEc are documented elsewhere.