WIS Sync monitor runs 4-hourly

Starting 12Z yesterday (2012-02-08T12Z), the monitor runs 4 hourly instead
of 3 hourly before. This is due to increase of harvesting/processing time
these days. Apologies for any inconvenience.

WIS Sync Monitor Updated again

The WIS metadata sync monitor http://toyoda-eizi.net/2011/syncmon/mon.html
had pretty much updated. The greatest beauty is that we serve each archived
file (not only zipfiles): you can save URLs in the RSS
http://toyoda-eizi.net/2011/syncmon/news.atom and click them later days!

I hope that won't cause too much load on my server.

Each report in RSS is issued in XHTML instead of Atom Entry XML which was
hard to handle in mobile devices. For example the "change since last
monitor" page becomes HTML like
http://toyoda-eizi.net/za/hist2012W06.zip/20120206T03/DIAG.html
and each link leads you to archived list of identifiers. That is useful in
maintaining sync network.

WIS sync monitor updated

The WIS metadata sync monitor http://toyoda-eizi.net/2011/syncmon/mon.html
has been updated. The diffs no longer indicate lists of identifiers, as
they became unrealistically huge these days. The numbers are indicated
instead.

(1) Snapshot Diffs fixed - it has long been broken
http://toyoda-eizi.net/2011/syncmon/curr/DIFF.atoment

(2) CSV version of Snapshot Diffs added
http://toyoda-eizi.net/2011/syncmon/curr/DIFF.csv

(3) Diag Diffs fixed - it has long been broken
http://toyoda-eizi.net/2011/syncmon/curr/DIAG.atoment

(4) CSV version of Diag Diffs added
http://toyoda-eizi.net/2011/syncmon/curr/DIAG.csv

There were also internal improvements to collect data more robustly. Even
though I've paid every efforts of ugly parallel programming, the growing
size and slowing performance of servers makes the process a bit slower --
the update will be visible about 80 minutes after every 3 hours.

If you have RSS reader, try following the feed:
http://toyoda-eizi.net/2011/syncmon/news.atom

WISTEST-* sets removed

The WISTEST-* OAI sets was made for presenation at CG-XVI. The sets
included OAI identifiers in operational namespace
"urn:x-wmo:md:int.wmo.wis::*" and hence caused trouble in loading the WMO
baseline metadata.

After the test sets removed, 2702 records of baseline MD was inserted into
WIS-UNASSOCIATED set. It reflects only onto the set WIS-CATALOGUE, and I
believe that won't affect synchronisation among CMA/DWD/JMA.

fixed: size of OAI set displayed incorrectly

In the WIS community there is a convention that the size of OAI set is
included in ListSet response.

http://www.wis-jma.go.jp/meta/oaiprovider.jsp?verb=ListSets

Until last week there was an problem that the size of WIS-GISC-TOKYO is not
displayed correctly. It is now fixed and it returns reliable number.

The documentation at http://www.wis-jma.go.jp/cms/metadata/oai-pmh/ uses the
ListSets internally, and hence fixed accordingly.

OAI Sync Monitor update

Recent updates of OAI Sync Monitor
http://toyoda-eizi.net/2011/syncmon/mon.html

- Beijing joined and the set WIS-GISC-BEIJING introduced
- Temporary set WIS-UNASSOCIATED introduced
- Testing set WISTEST-* no longer monitored.

metadata updates

- GISC Tokyo has loaded GTS baseline metadata for VBRR, VNNN, VTBB, VLIV,
VMMC, RPMM, VHHH.
- Set WIS-WIMMS created.
- most of WISTEST-JMA set has been deleted; right now deletion in terms of
OAI so other synching centres may harvest that del-notification to react.
Four records are kept just for testing.