latest text (CBS recommendation) of WMO Core Metadata Profile v1.3

So sorry for late announcement: it should have been posted public to be basis of discussion.....

 

In this September, WMO/CBS has recommended the WMO Core Metadata Profile (WCMP) version 1.3 in its fifteenth session. 

https://docs.google.com/open?id=0B9zHt7XJp45Ta3I3SWFRbWN0TVk

 

Even though it has to be approved by PTC and Excecutive Council before formally taking force, but discussion on the WCMP is starting up, and people have to refer to this as the base version.

 

WMO Codes updated: Fast Track 2012bis & Inter-CBS 2012

Effective 7th November 2012,  two sets of amendments to WMO's Manual on Codes are operational:

 - 2nd fast track procedure in 2012

 - inter-sessional procedure in 2012

 

Private notes on the changes:

http://redmine.toyoda-eizi.net/projects/wmocodes/activity

Changes to Tsunami-related msgs by NOAA/WCATWC eff Nov1 2012

There was NOAA notice:
http://www.nws.noaa.gov/os/notification/scn12-33atwc_products.txt

It says:
- two bulletin headers are discontinued (WEPA43 and WEXX22 PAAQ) and others
(WEPA41 WEXX20 WEAK51 WEXX30 WEAK53 WEXX32 SEAK71 SEUS71 SEXX60 WEPA43 and
WEXX22 PAAQ) will be sent in newer format.
- details will be http://wcatwc.arh.noaa.gov/ExampleProducts

All above-mentioned messages are disseminated internationally and found at
GISC Tokyo.
http://www.wis-jma.go.jp/data/browse?Indicator=PAAQ

migration of DDB (distributed database) service of JMA to GISC Tokyo

Since (I think) 1997, Japan Meteorological Agency (JMA) operated a server
http://ddb.kishou.go.jp/ as a part of DDB (distributed database), which was
a subprogramme of WWW (World Weather Watch) of WMO (World Meteorological
Organization).

More than a decade has passed, and now the infrastructure of WMO programmes
has been reorganized into WIS (WMO Information System). JMA has been
operating GISC Tokyo http://www.wis-jma.go.jp/cms/ since August 2011,
together with TCC (Tokyo Climate Center)
http://ds.data.jma.go.jp/tcc/tcc/index.html .

The hardware for the "ddb" server will be decommissioned at the end of
February 2013. But most of services are already available also at GISC
Tokyo or TCC. Users are advised to migrate old URL to new ones. Please
check actual pages in http://ddb.kishou.go.jp/ for detailed migration
instruction.

Termination of direct broadcasting to MDUS and SDUS Users via MTSAT

There was a news release dated 3 September.
http://www.jma.go.jp/jma/jma-eng/satellite/nmhs/transition.html

The release links to a PDF including details of the product of Himawari-8
and 9.

"mean" radius of the Earth

I usually consider the Earth as a sphere when thinking of map projection.
That's natural for meteorologist. The question is what's the radius we
should use in this context.

In the framework of international operational meteorology, i.e. World
Weather Watch programme of WMO, I think the Manual on Codes is the only
document that mention about the radius of spherical Earth.

: 6367.47 km:
GRIB Edition 1, code table 7,
GRIB Edition 2, code table 3.2, code 0 (which says 6367 470.0 m)
: 6 371 229.0 m:
GRIB Edition 2, code table 3.2, code 6,
Attachment for icosahedron-based grid
: 6 371 200 m:
GRIB Edition 2, code table 3.2, code 8

But I don't know which to choose, since these numbers do not have
definition.

From the viewpoint of cartography, there are several kinds of "spherical
radius" of Earth. Wikipedia http://en.wikipedia.org/wiki/Earth_radius gives
a convenient summary. Now I'm able to compute those values for well-known
spheroids. Interpretations:

: 6367.47 km:
Rectifying radius for (probably) GRS-67 or International 1967 spheroid.
Later GRIB2 added trailing zeroes without referring back to the
definition.
: 6 371 229.0 m:
Mean radius of Hayford 1910 (also known as International 1924) spheroid,
rounded to nearest metre.
Source of the last digit (of 0.1 m) is unclear, but probrably just an
inappropriate handling.
: 6 371 200 m:
Mean, volumetric, or authalic radius for Hayford 1910 spheroid, rounde to
nearest 100 m.
No better interpretation was found.

Now that the CBS Resolution says WGS 84 is the standard frame, I think
neither of above is appropriate any longer. It would be much more
reasonable to use 6 371 007.1 (authalic) or 6 371 008.7 (mean) for WGS-84,
depending on application. It might be meaningful to use rectifying radius
(6367449.1) if we are really interested in the length of meridian, but it's
unlikely.


::: program :::

#!/usr/bin/ruby
params = (<<EOF).split(/\n/)
Maupertuis (1738) 6,397,300 6,363,806.283 191 France
Plessis (1817) 6,376,523.0 6,355,862.9333 308.64 France
Everest (1830) 6,377,299.365 6,356,098.359 300.80172554 India
Everest 1830 Modified (1967) 6,377,304.063 6,356,103.0390 300.8017 West
Malaysia & Singapore
Everest 1830 (1967 Definition) 6,377,298.556 6,356,097.550 300.8017
Brunei & East Malaysia
Airy (1830) 6,377,563.396 6,356,256.909 299.3249646 Britain
Bessel (1841) 6,377,397.155 6,356,078.963 299.1528128 Europe, Japan
Clarke (1866) 6,378,206.4 6,356,583.8 294.9786982 North America
Clarke (1878) 6,378,190 6,356,456 293.4659980 North America
Clarke (1880) 6,378,249.145 6,356,514.870 293.465 France, Africa
Helmert (1906) 6,378,200 6,356,818.17 298.3
Hayford (1910) 6,378,388 6,356,911.946 297 USA
International (1924) 6,378,388 6,356,911.946 297 Europe
NAD 27 (1927) 6,378,206.4 6,356,583.800 294.978698208 North America
Krassovsky (1940) 6,378,245 6,356,863.019 298.3 USSR
WGS66 (1966) 6,378,145 6,356,759.769 298.25 USA/DoD
Australian National (1966) 6,378,160 6,356,774.719 298.25 Australia
New International (1967) 6,378,157.5 6,356,772.2 298.24961539
GRS-67 (1967) 6,378,160 6,356,774.516 298.247167427
South American (1969) 6,378,160 6,356,774.719 298.25 South America
WGS-72 (1972) 6,378,135 6,356,750.52 298.26 USA/DoD
GRS-80 (1979) 6,378,137 6,356,752.3141 298.257222101 Global ITRS
WGS-84 (1984) 6,378,137 6,356,752.3142 298.257223563 Global GPS
IERS (1989) 6,378,136 6,356,751.302 298.257
IERS (2003) 6,378,136.6 6,356,751.9 298.25642
EOF

for line in params
name, as, bs, finv = line.split(/\t/)
h = Hash.new
h[:equatorial] = a = as.gsub(/,/, '_').to_f
h[:polar] = b = bs.gsub(/,/, '_').to_f
f = 1.0 / finv.to_f
h[:mean] = (a + a + b) / 3.0
h[:volumetric] = (a * a * b) ** (1.0 / 3.0)
e = Math::sqrt(f * (2.0 - f))
halfq = 0.5 * (1.0 + ((1.0 - e * e) * 0.5 / e) * Math::log((1.0 + e) /
(1.0 - e)))
h[:authalic] = a * Math::sqrt(halfq)
h[:rectifying] = ((a ** 1.5 + b ** 1.5) * 0.5) ** (1.0 / 1.5)
for k in [:equatorial, :polar, :mean, :volumetric, :authalic, :rectifying]
fmt = sprintf("%%-11.%uf %%32s %%s\n", (k == :rectifying) ? 1 : 3)
printf(fmt, h[k], name, k.to_s)
end
end

::: RESULT :::

6355862.933 Plessis (1817) polar
6356078.963 Bessel (1841) polar
6356097.550 Everest 1830 (1967 Definition) polar
6356098.359 Everest (1830) polar
6356103.039 Everest 1830 Modified (1967) polar
6356256.909 Airy (1830) polar
6356456.000 Clarke (1878) polar
6356514.870 Clarke (1880) polar
6356583.800 Clarke (1866) polar
6356583.800 NAD 27 (1927) polar
6356750.520 WGS-72 (1972) polar
6356751.302 IERS (1989) polar
6356751.900 IERS (2003) polar
6356752.314 GRS-80 (1979) polar
6356752.314 WGS-84 (1984) polar
6356759.769 WGS66 (1966) polar
6356772.200 New International (1967) polar
6356774.516 GRS-67 (1967) polar
6356774.719 South American (1969) polar
6356774.719 Australian National (1966) polar
6356818.170 Helmert (1906) polar
6356863.019 Krassovsky (1940) polar
6356911.946 Hayford (1910) polar
6356911.946 International (1924) polar
6363806.283 Maupertuis (1738) polar
6366197.2 Plessis (1817) rectifying
6366702.5 Everest 1830 (1967 Definition) rectifying
6366703.3 Everest (1830) rectifying
6366708.0 Everest 1830 Modified (1967) rectifying
6366742.5 Bessel (1841) rectifying
6366914.6 Airy (1830) rectifying
6367327.6 Clarke (1878) rectifying
6367386.6 Clarke (1880) rectifying
6367399.7 Clarke (1866) rectifying
6367399.7 NAD 27 (1927) rectifying
6367447.2 WGS-72 (1972) rectifying
6367448.1 IERS (1989) rectifying
6367448.7 IERS (2003) rectifying
6367449.1 GRS-80 (1979) rectifying
6367449.1 WGS-84 (1984) rectifying
6367456.9 WGS66 (1966) rectifying
6367469.3 New International (1967) rectifying
6367471.7 GRS-67 (1967) rectifying
6367471.8 South American (1969) rectifying
6367471.8 Australian National (1966) rectifying
6367513.6 Helmert (1906) rectifying
6367558.5 Krassovsky (1940) rectifying
6367654.5 Hayford (1910) rectifying
6367654.5 International (1924) rectifying
6369628.860 Plessis (1817) volumetric
6369634.826 Plessis (1817) authalic
6369636.311 Plessis (1817) mean
6370223.708 Everest 1830 (1967 Definition) volumetric
6370224.517 Everest (1830) volumetric
6370229.209 Everest 1830 Modified (1967) volumetric
6370229.991 Everest 1830 (1967 Definition) authalic
6370230.799 Everest (1830) authalic
6370231.554 Everest 1830 (1967 Definition) mean
6370232.363 Everest (1830) mean
6370235.491 Everest 1830 Modified (1967) authalic
6370237.055 Everest 1830 Modified (1967) mean
6370283.158 Bessel (1841) volumetric
6370289.510 Bessel (1841) authalic
6370291.091 Bessel (1841) mean
6370453.310 Airy (1830) volumetric
6370459.655 Airy (1830) authalic
6370461.234 Airy (1830) mean
6370937.089 Clarke (1878) volumetric
6370943.691 Clarke (1878) authalic
6370945.333 Clarke (1878) mean
6370990.707 Clarke (1866) volumetric
6370990.707 NAD 27 (1927) volumetric
6370996.142 Clarke (1880) volumetric
6370997.241 Clarke (1866) authalic
6370997.241 NAD 27 (1927) authalic
6370998.859 WGS-72 (1972) volumetric
6370998.867 Clarke (1866) mean
6370998.867 NAD 27 (1927) mean
6370999.786 IERS (1989) volumetric
6371000.385 IERS (2003) volumetric
6371000.790 GRS-80 (1979) volumetric
6371000.790 WGS-84 (1984) volumetric
6371002.744 Clarke (1880) authalic
6371004.387 Clarke (1880) mean
6371005.250 WGS-72 (1972) authalic
6371006.177 IERS (1989) authalic
6371006.762 IERS (2003) authalic
6371006.840 WGS-72 (1972) mean
6371007.181 GRS-80 (1979) authalic
6371007.181 WGS-84 (1984) authalic
6371007.767 IERS (1989) mean
6371008.367 IERS (2003) mean
6371008.608 WGS66 (1966) volumetric
6371008.771 GRS-80 (1979) mean
6371008.771 WGS-84 (1984) mean
6371014.999 WGS66 (1966) authalic
6371016.590 WGS66 (1966) mean
6371021.085 New International (1967) volumetric
6371023.523 GRS-67 (1967) volumetric
6371023.591 South American (1969) volumetric
6371023.591 Australian National (1966) volumetric
6371027.476 New International (1967) authalic
6371029.067 New International (1967) mean
6371029.915 GRS-67 (1967) authalic
6371029.982 South American (1969) authalic
6371029.982 Australian National (1966) authalic
6371031.505 GRS-67 (1967) mean
6371031.573 South American (1969) mean
6371031.573 Australian National (1966) mean
6371064.744 Helmert (1906) volumetric
6371071.133 Helmert (1906) authalic
6371072.723 Helmert (1906) mean
6371109.694 Krassovsky (1940) volumetric
6371116.083 Krassovsky (1940) authalic
6371117.673 Krassovsky (1940) mean
6371221.266 Hayford (1910) volumetric
6371221.266 International (1924) volumetric
6371227.711 Hayford (1910) authalic
6371227.711 International (1924) authalic
6371229.315 Hayford (1910) mean
6371229.315 International (1924) mean
6376523.000 Plessis (1817) equatorial
6377298.556 Everest 1830 (1967 Definition) equatorial
6377299.365 Everest (1830) equatorial
6377304.063 Everest 1830 Modified (1967) equatorial
6377397.155 Bessel (1841) equatorial
6377563.396 Airy (1830) equatorial
6378135.000 WGS-72 (1972) equatorial
6378136.000 IERS (1989) equatorial
6378136.600 IERS (2003) equatorial
6378137.000 GRS-80 (1979) equatorial
6378137.000 WGS-84 (1984) equatorial
6378145.000 WGS66 (1966) equatorial
6378157.500 New International (1967) equatorial
6378160.000 GRS-67 (1967) equatorial
6378160.000 South American (1969) equatorial
6378160.000 Australian National (1966) equatorial
6378190.000 Clarke (1878) equatorial
6378200.000 Helmert (1906) equatorial
6378206.400 Clarke (1866) equatorial
6378206.400 NAD 27 (1927) equatorial
6378245.000 Krassovsky (1940) equatorial
6378249.145 Clarke (1880) equatorial
6378388.000 Hayford (1910) equatorial
6378388.000 International (1924) equatorial
6380564.1 Maupertuis (1738) rectifying
6386115.886 Maupertuis (1738) volumetric
6386131.541 Maupertuis (1738) authalic
6386135.428 Maupertuis (1738) mean
6397300.000 Maupertuis (1738) equatorial

The 15th ECMWF Workshop on High Performance Computing in Meteorology

draft programme published
http://www.ecmwf.int/newsevents/meetings/workshops/2012/high_performance_computing_15th/

Final report of IPET-DRC-IV released

IPET-DRC is an expert team in charge of maintenance of WMO Codes, such as
SYNOP, GRIB, or BUFR.
IPET-DRC held its fourth meeting in Exeter, UK from 21 to 25 May 2012; the
final report is now available from the WMO Web at:

http://www.wmo.int/pages/prog/www/ISS/Meetings/IPET-DRC_Exeter2012/Report/Report_IPETDRC-IV_Exeter2012.doc

http://www.wmo.int/pages/prog/www/ISS/Meetings/IPET-DRC_Exeter2012/IPET-DRC_DocPlan.html

[PS. pdf and GDoc versions are available here: http://redmine.toyoda-eizi.net/projects/wmocodes/wiki/IPET-DRC-IV_2012_Exeter ]

ISO19139 memo - // trick to select MD_DigitalTransferOptions scattered around a record

In current ISO 19139, there are three places to hold URL with authority:
distributionInfo/*/distributor/*/distributorTransferOptions/*/onLine/*/linkage
and distributionInfo/*/transferOptions/*/onLine/*/linkage.

(Darn long path!)

distributorTransferOptions exists for situation in which a dataset has more
than one distributor and each has different linkage.
In other words, it is enough to use transferOptions (a) if there is only one
distributor or (b) if all distributor share the same linkage.

It is painful to prepare for two possibilities. But there is a trick. We
can select //onLine/*/linkage to get entire set of URL's. There is no risk
of selecting MD_DigitalTransferOptions located anywhere because it is only
allowed at these two places.

paper on netCDF interface for atmospheric models

My involvement is really merginal though:

Ishiwatari et al: "Gtool5": a Fortran90 library of input/output interfaces
for self-descriptive multi-dimensional numerical data.
Geosci. Model Dev., 5, 449-455, 2012
http://www.geosci-model-dev.net/5/449/2012/
doi:10.5194/gmd-5-449-2012

new metadata for World Data Centre for Greenhouse Gases

The World Data Centre for Greenhouse Gases (WDCGG), operated by Japan
Meteorological Agency, has moved to new website
http://ds.data.jma.go.jp/gmd/wdcgg/ effective 29th February 2012. So we've
re-generated WIS version metadata to reflect new URL structure. Now 1570
records are available.

You can browse list of records at GISC Tokyo:
http://www.wis-jma.go.jp/meta/search.jsp?oai_provider=0&text=WDCGG

You can also use GISC's SRU interface. This example searches CO2 observation
in sub-sahara Africa:
http://www.wis-jma.go.jp/meta/sru.jsp?version=1.1&operation=searchRetrieve&query=CO2+and+geo.bounds+within%2Fpartial%2Fnwse+%2220+-20+-45+50%22

And the good news is we have fixed URL link in metadata records, so the
catalogue now links to real data for example
http://ds.data.jma.go.jp/gmd/wdcgg/cgi-bin/wdcgg/accessdata.cgi?select=inventory&index=NMB123S00-NOAA

Metadata entry from Cambodia

On 22nd March, Department of Meteorology (DOM) of Cambodia has submitted an
metadata record describing their GTS bulletin SMKP01 VDPP.

http://www.wis-jma.go.jp/meta/search.jsp?text=SMKP01VDPP

This is a great step!

take 2

this is a test of blog posting

metadata records for GRIB data at GISC Cache Tokyo

I'm now publishing metadata records for GRIB data in two-tier structure.

== Tier 1: Per-bulletin record

Example: http://www.wis-jma.go.jp/meta/search.jsp?text=HVXO10RJTD

The record has two links in "identification info". One links to actual
data:

Example: http://www.wis-jma.go.jp/data/browse?Indicator=RJTD&LinkText=HVXO10

Another links to metadata of a product which the bulletin belongs to.

Example:
http://www.wis-jma.go.jp/meta/search.jsp?text=GSM25X&keywords=GRIB+Aggregate

== Tier 2: Per-product record

Here I define "product" as a set of GRIB bulletins that share code-form,
model, resolution, and spatial area.

Example:
http://www.wis-jma.go.jp/meta/search.jsp?text=GSM25X&keywords=GRIB+Aggregate

This one links to actual data in the product.

Example:
http://www.wis-jma.go.jp/data/browse?Type=GRIB&Discipline=Global+Spectral+Model&Resolution=2.5+2.5&Area=90.0+0.0+0.0+357.5

WIS Sync monitor runs 4-hourly

Starting 12Z yesterday (2012-02-08T12Z), the monitor runs 4 hourly instead
of 3 hourly before. This is due to increase of harvesting/processing time
these days. Apologies for any inconvenience.

WIS Sync Monitor Updated again

The WIS metadata sync monitor http://toyoda-eizi.net/2011/syncmon/mon.html
had pretty much updated. The greatest beauty is that we serve each archived
file (not only zipfiles): you can save URLs in the RSS
http://toyoda-eizi.net/2011/syncmon/news.atom and click them later days!

I hope that won't cause too much load on my server.

Each report in RSS is issued in XHTML instead of Atom Entry XML which was
hard to handle in mobile devices. For example the "change since last
monitor" page becomes HTML like
http://toyoda-eizi.net/za/hist2012W06.zip/20120206T03/DIAG.html
and each link leads you to archived list of identifiers. That is useful in
maintaining sync network.

WIS sync monitor updated

The WIS metadata sync monitor http://toyoda-eizi.net/2011/syncmon/mon.html
has been updated. The diffs no longer indicate lists of identifiers, as
they became unrealistically huge these days. The numbers are indicated
instead.

(1) Snapshot Diffs fixed - it has long been broken
http://toyoda-eizi.net/2011/syncmon/curr/DIFF.atoment

(2) CSV version of Snapshot Diffs added
http://toyoda-eizi.net/2011/syncmon/curr/DIFF.csv

(3) Diag Diffs fixed - it has long been broken
http://toyoda-eizi.net/2011/syncmon/curr/DIAG.atoment

(4) CSV version of Diag Diffs added
http://toyoda-eizi.net/2011/syncmon/curr/DIAG.csv

There were also internal improvements to collect data more robustly. Even
though I've paid every efforts of ugly parallel programming, the growing
size and slowing performance of servers makes the process a bit slower --
the update will be visible about 80 minutes after every 3 hours.

If you have RSS reader, try following the feed:
http://toyoda-eizi.net/2011/syncmon/news.atom

WISTEST-* sets removed

The WISTEST-* OAI sets was made for presenation at CG-XVI. The sets
included OAI identifiers in operational namespace
"urn:x-wmo:md:int.wmo.wis::*" and hence caused trouble in loading the WMO
baseline metadata.

After the test sets removed, 2702 records of baseline MD was inserted into
WIS-UNASSOCIATED set. It reflects only onto the set WIS-CATALOGUE, and I
believe that won't affect synchronisation among CMA/DWD/JMA.