2011-12-27

fixed: size of OAI set displayed incorrectly

In the WIS community there is a convention that the size of OAI set is
included in ListSet response.

http://www.wis-jma.go.jp/meta/oaiprovider.jsp?verb=ListSets

Until last week there was an problem that the size of WIS-GISC-TOKYO is not
displayed correctly. It is now fixed and it returns reliable number.

The documentation at http://www.wis-jma.go.jp/cms/metadata/oai-pmh/ uses the
ListSets internally, and hence fixed accordingly.

2011-12-15

OAI Sync Monitor update

Recent updates of OAI Sync Monitor
http://toyoda-eizi.net/2011/syncmon/mon.html

- Beijing joined and the set WIS-GISC-BEIJING introduced
- Temporary set WIS-UNASSOCIATED introduced
- Testing set WISTEST-* no longer monitored.

2011-12-13

metadata updates

- GISC Tokyo has loaded GTS baseline metadata for VBRR, VNNN, VTBB, VLIV,
VMMC, RPMM, VHHH.
- Set WIS-WIMMS created.
- most of WISTEST-JMA set has been deleted; right now deletion in terms of
OAI so other synching centres may harvest that del-notification to react.
Four records are kept just for testing.

WIS Sync Monitor revived

My monitor has been refreshed. It has now updated list of servers and sets,
and there is a bit nicer top page with minimal description.

http://toyoda-eizi.net/2011/syncmon/mon.html

2011-12-06

all records in WIS-GISC-TOKYO confirmed to have non-empty dateStamp

Hi,

I've run a test program to check all records in WIS-GISC-TOKYO set. This
time no record has gco:nilReason="missing".

Best,
Eizi

WDCGG metadata updated tentatively

Hi,

I've uploaded 1653 records of DAR metadata for WDCGG. There was problem in
converter from GAWSIS format to ISO 19139, which generated <gmd:dateStamp
gco:nilReason="missing"/> for most of records. Now that should be fixed.

This is only a quick fix: the metadata is all tested to be compliant to ISO
19139 schema, but there should be considerable difference of "style" from
recent agreements in IPET-MDI (and hence the latest version of MF baseline
metadata).

--
Best,
Eizi

2011-11-28

resourceFormat and distributionFormat

Now I'm writing schematron to examine how elements of ISO 19115 is used in
actual WIS Initial MD Set.

Today I had to write following schematron.

<pattern name="fmts-eqv">
<!-- activates only if there is resourceFormat -->
<rule context="gmd:resourceFormat">
<assert test="(*/gmd:name/*/text()
=
/*/gmd:distributionInfo/*/gmd:distributionFormat/*/gmd:name/*/text())
or ('BUFR'
=
/*/gmd:distributionInfo/*/gmd:distributionFormat/*/gmd:name/*/text())
or ('CREX'
=
/*/gmd:distributionInfo/*/gmd:distributionFormat/*/gmd:name/*/text())
">resourceFormat/name and distributionFormat/name <!--
-->must be equal, or the latter be BUFR/CREX</assert>
</rule>
</pattern>

Please tolerate I had to write three similar comparisons since libxml2
doesn't support match() function.

The point is resourceFormat/*/name does not necessarily equal to
distributionFormat/*/name. Surprising, isn't it?

That takes place for TDCF (i.e. BUFR and CREX) messages converted from
alphanumeric data. This is very interesting and (so far) practical
approach. But there's also limitation. I'm curious what to do if TDCF is
directly created at the data source.

2011-04-25

a patch for GNOME libxml2 to work with GML XML Schema

Caution: this only suppresses error abortion. I didn't read in depth
and can't guarantee the consequences.

--- xmlschemas.c.orig 2011-04-24 14:58:16.000000000 +0000
+++ xmlschemas.c 2011-04-24 15:47:50.000000000 +0000
@@ -15158,7 +15158,11 @@
}
if ( (WXS_IS_LIST(type) || WXS_IS_UNION(type)) &&
(WXS_IS_RESTRICTION(type) == 0) &&
- (! WXS_IS_ANY_SIMPLE_TYPE(baseType))) {
+ (
+ (! WXS_IS_ANY_SIMPLE_TYPE(baseType))
+ && (baseType->type != XML_SCHEMA_TYPE_SIMPLE)
+ )
+ ) {
xmlSchemaPCustomErr(ctxt,
XML_SCHEMAP_ST_PROPS_CORRECT_1,
WXS_BASIC_CAST type, NULL,

2011-04-07

What elements/types in GML are used in ISO 19139?

I did a survey.

= simple references
== reference to attributegroup in GML
ref gml:AggregationAttributeGroup

== reference to elements in GML
ref gml:AbstractCoordinateOperation
ref gml:AbstractCoordinateSystem
ref gml:AbstractCRS
ref gml:AbstractDatum
ref gml:AbstractGeneralOperationParameter
ref gml:AbstractGeometry
ref gml:AbstractTimePrimitive
ref gml:BaseUnit
ref gml:ConventionalUnit
ref gml:CoordinateSystemAxis
ref gml:DerivedUnit
ref gml:Ellipsoid
ref gml:OperationMethod
ref gml:Point
ref gml:pos
ref gml:PrimeMeridian
ref gml:UnitDefinition

= type reference
== attributes using GML type
type gml:NilReasonType

== elements using GML type
type gml:AngleType
type gml:CodeType
type gml:LengthType
type gml:MeasureType
type gml:ScaleType

== types extending GML's one
base gml:AbstractCoordinateOperationType
base gml:AbstractCRSType
base gml:AbstractDatumType
base gml:AffineCSType
base gml:BaseUnitType
base gml:CartesianCSType
base gml:CompoundCRSType
base gml:ConcatenatedOperationType
base gml:ConventionalUnitType
base gml:ConversionType
base gml:CoordinateSystemAxisType
base gml:CylindricalCSType
base gml:DefinitionType
base gml:DerivedCRSType
base gml:DerivedUnitType
base gml:DictionaryType
base gml:EllipsoidalCSType
base gml:EllipsoidType
base gml:EngineeringCRSType
base gml:EngineeringDatumType
base gml:GeocentricCRSType
base gml:GeodeticCRSType
base gml:GeodeticDatumType
base gml:GeographicCRSType
base gml:IdentifiedObjectType
base gml:ImageCRSType
base gml:ImageDatumType
base gml:LinearCSType
base gml:OperationMethodType
base gml:OperationParameterGroupType
base gml:OperationParameterType
base gml:PassThroughOperationType
base gml:PolarCSType
base gml:PrimeMeridianType
base gml:ProjectedCRSType
base gml:SphericalCSType
base gml:TemporalCRSType
base gml:TemporalDatumType
base gml:TimeCSType
base gml:TransformationType
base gml:UnitDefinitionType
base gml:UserDefinedCSType
base gml:VerticalCRSType
base gml:VerticalCSType
base gml:VerticalDatumType

== elements of types that extend GML's type
substitutionGroup gml:AffineCS
substitutionGroup gml:BaseUnit
substitutionGroup gml:CartesianCS
substitutionGroup gml:CompoundCRS
substitutionGroup gml:ConcatenatedOperation
substitutionGroup gml:ConventionalUnit
substitutionGroup gml:Conversion
substitutionGroup gml:CoordinateSystemAxis
substitutionGroup gml:CylindricalCS
substitutionGroup gml:Definition
substitutionGroup gml:DerivedCRS
substitutionGroup gml:DerivedUnit
substitutionGroup gml:Dictionary
substitutionGroup gml:Ellipsoid
substitutionGroup gml:EllipsoidalCS
substitutionGroup gml:EngineeringCRS
substitutionGroup gml:EngineeringDatum
substitutionGroup gml:GeocentricCRS
substitutionGroup gml:GeodeticCRS
substitutionGroup gml:GeodeticDatum
substitutionGroup gml:GeographicCRS
substitutionGroup gml:ImageCRS
substitutionGroup gml:ImageDatum
substitutionGroup gml:LinearCS
substitutionGroup gml:OperationMethod
substitutionGroup gml:OperationParameter
substitutionGroup gml:OperationParameterGroup
substitutionGroup gml:PassThroughOperation
substitutionGroup gml:PolarCS
substitutionGroup gml:PrimeMeridian
substitutionGroup gml:ProjectedCRS
substitutionGroup gml:SphericalCS
substitutionGroup gml:TemporalCRS
substitutionGroup gml:TemporalDatum
substitutionGroup gml:TimeCS
substitutionGroup gml:Transformation
substitutionGroup gml:UnitDefinition
substitutionGroup gml:UserDefinedCS
substitutionGroup gml:VerticalCRS
substitutionGroup gml:VerticalCS
substitutionGroup gml:VerticalDatum

2011-04-05

structural difference between GML 3.2.0 and 3.2.1

The ISO version of XML schema for Geographical Metadata ISO 19139, available
at http://www.isotc211.org/2005/gmx/gmx.xsd, imports so-called version 3.2.0
of GML which was draft ISO 19136. The text of ISO 19139 cites ISO 19136,
which was later published as version 3.2.1.

The two GML versions are approximately same, except for following:

* Namespace "http://www.opengis.net/gml" changed to
http://www.opengis.net/gml/3.2

* annotation/appinfo/@source changed from
"urn:ogc:specification:gml:schema-xsd:coordinateReferenceSystems:3.2.0" to
"urn:x-ogc:specification:gml:schema-xsd:coordinateReferenceSystems:3.2.1"
(from "ogc" to "x-ogc" and from "3.2.0" to "3.2.1"),

* removed deprecated elements: gml:Null, gml:coordinateOperationRef,
gml:singleOperationRef, gml:AbstractOperation, gml:operationRef,
gml:generalConversionRef, gml:generalTransformationRef,
gml:usesSingleOperation, gml:concatenatedOperationRef, gml:usesOperation,
gml:passThroughOperationRef, gml:usesMethod, gml:usesValue,
gml:conversionRef, gml:transformationRef, gml:dmsAngleValue,
gml:valueOfParameter, gml:includesValue, gml:valuesOfGroup,

* removed deprecated types: gml:OperationPropertyType,

* changed complexType gml:OperationMethodType so that it (1) may choose its
first child element gml:formulaCitation instead of formerly mandated
gml:formula, and (2) has optional gml:parameter instead of
gml:generalOperationParameter which is now banned,

* added element gml:formulaCitation, gml:parameter,

* deprecated elements: gml:generalOperationParameter, gml:usesParameter,

* changed conplexType gml:OperationParameterGroupType so that it has
optional elements gml:parameter instead of gml:generalOperationParameter
which is now banned,

* changed complexType EngineeringCRS may choose its first child element from
affineCS, cartesianCS, cylindricalCS, linearCS, polarCS, sphericalCS, or
userDefinedCS, instead of now-deprecated coordinateSystem,

* changed complexType CoordinateSystemAxisType so that its attribute gml:uom
has now type gml:UomIdentifier instead of anyURI,

* changed type of element gml:rangeParameters from gml:RangeParametersType
to AssociationRoleType,

* changed complexType FileType so that its first child gml:fileName is
deprecated and gml:fileReferece is only recommended,

I'm afraid the list is incomplete but I found it is counter-productive to
complete it. Above are only one third of full diff(1) output

2011-02-15

Examples of GML fragment in ISO 19115/19139 gmd:verticalElement

Next one is vertical.

Meteorology is really indifferent about the concept of CRS, so there is
bunch of altitude data without reference system specification. Luckily there
is EPSG:5714 registered as "msl height" so we can express it in globally
understood framework.

<gmd:verticalElement>
<gmd:EX_VerticalExtent id="boundingVerticalExtent">
<gmd:minimumValue>
<gco:Real>7.7</gco:Real>
</gmd:minimumValue>
<gmd:maximumValue>
<gco:Real>7.7</gco:Real>
</gmd:maximumValue>
<gmd:verticalCRS>
<gml:VerticalCRS gml:id="crs.msl_height">
<!-- EPSG:5714 simply means "msl height" -->
<gml:identifier codeSpace="urn:ogc:def:crs:EPSG::">5714</gml:identifier>
<!-- instruction in ISO 19136 12.2.3.4 -->
<gml:scope>not known</gml:scope>
<gml:verticalCS/>
<gml:verticalDatum />
</gml:VerticalCRS>
</gmd:verticalCRS>
</gmd:EX_VerticalExtent>
</gmd:verticalElement>

What if we use pressure coordinate or height above surface? I don't know.
Maybe WMO has to do something. It's just a creating couple of URI and code,
so it should be easy if people understands the way.

Examples of GML fragment in ISO 19115/19139 gmd:temporalElement

Sometimes ISO 19139 uses GML fragments to describe complex structure. But
it is too simply documented for novice to understand how to write it. I'm a
novice too, but unfortunately had to develop a guideline.....

One of the most often-used GML in 19139 should be temporal reference in
extent information. The simplest one should be an instance (single point on
a time axis):

<gmd:EX_TemporalExtent id="time_instance">
<gmd:extent>
<gml:TimeInstant gml:id="instant">
<gml:timePosition>1992-01-01T00:00:00Z</gml:timePosition>
</gml:TimeInstant>
</gmd:extent>
</gmd:EX_TemporalExtent>
</gmd:temporalElement>

Second most useful should be a time range:

<gmd:temporalElement>
<gmd:EX_TemporalExtent id="boundingTemporalExtent">
<gmd:extent>
<gml:TimePeriod gml:id="validTimeRange">
<gml:beginPosition>1992-01-01T00:00:00Z</gml:beginPosition>
<gml:endPosition>2007-12-31T00:00:00Z</gml:endPosition>
</gml:TimePeriod>
</gmd:extent>
</gmd:EX_TemporalExtent>
</gmd:temporalElement>

Really easy, isn't it?

The time in timePosition/beginPosition/endPosition can be one of:

- xs:date
- xs:gYearMonth
- xs:gYear
- xs:time
- xs:dateTime
- xs:anyURI
- xs:decimal

Decimal is intended for numeric time, for example GPS time in seconds. URI
is intended for indicating commonly-shared era, which I don't know how to
use.

<gml:endPosition
frame="tag:toyoda.eizi@gmail.com:2011-02-12:mytime_in_seconds">23131231231.1231</gml:endPosition>

But I believe it is more explicit to use gml:timeInterval.

<gml:TimePeriod gml:id="validTimeRange3">
<gml:beginPosition>1992-01-01T00:00:00Z</gml:beginPosition>
<gml:end/>
<gml:timeInterval unit="second">32.75232143</gml:timeInterval>
</gml:TimePeriod>

Complex calendar duration in ISO format can be shown in gml:duration.

<gml:TimePeriod gml:id="validTimeRange4">
<gml:beginPosition>1992-01-01T00:00:00Z</gml:beginPosition>
<gml:end/>
<gml:duration>P2Y2M3D</gml:duration>

</gml:TimePeriod>

The 2nd example can be verbose. I don't know benefit of this form, except
each of "begin" and "end" has id attribute.

<gmd:temporalElement>
<gmd:EX_TemporalExtent id="boundingTemporalExtent">
<gmd:extent>
<gml:TimePeriod gml:id="validTimeRange">
<gml:begin>
<gml:TimeInstant gml:id="begdate">
<gml:timePosition>1992-01-01T00:00:00Z</gml:timePosition>
</gml:TimeInstant>
</gml:begin>
<gml:end>
<gml:TimeInstant gml:id="enddate">
<gml:timePosition>2007-12-31T00:00:00Z</gml:timePosition>
</gml:TimeInstant>
</gml:end>
</gml:TimePeriod>
</gmd:extent>
</gmd:EX_TemporalExtent>
</gmd:temporalElement>

something like strcmp(3) in XSLT 1.0

I had a need to compare alphanumeric string. Unfortunately XSLT 1.0 does
not have string comparison, nor even function to extract character code from
character. So following example works only for string with letters and
numbers. All other characters are treated equally, just after "z" and "_"
of the dictionary order.

<xsl:template name="strcmpAN">
<xsl:param name="a"/>
<xsl:param name="b"/>
<xsl:variable name="ia">
<xsl:call-template name="charCodeAN">
<xsl:with-param name="s" select="$a"/>
</xsl:call-template>
</xsl:variable>
<xsl:variable name="ib">
<xsl:call-template name="charCodeAN">
<xsl:with-param name="s" select="$b"/>
</xsl:call-template>
</xsl:variable>
<xsl:choose>
<xsl:when test="$a = $b">
<xsl:value-of select="number(0)"/>
</xsl:when>
<xsl:when test="string-length($b) = 0">
<xsl:value-of select="number(1)"/>
</xsl:when>
<xsl:when test="string-length($a) = 0">
<xsl:value-of select="number(-1)"/>
</xsl:when>
<xsl:when test="$ia &gt; $ib">
<xsl:value-of select="number(1)"/>
</xsl:when>
<xsl:when test="$ia &lt; $ib">
<xsl:value-of select="number(-1)"/>
</xsl:when>
<xsl:otherwise>
<xsl:call-template name="strcmpAN">
<xsl:with-param name="a" select="substring($a, 2)"/>
<xsl:with-param name="b" select="substring($b, 2)"/>
</xsl:call-template>
</xsl:otherwise>
</xsl:choose>
</xsl:template>

<xsl:template name="charCodeAN">
<xsl:param name="s"/>
<xsl:param name="i" select="1"/>
<xsl:variable name="CHARTABLE"
select="'
0123456789AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYyZz_'"/>
<xsl:choose>
<xsl:when test="$i &gt; string-length($CHARTABLE)">
<xsl:value-of select="string-length($CHARTABLE) + 1"/>
</xsl:when>
<xsl:when test="substring($CHARTABLE, $i, 1) = substring($s, 1, 1)">
<xsl:value-of select="$i"/>
</xsl:when>
<xsl:otherwise>
<xsl:call-template name="charCodeAN">
<xsl:with-param name="s" select="$s"/>
<xsl:with-param name="i" select="$i + 1"/>
</xsl:call-template>
</xsl:otherwise>
</xsl:choose>
</xsl:template>

2011-01-26

Reverse-engineered RelaxNG schema for GAW XML Catalogue

GAW (Global Atmospheric Watch) has its legacy catalogue system. I called it
legacy, but I don't mean it is bad or inferior. Rather I'm interested in
the fact that there already exists an operating network. Why the WIS don't
make use of it to achieve efficient implementation of catalogue? So I've
got an XML sample data from WDCGG and analysed its structure.
grammar {
start =
element wdcgg {
element station {
element station_name { xsd:token { maxLength = "64" } },
element id {
xsd:Name { length = "9" pattern = "\s*\w\w\w\d\d\d\w\d\d\s*" }
},
element latitude {
xsd:float { maxInclusive = "90" minInclusive = "-90" }
},
element longitude {
xsd:float { maxInclusive = "180" minInclusive = "-180" }
},
element altitude {
xsd:float { maxInclusive = "5079" minInclusive = "0" }
},
element wmo_region { WMORegion },
element gaw_category { GawCategory },
element country { xsd:token },
element organization { xsd:token },
element description { xsd:string },
element parameter {
element parameter_name {
xsd:NMTOKEN { maxLength = "7" pattern = "\s*[0-9A-Za-z]+\s*" }
},
element status_of_report { xsd:string },
element start_of_data { DateOrEmpty },
element end_of_data { DateOrEmpty },
element last_update { DateOrEmpty },
element contributor {
element organization { xsd:token },
element country { xsd:token }
}+,
element contact_person {
element name { xsd:token },
element organization { xsd:token },
element phone { xsd:token },
element fax { xsd:token },
element email { xsd:token }
}*
}*
}*
}
DateOrEmpty =
empty
| xsd:date { pattern = "\s*[0-9]{4}-[0-9]{2}-[0-9]{2}\s*" }
WMORegion =
"REGION I (Africa)"
| "REGION II (Asia)"
| "REGION III (South America)"
| "REGION IV (North and Central America)"
| "REGION V (South-West Pacific)"
| "REGION VI (Europe)"
| "ANTARCTICA"
GawCategory =
empty
| "Regional"
| "Global"
| "Contributing"
| "Non-GAW (international)"
}