Harvesting issue - v3.14.5 not downloading metadata from DWD - error - cannot find namespace to set for element gmd:MD_Metadata about openwis4

[rogers492]

From @openwis-ss on April 18, 2016 13:58

Hi Dom,

Does issue #171 also fix this issue ?

Cheers,

Martin

[rogers492]

From @woollattd on April 18, 2016 15:4

Hi
Afraid it hasn't

[rogers492]

From @openwis-ss on April 18, 2016 15:58

Ok no worries, thought it looked related

[rogers492]

From @woollattd on April 21, 2016 7:57

updating title and first post to be more informative
as this occurs in v3.14.5 as well.

[rogers492]

From @woollattd on May 13, 2016 8:19

Email from Birgit at DWD

Dear Dominic,

on the TT-Gisc Melbourne conference our collegues learned that the LGAT issue is caused by a software problem in openwis, which seams not to be easy to get a fix of.

Assumed trigger is the part in the header: xmlns=http://www.openarchives.org/OAI/2.0/, which no other record has.

We would kindly ask you to test on your (test-)system with this part removed.

We would like to get rid of these inconsistencies if possible.
By the way, the record validates perfectly against XMLSpy as it is.

Best regards
Birgit

[rogers492]

From @woollattd on May 13, 2016 8:35

I now have access to test harvesting on 3.14.5 at the Met Office and have found it is missing 60 files - all of them LGAT.
DWD are asking to have this fixed.
Here's the list of files we are missing from the DWD site http://oai.dwd.de/oaimonitorgui/#complete

urn:x-wmo:md:int.wmo.wis::CSGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::CUGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::FAGR31LGAT;0;
urn:x-wmo:md:int.wmo.wis::FAGR32LGAT;0;
urn:x-wmo:md:int.wmo.wis::FAGR33LGAT;0;
urn:x-wmo:md:int.wmo.wis::FAGR34LGAT;0;
urn:x-wmo:md:int.wmo.wis::FCGR31LGAT;0;
urn:x-wmo:md:int.wmo.wis::FCGR32LGAT;0;
urn:x-wmo:md:int.wmo.wis::FCGR33LGAT;0;
urn:x-wmo:md:int.wmo.wis::FPGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::FQME22LGAT;0;
urn:x-wmo:md:int.wmo.wis::FQME24LGAT;0;
urn:x-wmo:md:int.wmo.wis::FQME26LGAT;0;
urn:x-wmo:md:int.wmo.wis::FQME28LGAT;0;
urn:x-wmo:md:int.wmo.wis::FQME50LGAT;0;
urn:x-wmo:md:int.wmo.wis::FQME51LGAT;0;
urn:x-wmo:md:int.wmo.wis::FQME52LGAT;0;
urn:x-wmo:md:int.wmo.wis::FTGR31LGAT;0;
urn:x-wmo:md:int.wmo.wis::FTGR32LGAT;0;
urn:x-wmo:md:int.wmo.wis::IOBD01LGAT;0;
urn:x-wmo:md:int.wmo.wis::ISCD01LGAT;0;
urn:x-wmo:md:int.wmo.wis::ISID20LGAT;0;
urn:x-wmo:md:int.wmo.wis::ISID21LGAT;0;
urn:x-wmo:md:int.wmo.wis::ISID22LGAT;0;
urn:x-wmo:md:int.wmo.wis::ISMD01LGAT;0;
urn:x-wmo:md:int.wmo.wis::ISMD20LGAT;0;
urn:x-wmo:md:int.wmo.wis::ISMD21LGAT;0;
urn:x-wmo:md:int.wmo.wis::IUKD01LGAT;0;
urn:x-wmo:md:int.wmo.wis::IUKD02LGAT;0;
urn:x-wmo:md:int.wmo.wis::IUKD03LGAT;0;
urn:x-wmo:md:int.wmo.wis::IUSD01LGAT;0;
urn:x-wmo:md:int.wmo.wis::IUSD02LGAT;0;
urn:x-wmo:md:int.wmo.wis::IUSD03LGAT;0;
urn:x-wmo:md:int.wmo.wis::IUUD01LGAT;0;
urn:x-wmo:md:int.wmo.wis::SAGR31LGAT;0;
urn:x-wmo:md:int.wmo.wis::SAGR32LGAT;0;
urn:x-wmo:md:int.wmo.wis::SAGR33LGAT;0;
urn:x-wmo:md:int.wmo.wis::SIGR20LGAT;0;
urn:x-wmo:md:int.wmo.wis::SIGR21LGAT;0;
urn:x-wmo:md:int.wmo.wis::SIGR22LGAT;0;
urn:x-wmo:md:int.wmo.wis::SIGR55LGAT;0;
urn:x-wmo:md:int.wmo.wis::SIGR56LGAT;0;
urn:x-wmo:md:int.wmo.wis::SIVF20LGAT;0;
urn:x-wmo:md:int.wmo.wis::SMGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::SMGR20LGAT;0;
urn:x-wmo:md:int.wmo.wis::SMGR21LGAT;0;
urn:x-wmo:md:int.wmo.wis::SMGR55LGAT;0;
urn:x-wmo:md:int.wmo.wis::SMGR56LGAT;0;
urn:x-wmo:md:int.wmo.wis::SMVF01LGAT;0;
urn:x-wmo:md:int.wmo.wis::SNVF01LGAT;0;
urn:x-wmo:md:int.wmo.wis::UEGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::UGGR20LGAT;0;
urn:x-wmo:md:int.wmo.wis::UKGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::ULGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::UPGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::USGR01LGAT;0;
urn:x-wmo:md:int.wmo.wis::WEME40LGAT;0;
urn:x-wmo:md:int.wmo.wis::WEME42LGAT;0;
urn:x-wmo:md:int.wmo.wis::WSGR31LGAT;0;
urn:x-wmo:md:int.wmo.wis::WWME22LGAT;0;

[rogers492]

From @woollattd on May 13, 2016 8:41

@leon
are you able to take a look? And does this happen on your harvest from DWD too?

[rogers492]

From @lmika-bom on May 16, 2016 3:58

Hi Dom. I've asked @ywang-bom if he can take a look at this.

[rogers492]

From @woollattd on May 16, 2016 7:57

Fab
Thanks

[rogers492]

From @ywang-bom on May 20, 2016 4:33

This issue is caused by the default namespace (http://www.openarchives.org/OAI/2.0/) used in the incoming metadata file. OpenWIS tries to honor ISO19139 recommendation of not using default namespace in metadata XML file. So it tries to fill in the namespace prefix by looking it up using the URI. But the lookup is only done in ISO19139 namespaces, which does not contain any OAI-PMH namespaces. So when the lookup fails, openwis errors out and refuse to ingest the metadata.

The fix (pull request #185) is to allow default namespace if it is not part of ISO19139, i.e. OpenWIS will now only ensure all ISO19139 namespaces are explicitly declared instead of ensuring everything.

[rogers492]

From @lmika-bom on February 21, 2017 3:25

This has been fixed for v3 but check for v4.

[rogers492]

Development of OpenWISv4 is stopped.