metadata101 / iso19139.ca.hnap Goto Github PK
View Code? Open in Web Editor NEWISO Harmonized North American Profile (HNAP)
License: GNU General Public License v2.0
ISO Harmonized North American Profile (HNAP)
License: GNU General Public License v2.0
I'm not sure where the formatters are used but they currently don't seem to be working.
Did a local build, went to metadata and template -> formatters and attempted to use the HNAP formatters and they all seem to fail with the loaded HNAP templates/sample data (b16db0fe-a4ec-4844-819e-630e33137b6d).
<apiError>
<code>runtime_exception</code>
<description>Failed to compile stylesheet. 20 errors detected.</description>
<message>TransformerConfigurationException</message>
</apiError>
The following errors was logged to the console
Error at xsl:call-template on line 97 column 45 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 114 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 127 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 140 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 153 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 166 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 179 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 192 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 204 column 45 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 257 column 47 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 394 column 48 of view.xsl:
XTSE0650: No template exists named parentCollection
Error at xsl:call-template on line 402 column 47 of view.xsl:
XTSE0650: No template exists named relatedDatasets
Error at xsl:call-template on line 431 column 49 of view.xsl:
XTSE0650: No template exists named thumbnail
Error at xsl:call-template on line 469 column 43 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 524 column 45 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 538 column 45 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 552 column 45 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 569 column 45 of view.xsl:
XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 590 column 57 of view.xsl:
XTSE0650: No template exists named showMetadataExportIcons
Error at xsl:call-template on line 597 column 43 of view.xsl:
XTSE0650: No template exists named showPanel
This page contains the following errors:
error on line 13 at column 78: Entity 'nbsp' not defined
Below is a rendering of the page up to the first error.
Was the following change to the NAP specification done for ECCC or FGP? Is this an HNAP requirement? Can this be done via the schematron. It seems odd to modify the NAP specification for one change? Just wondering if this should be reverted?
Previously value was minOccurs="0"
from NAP specification
I the sample "Harbour Porpoise Presence, Maritimes Region" is loaded. Then going to the view the sample record and attempting to "add to map" will produced the following error.
Biota002_MarineMammals_HarbourPorpoise_PH_SJ_Presence
The WMS getCapabilities request failed, Failed to parse GetCapabilities operation response. Try again later or contact service provider.
I believe a lot of the published Open Data records also have similar issues. So I don't believe it is just a matter of fixing the sample records. May need a goenetwork core modification?
Any suggestions?
Replace "bold" class name with WET standard "h6" class name to enable bold text in html template
Ref: https://wet-boew.github.io/wet-boew-styleguide/design/sizing-en.html
Before:
<div class="col-md-6">
<span class="bold">Geographic Extent</span>
<br>
<span itemprop="spatial">SW: -141 41.68<br>NE: -52.62 83.11</span>
</div>
<div class="col-md-6">
<span class="bold">Time Period</span>
<br>
From:
<span itemprop="temporal">2001</span>
<br>
To:
<span itemprop="temporal">2006</span>
</div>
After:
<div class="col-md-6">
<span class="h6">Geographic Extent</span>
<br>
<span itemprop="spatial">SW: -141 41.68<br>NE: -52.62 83.11</span>
</div>
<div class="col-md-6">
<span class="h6">Time Period</span>
<br>
From:
<span itemprop="temporal">2001</span>
<br>
To:
<span itemprop="temporal">2006</span>
</div>
Lines of code to change are in line 396 and 410:
/iso19139.nap/src/main/plugin/iso19139.nap/formatter/ec-view/view.xsl
If I upload the following file and go to the viewer (taken from FGP/Open Data) and go to the viewer
The English Title looks correct but the French title is incorrect. It also seems like it is a json array with duplicate values.
["Cartographie des débarquements de la pêche côtière du homard et de l'effort de pêche sur un quadrillage statistique de la région des Maritimes (2012 à 2014)","Cartographie des débarquements de la pêche côtière du homard et de l'effort de pêche sur un quadrillage statistique de la région des Maritimes (2012 à 2014)"]
I have attempted to debug this with no luck.
I included a sample of the xml file that was uploaded.
64f741d7-1129-49dd-9e5c-2b1de79024f0.zip
The record also exist on open data.
https://open.canada.ca/data/en/dataset/64f741d7-1129-49dd-9e5c-2b1de79024f0
This is affecting the 3.8 and 3.10 branch - did not test 3.6
The config-editor.xml configuration references several angular components that are not inlcuded by geonetwork-core:
data-gn-country-selector-ec
data-gn-organisation-entry-selector-ec
These need to be removed and / or replaced for the editor to function.
HNAP plugin currently does not support multilingual gmd:filename. It seems to be based on the older ECCC/FGP code.
We need to be able to upload multilingual thumbnails and other resources.
Here is what seems to be produced based on the samples.
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Ricker_Beam_Trawl_Transects_Map.JPG</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription>
<gco:CharacterString>large_thumbnail</gco:CharacterString>
</gmd:fileDescription>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Ricker_Beam_Trawl_Transects_Map.JPG</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription>
<gco:CharacterString>large_thumbnail_fre</gco:CharacterString>
</gmd:fileDescription>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
Problem
gmd:fileDescription is not a real description of the resource. Instead it is a code. Of which the code are very specific to ECCC/FGP.
There is no linkage between the first and second resource even though it is the same data in both French and English
Much more difficult to validate that both English and French exists.
Here is the changes that is suggested
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Ricker_Beam_Trawl_Transects_Map.JPG</gco:CharacterString>
<gmd:PT_FreeText>
<gmd:textGroup>
<gmd:LocalisedCharacterString xmlns="" locale="#fra">Ricker_Beam_Trawl_Transects_Map.JPG</gmd:LocalisedCharacterString>
</gmd:textGroup>
</gmd:PT_FreeText>
</gmd:fileName>
<gmd:fileDescription>
<gco:CharacterString>Resource description</gco:CharacterString>
<gmd:PT_FreeText>
<gmd:textGroup>
<gmd:LocalisedCharacterString xmlns="" locale="#fra">Description de ressource</gmd:LocalisedCharacterString>
</gmd:textGroup>
</gmd:PT_FreeText>
</gmd:fileDescription>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
If anyone has any extra information to supply in regards to this issue then please add some comments.
Extent Description field is and option HNAP field. If a value is entered from the advanced view, the value does not show up in the Basic View
As this is part of the HNAP fields, it should display in the basic view.
@ianwallen you recently added a snippet to add-schema.sh
to help address this, it seems to no longer be working:
[ERROR] testExecListOne(org.fao.geonet.services.metadata.schema.SchematronCriteriaGroupServiceIntegrationTest) Time elapsed: 0.001 s <<< ERROR!
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 10 in XML document from URL [jar:file:/Users/jgarnett/.m2/repository/org/geonetwork-opensource/schema-iso19139.ca.HNAP/3.11.0-SNAPSHOT/schema-iso19139.ca.HNAP-3.11.0-SNAPSHOT.jar!/config-spring-geonetwork.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 10; columnNumber: 75; cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'context:component-scan'.
Caused by: org.xml.sax.SAXParseException: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'context:component-scan'.
Original comments bellow from Jose:
Related to duplicated gmd:supplementalInformation, I found the problem, the element cardinality is 0:1, but can be multiple napec:EC_CorporateInfo/napec:EC_Project elements
An option can be to add all in the same element i different lines, we add now:
gco:CharacterString
Project; PROJECT TEXT_1
</gco:CharacterString>
Can be added like:
gco:CharacterString
Project; PROJECT TEXT_1
Project; PROJECT TEXT_2
</gco:CharacterString>
Publication of new drafts:
Need re-confirmation – if an editor submits metadata for review, the CR can approve the metadata OR publish (which approves the metadata by default) correct? Or, if the metadata is submitted, the CR must approve THEN publish - separate actions?
Review submission/publish of working copies: problem.
As an editor;
As Content Reviewer;
2020-09-05: in testing this afternoon, I've submitted a metadata record "Publication in GeoCat Live - Test 7" (48fcb49e-993f-4eef-bb55-f4c65a50c191) as an editor successfully but the metadata fails to appear for approval/publish in the CR account. Can't account for why this would happen. Workspaces are a match.
for the online resource section of the editor the following behaviour is observed:
If you type the resource name info in English and/or French then choose or modify what was selected for protocol, the resource name information disappears and must be reentered. Is this the expected behaviour? Creates more work for user if they selected the wrong protocol to start.
@davidblasby comment - "It's a issue with the language management in the dialog, HNAP uses fra for french and seem not well managed."
I need some help in creating an xslt or XQuery that updates all gdm:voice to be bilingual. is there an existing sample maybe in this repo that could help me?
If not null Change from:
<gmd:voice>
<gco:CharacterString>819-997-2800</gco:CharacterString>
</gmd:voice>
Changed to:
<gmd:voice xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>819-997-2800</gco:CharacterString>
<gmd:PT_FreeText>
<gmd:textGroup>
<gmd:LocalisedCharacterString locale="#fra">819-997-2800</gmd:LocalisedCharacterString>
</gmd:textGroup>
</gmd:PT_FreeText>
</gmd:voice>
The initial schema plugin is arriving as #1 against branch 3.8.x.
We should decide which active branches to maintain:
I would like to understand the process needed to review and edit French and English update help tool. Where do are tips managed within the repository?
The readme.md does not seem to have the correct ./add-schema.sh command for all branches.
May want to review this to see if it is correct.
There is an odd inconsistency preventing validating:
The check in schematrong-rules-common.sch
for:
<sch:let name="coreSubjectThesaurusExists"
value="count(gmd:descriptiveKeywords[*/gmd:thesaurusName/*/gmd:title/*/text() = 'Government of Canada Core Subject Thesaurus' or
*/gmd:thesaurusName/*/gmd:title/*/text() = 'Government of Canada Core Subject Thesaurus']) > 0" />
Does not match with GC_Core_Subject.rdf
has the following title:
<skos:ConceptScheme rdf:about="http://www.thesaurus.gc.ca/#CoreSubjectThesaurus">
<dc:title>GC Core Subject</dc:title>
Forgive me if I word this wrong, I am pulling on a piece of string to describe a challenge.
The HNAP schema also has some keywords thesauri built-in
The HNAP schema (core-geonetwork) has a workflow for adding additional keyword thesauri:
Use the admin menu to upload a thesauri
Define a template that references the new thesauri
Any metadata record created from that template can access and use the keywords defined
I would like to discuss how we can manage additional optional thesauri in the plugin, perhaps as a build profile, in order to manage these under version control. Would such a proposal be welcome?
Security note currently contain the following options
The data is also stored in a free form text field and uses a helper for a suggested list of options.
We are looking to add a schematron validation for this list so that even though it is free form, we want a value from the list.
But we are also being asked to only limit the values to Unclassified, Protected A and Protected B as we are not allowed to store data with higher security classifications.
I was wondering what the suggestions for applying this change?
I plan on making a schematron that will check the values based on the same thesaurus that generates the helper list.
Comments?
Feature: Edit mode > Add associated resources > example: link an online resource.
Users can list both a resource name and a resource description.
In the metadata profile, users can see all information input as part of adding the resource. (ie. file name/resource name, resource description, link to resource (url), etc).
If a user searches the metadata, there are quick link options to access the resources that have been added to the metadata. This is where it appears we have problem. From the search results (grid or list), users can click on the link or map buttons to view the list of available resources.
In this view, the catalogue only lists the resources by their description NOT their name. It is difficult to determine which resource to access based on the resource description.
There is a new metadata template with French as the main language "Profile - HNAP Level 1 (French)"/"Profile - PNAH niveau 1 (française)"
When looking at the list of templates with the ui set to English - I would expect to see "Profile - HNAP Level 1 (French)" but I get the French.
Also the default display in English also show other French data.
Oddly, if I used the formatter such as "/formatters/xsl-view?root=div&view=default" then it displays as expected.
Sample data shows inconsistencies between date displayed in xml
, default
display mode, and full
display mode.
Ocean Salmon Program b16db0fe-a4ec-4844-819e-630e33137b6d
2018-12-17T12:41:20
(displayed full Mon Dec 17 2018 12:41:20 GMT-0800
)1993
(display full: 1992
default 1992-12-31
)2018
(display full 2017
)2018-12
(display full: Dec 2018
)1990-07-25
(display full: 25 Jul 1990
)1993-07-27
(display full: 27 Jul 1993
)Harbour Porpoise Presence, Maritimes Region 58ea48ab-f052-48ab-9c18-4353e51b8bea
2018-09-21T12:08:41
(displayed full Fri Sep 21 2018 12:08:41 GMT-0700
)2016-11-18
(displayed full: 18 Nov 2016
default 2016-11-18
)2018-06-21
(displayed full 21 Jun 2018
)2018-05-09
(displayed full 09 May 2018
)Sites d'échouerie potentiels 3b0ccbbe-e0f7-4e6f-a6aa-8b327c69a169
2020-03-25T14:12:19
(displayed Wed Mar 25 2020 14:12:19 GMT-0700
)2001
(display default: 2000-12-31
full:2000
)2018
(display full: 2017
)1978
(displayed full 1978
)2000
(displayed full 2000
)A couple things for a release, based on core-genetwork example:
3.10.4-SNAPSHOT
to 3.10.4
jar
and zip
to release3.10.4
to 3.10.5-SNAPSHOT
to match the 3.10.x
branch.3.10.4
milestone to collect a list of closed issues and pull requestsKnown issues:
3.10.4
branch uses schemas 3.7
as a parent there is no great way to enforce we are getting the correct transitive dependencies from core-geonetwork. As a workaround please be sure to build the core-geonetwork 3.10.4
tag locally. To avoid conflict all the transitive dependencies from core-geonetwork have been marked as scope provided.Getting errors after installing the HNAP plugin.
Here are the steps to reproduce
git checkout tags/3.10.1
mvn clean install -Penv-dev -Djava.awt.headless=true -Dmaven.wagon.http.pool=false -Dhttp.keepAlive=false
mvn clean install -Penv-dev -Dtest=XmlValidationTest -pl core -e
All success
Now add the HNAP schema and perform the last test again.
./add-schema.sh iso19139.ca.HNAP http://github.com/metadata101/iso19139.ca.HNAP 3.10.x
mvn clean install -Penv-dev -Dtest=XmlValidationTest -pl core -e
It fails with the following error.
[INFO] Running org.fao.geonet.XmlValidationTest
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.381 s <<< FAILURE! - in org.fao.geonet.XmlValidationTest
[ERROR] testBasicValidation(org.fao.geonet.XmlValidationTest) Time elapsed: 3.661 s <<< FAILURE!
java.lang.AssertionError
at org.fao.geonet.XmlValidationTest.testBasicValidation(XmlValidationTest.java:53)
It seems to fail because the test for file "kernel/valid-metadata.iso19139.xml autodetects" is being autodetected as being HNAP. The autodetect in in schema-indent.xml seems to be correct so I'm not sure why this error is being generated.
Can someone else test this to see if they can reproduce the same results.
I tried to debug this with Intellij but I can't seem to get my environment setup correctly to be able to step though the code.
Thank you
2aa3a4ff-7c4c-44d0-a4e2-9ab1a1df0d56.zip
I imported this attached mef published ECCC I am unable to completely validate in the editor. see image. you can test with the mef attached.
As HNAP requires the main language the main language was added to the Simple editor form as read only. But it remained on the full view as editable.
It was done that way because changing the main language of the document without changing all the locales messes up the metadata record and confuses the editor. But it should work as expected for unilingual metadata - which I believe is used by others. So maybe making the field read-only was not a great idea.
I think we should look into seeing if there is a better way of applying the language change. Then we could allow users to change the language.
I'm not sure if this is possible in the editor?
Looking for some feedback on this issue.
when validating record the Thesaurus returns validation error.
Data Identification : Thesaurus cited responsible party role is required
HNAP validation rules for FGP publication (multilingual)
Data Identification : Thesaurus cited responsible party organisation is required
Does the editor add responsible party info for local and external Thesaurus?
@davidblasby has this already been addressed? and therefor just needs testing?
Geonetwork core has a metadata tab in the advanced view. But this does not exists in hnap
Looking to see the file identifier and other metadata information
I would expect the metadata standard fields to also be displayed as non editable field.
Odd that there is a language under identification and under metadata - not sure what the difference is?
I would also expect the metadata character set to be in metadata? But I see it in identification tab - should probably investigate this as well.
I would like to propose an hnap user manual, using the following structure:
docs/
published as github pages (a repository settings change option rather than gh-pages
branch)src/sphinx
sphinx rst
documentationpom.xml
goals to to build docs if sphinx-build installed and publish into docs folderUser focused documentation in contrast the the geonetwork user manual which combines information for visitors, editors and administrators (sometimes in a single paragraph!)
Working on writing guidelines to keep documentation consistent and gradually improve the geonetwork user manual.
In addition to publishing using github pages above, this documentation could be included in the running geonetwork application (for example accessed by a link in the application footer.)
Note core-geonetwork
already publishes documentation added to the geonetwork
web app doc
folder, we are just taking over this functionality to include a user manual rather than the community docs.
Reviewing other active schema plugins:
iso19139: possibility to share general content such as how to search, view and download records. Large range of translations desirable. Long term it may be desirable to setup iso19139 documentation as a base, and use maven to carefully add additional pages.
dutch schemas: Likely only needed in the dutch language, make use of a specific dutch skin so unlikely to collaborate on iso19139 content above.
FILE TITLE
index User Manual
search/index Catalogue
search/records Search records
search/detail View Record
search/download Download record
editor Editor
editor/edit Edit
editor/validation Validation
editor/publish Publish
editor/create Create
admin Administration
admin/settings Settings
The above draft structure is what I have on hand to start the activity. Adding additional pages to cover editor/workflow
can be added over time. The docs use sphinx external links to link to the geonetwork docs as reference material.
Once Rdf's are renamed #53 @davidblasby if you are available can you add the following titles:
rdf-identifier | en-Title | fr-Title |
---|---|---|
EC_Branch | Branch | Direction Générale |
GC_Core_Subject | GC Core Subject | Sujet central de la GC |
EC_Directorate | Directorate | Direction |
GC_Security_Level | GC Security Level | GC Restrictions de manipulation |
GC_Geographic_Scope | Geography | Portée Géographique |
GC_Departments | Government of Canada Departments & Agencies | Ministères et organismes du gouvernement du Canada |
ISO_Countries | Countries | Pays |
EC_Program_PAA | Program (PAA) | Portée du programme (PAA) |
EC_Functions | Business Functions | Fonctions de l'entreprise |
NE_Global_Regions | Global Regions | Régions mondiales |
GC_State_Province | Canada/US States and Provinces | Canada/US États et provinces |
GC_Resource_Format | Resource Format | Format de fichier |
The language links are not consistently in the same order.
In the screenshot below we can see that the organization selector has Francais and then English but the other fields have English and then Francais.
Tried to create new record based on different templates. some are correct and some are not.
The screenshot above was done using the "FGP - Dataset Metadata - ISO19115 NAP (Default)" template.
Note that after I start editing the record - I switched from XML view and back to Basic view and then seems to have corrected itself. So it seems that the update-fixed-info.xsl fixes the problem But we should identify why they are in different order.
Getting the following error.
Thesaurus cited responsible party organisation is required
When I look at the xml I see the following.
<gmd:organisationName xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString/>
<gmd:PT_FreeText>
<gmd:textGroup>
<gmd:LocalisedCharacterString locale="#eng">Government of Canada; Library and Archives Canada</gmd:LocalisedCharacterString>
</gmd:textGroup>
<gmd:textGroup>
<gmd:LocalisedCharacterString locale="#fra">Gouvernement du Canada; Bibliothèque et Archives Canada</gmd:LocalisedCharacterString>
</gmd:textGroup>
</gmd:PT_FreeText>
</gmd:organisationName>
It should be placing the english text into the gco:CharacterString instead of gmd:LocalisedCharacterString
We have a number of active branches, and would like to share:
The discussion can work on the governance over time, the technical side is handled in other tickets.
Anyone know why we have the same dependency artifact twice in the pom.xml?
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>core</artifactId>
<version>${gn.project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>schema-core</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
Also anyone know why the plugin folder is marked as a test resource and why there is reference to a test resource folder that does not exists?
<testResources>
<testResource>
<directory>src/test/resources/org/fao/geonet</directory>
</testResource>
<testResource>
<directory>src/main/plugin/iso19139.ca.HNAP</directory>
</testResource>
</testResources>
I noticed that these same issues exist in schema iso19139 - So may be they were copied over from there?
Looking at the following location I can see that there are still some English text that needs some French translation.
https://github.com/metadata101/iso19139.ca.HNAP/tree/3.10.x/src/main/plugin/iso19139.ca.HNAP/loc/fre
Someone will need to go over the files in this folder and ensure that translation has been done.
Metadata which includes French thumbnail (gmd:fileDescription = thumbnail_fre and large_thumbnail_fre) are not imported correctly.
Looks like thumbnail_fre and large_thumbnail_fre is being ignored when updating/adding http:// to the gmd:fileName during import.
Before import this is the xml
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Preview_Image_s.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription>
<gco:CharacterString>thumbnail</gco:CharacterString>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Preview_Image.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription>
<gco:CharacterString>large_thumbnail</gco:CharacterString>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Preview_Image_s_fre.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription>
<gco:CharacterString>thumbnail_fre</gco:CharacterString>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Preview_Image.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription>
<gco:CharacterString>large_thumbnail_fre</gco:CharacterString>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
And this is what it is after the import.
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>http://localhost/geonetwork/srv/api/records/97ab458b-44b2-4cb4-8e06-8351e1c745b3/attachments/Preview_Image_s.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>thumbnail</gco:CharacterString>
<gmd:PT_FreeText/>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
xsi:type="napm:napMD_FileFormatCode_PropertyType"
codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>http://localhost/geonetwork/srv/api/records/97ab458b-44b2-4cb4-8e06-8351e1c745b3/attachments/Preview_Image.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>large_thumbnail</gco:CharacterString>
<gmd:PT_FreeText/>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
xsi:type="napm:napMD_FileFormatCode_PropertyType"
codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Preview_Image_s_fre.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>thumbnail_fre</gco:CharacterString>
<gmd:PT_FreeText/>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
xsi:type="napm:napMD_FileFormatCode_PropertyType"
codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
<gmd:graphicOverview>
<gmd:MD_BrowseGraphic>
<gmd:fileName>
<gco:CharacterString>Preview_Image.png</gco:CharacterString>
</gmd:fileName>
<gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>large_thumbnail_fre</gco:CharacterString>
<gmd:PT_FreeText/>
</gmd:fileDescription>
<gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
xsi:type="napm:napMD_FileFormatCode_PropertyType"
codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
codeListValue="RI_716">
<gco:CharacterString>png; png</gco:CharacterString>
</gmd:fileType>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
This is a sample that can be used for testing the upload.
97ab458b-44b2-4cb4-8e06-8351e1c745b3.zip
I have an interesting challenge for non GC use, the file schematron-rules-non-multilingual.sch#L136-L151
<!-- Use Limitation -->
<sch:rule context="//gmd:identificationInfo/gmd:MD_DataIdentification
|//*[@gco:isoType='gmd:MD_DataIdentification']
|//*[@gco:isoType='srv:SV_ServiceIdentification']">
<sch:let name="open-licenses" value="document(concat('file:///', replace(concat($thesaurusDir, '/external/thesauri/theme/GC_Open_licenses.rdf'), '\\', '/')))"/>
<sch:let name="openLicense" value="count(gmd:resourceConstraints/gmd:MD_LegalConstraints/gmd:useLimitation[
(normalize-space(gco:CharacterString) = $open-licenses//rdf:Description/ns2:prefLabel[@xml:lang=$mainLanguage2char])])" />
<sch:assert
test="$openLicense > 0"
>$loc/strings/OpenLicense</sch:assert>
</sch:rule>
The above rule wants to ensure the record is available as one of the approved open data licenses defined by GC_Open_licenses.rdf
thesaurus.
Options:
a) Split this into a seperate schematron file, so the check can be disabled as a configuration option
b) Replace GC_Open_licenses.rdf
with an appropriate license
c) change the $openLicense
check
d) can we make a configuration options (to send into the schematron)
When importing data from FGP, we see that the resource descriptions seems to contains multiple fields.
The same field on FGP looks like the following
I don't believe that using structured data into a free text description fields is part of the ISO standard however it is what it is! If not using the same form as FGP, it makes it near impossible for people to know that formatted text needs to go into the description.
Any ideas on how this will be supported in this profile?
Having trouble with updating an older hnap installation:
Results in:
EC_Core_Subject.rdf
GC_Core_Subject.rdf
files defined, with title "Government of Canada Core Subject Thesaurus"GC_Core_Subject.rdf
has title "GC Core Subject"The SchemaInitializerThesauri will copy any missing RDF files into the data directory:
EC_Core_Subject.rdf
aboveGC_Core_Subject.rdf
title aboveIf this kind of activity is common we may wish to build some additional automation.
Contact information for the supplied sample metadata record does not show the contact information correctly.
It shows the following for English (RI_418 needs to be converted)
Government of Canada; Fisheries and Oceans Canada; Oceans and Coastal Management Division
RI_418:
And for French, there is no contact information.
64f741d7-1129-49dd-9e5c-2b1de79024f0.zip
Record also exists on open data - https://open.canada.ca/data/en/dataset/64f741d7-1129-49dd-9e5c-2b1de79024f0
The following files
contains the following
```
<owner_org>49E2ADF4-AD7A-43EB-85C8-6433D37ED62C</owner_org>
<creator>Environment and Climate Change Canada</creator>
<subject>
<value>economics_and_industry</value>
</subject>
I believe this should be changed to customizable so that DFO can use different values.
I'm not sure on the best approach to do this.
Unable to import FGP data into catalog while checking the validate option.
I have uploaded a zip file containing 3 files.
The only difference related to the stripped down versions is that one points to
(JDOM error)http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/gmd/gmd.xsd
And the other point to
(Correct error)http://www.isotc211.org/2005/gmd/gmd.xsd
http://www.isotc211.org/2005/gmd/gmd.xsd is the official gmd source.
http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/gmd/gmd.xsd is the GMD created for NAP/HNAP
Almost seems like there is a bug in the NAP XSD.
Attachment: TestXmlll.zip
The http://nap.geogratis.gc.ca/ server is decommissioned making this issue now critical (ie actively broken) and the metadata documents being published cannot be used by external tools. We are looking for a 5 to 10 year decision here, although realistically documents like these do not expire and are useful as a historical record.
Some good ideas from meeting (following the OGC opengis and w3c example layout):
@bo-lu has access to geo.ca
domain will request schemas.geo.ca
:
The other idea is to setup GitHub pages metadata101.github.io/schemas
:
The Metadata configuration configuration validation to remove schema location for validation is required to validate records that have been generated by geonetwork.
Notes:
Generated schemaLocation information:
Example:
<?xml version="1.0" encoding="UTF-8"?>
<gmd:MD_Metadata xmlns:gml="http://www.opengis.net/gml/3.2"
xmlns:gco="http://www.isotc211.org/2005/gco"
xmlns:gmd="http://www.isotc211.org/2005/gmd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:gfc="http://www.isotc211.org/2005/gfc"
xmlns:gmi="http://www.isotc211.org/2005/gmi"
xmlns:gsr="http://www.isotc211.org/2005/gsr"
xmlns:gmx="http://www.isotc211.org/2005/gmx"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:gss="http://www.isotc211.org/2005/gss"
xmlns:uuid="java:java.util.UUID"
xmlns:srv="http://www.isotc211.org/2005/srv"
xmlns:gts="http://www.isotc211.org/2005/gts"
xsi:schemaLocation="http://www.isotc211.org/2005/gmd http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/gmd/gmd.xsd http://www.isotc211.org/2005/srv http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/srv/srv.xsd http://www.geconnections.org/nap/napMetadataTools/napXsd/napm http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/napm/napm.xsd">
With remove schema location for validation true
, when validating the xsi:schemaLocation
above are removed (and geonetwork is able to validate the file).
``
Failed` to import MEF file '2aa3a4ff-7c4c-44d0-a4e2-9ab1a1df0d56.zip'. Check error for details.
XSD Validation error(s): <xsderrors> <error> <typeOfError>ERROR</typeOfError> <errorNumber>1</errorNumber> <message>cvc-complex-type.2.4.b: The content of element 'gml:TimePeriod' is not complete. One of '{"http://www.opengis.net/gml/3.2":endPosition, "http://www.opengis.net/gml/3.2":end}' is expected. (Element: gml:TimePeriod with parent element: gmd:extent)</message>
<xpath>gmd:identificationInfo/gmd:MD_DataIdentification/gmd:extent[1]/gmd:EX_Extent/gmd:temporalElement/gmd:EX_TemporalExtent/gmd:extent/gml:TimePeriod</xpath> </error> <error> <typeOfError>ERROR</typeOfError> <errorNumber>2</errorNumber> <message>cvc-complex-type.2.4.d: Invalid content was found starting with element 'gmd:supplementalInformation'. No child element is expected at this point. (Element: gmd:supplementalInformation with parent element: gmd:MD_DataIdentification)</message>
<xpath>gmd:identificationInfo/gmd:MD_DataIdentification/gmd:supplementalInformation[2]</xpath> </error> </xsderrors>
Is this actually a validation error for FGP or NHAP.... not clear to me what the actual issue is. See attached mef.
2aa3a4ff-7c4c-44d0-a4e2-9ab1a1df0d56.zip
Chris
It is quite hard to enter open license values by hand, to pass validation.
Updating config-editor.xml
to use keyword picker recommend:
<!--for name="gmd:useLimitation" use="textarea"/-->
<for name="gmd:useLimitation"
use="data-gn-keyword-picker">
<directiveAttributes data-thesaurus-key="external.theme.GC_Open_Licenses" data-focus-to-input="false"/>
</for>
The following has been provided by NrCAN FGP in discussion with @ianwallen:
The problem is that our geonetwork base is 3.8 and FGP geonetwork base is 3.6. In geonetwork 3.8 they applied a change to the language setting.
So prior to 3.8 if the languages were English(main) and French then they would have a English as the main language and French would be in the PT_LOCALE. So in this case there was only one PT_locale.
In 3.8, they still have English as the main language but they also have 2 PT_LOCALE – one for each language. The multiple PT_Locale is causing the issue.
I have included 2 version of the metadata that were attempted to be loaded. The 3.6 version loads correctly. The 3.8 version causes the issues – you test loading them if you want.So we either need to apply a transformation prior to uploading the data to FGP so that it remove the main language PT_locale. Or the FGP system is updated to support multiple PT_LOCALE. I believe it may be easier for us to apply a transformation prior to submitting the data to FGP.
I have already applied a bunch of fixes related to this issue in the new metadata 101 for HNAP (https://github.com/metadata101/iso19139.ca.HNAP) for the 3.8 branch - so I presume that someday FGP will be upgraded to support this.
I believe we should rename the none ECCC specific thesaurus files to GC.
i.e. I don't believe the EC_States.rdf
is specific to ECCC and can probably be applied to all government departments so it may be better to have it renamed to GC_States.rdf
.
As I don't know all the uses for the thesaurus, I believe this should be an ECCC task.
Steps to reproduce.
Create a metadata record. (can use sample metadata records)
Ensure that it is valid
Use the following api to get the xml record.
/{portal}/api/0.1/records/{metadataUuid}/formatters/xml
http://localhost/geonetwork/doc/api/index.html#/records/getRecordAsXmlOrJSON_1
Now attempt to load and validate this record with FGP
In our example - when we attempted to load the record to FGP we got the following errors.
There seems to be some issues with the export - I see that the original data that passes validation contained the following. (when editing the metadata record in xml mode)
<gmd:distributionFormat>
<gmd:MD_Format>
<gmd:name>
<gco:CharacterString>DOC</gco:CharacterString>
</gmd:name>
<gmd:version>
<gco:CharacterString>1.32</gco:CharacterString>
</gmd:version>
</gmd:MD_Format>
</gmd:distributionFormat>
And the one that was generated by the API looks like the following.
<gmd:distributionFormat>
<gmd:MD_Format>
<gmd:name xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>DOC</gco:CharacterString>
<gmd:PT_FreeText/>
</gmd:name>
<gmd:version>
<gco:CharacterString>1.32</gco:CharacterString>
</gmd:version>
</gmd:MD_Format>
</gmd:distributionFormat>
It is putting the gmd:PT_FreeText in the gmd:name and it should not
And it is also moving the gmd:locale was moved to the top before gmd:hierarchyLevel - not sure what that seems to be causing an error?
The drop down list is using the iso19139 dropdown list.
On FGP it is a help list.
I have fixed the issue where the wrong selection list was being selected and I added a REST API to the list.
#105
But I'm not sure how to convert the the drop down list to a helper list so that if we happen to get data that is not part of the dropdown list, it does not accidently remove the data.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.