I'm not sure where the formatters are used but they currently don't seem to be working.

Did a local build, went to metadata and template -> formatters and attempted to use the HNAP formatters and they all seem to fail with the loaded HNAP templates/sample data (b16db0fe-a4ec-4844-819e-630e33137b6d).

http://localhost:8080/geonetwork/srv/eng/md.format.xml?uuid=b16db0fe-a4ec-4844-819e-630e33137b6d&xsl=ec-view&schema=iso19139.ca.HNAP

<apiError>
<code>runtime_exception</code>
<description>Failed to compile stylesheet. 20 errors detected.</description>
<message>TransformerConfigurationException</message>
</apiError>

The following errors was logged to the console

Error at xsl:call-template on line 97 column 45 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 114 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 127 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 140 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 153 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 166 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 179 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 192 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 204 column 45 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 257 column 47 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 394 column 48 of view.xsl:
  XTSE0650: No template exists named parentCollection
Error at xsl:call-template on line 402 column 47 of view.xsl:
  XTSE0650: No template exists named relatedDatasets
Error at xsl:call-template on line 431 column 49 of view.xsl:
  XTSE0650: No template exists named thumbnail
Error at xsl:call-template on line 469 column 43 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 524 column 45 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 538 column 45 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 552 column 45 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 569 column 45 of view.xsl:
  XTSE0650: No template exists named showPanel
Error at xsl:call-template on line 590 column 57 of view.xsl:
  XTSE0650: No template exists named showMetadataExportIcons
Error at xsl:call-template on line 597 column 43 of view.xsl:
  XTSE0650: No template exists named showPanel

http://localhost:8080/geonetwork/srv/eng/md.format.xml?uuid=b16db0fe-a4ec-4844-819e-630e33137b6d&xsl=full_view&schema=iso19139.ca.HNAP

This page contains the following errors:
error on line 13 at column 78: Entity 'nbsp' not defined
Below is a rendering of the page up to the first error.

Was the following change to the NAP specification done for ECCC or FGP? Is this an HNAP requirement? Can this be done via the schematron. It seems odd to modify the NAP specification for one change? Just wondering if this should be reverted?

https://github.com/metadata101/iso19139.ca.HNAP/blob/3.10.x/src/main/plugin/iso19139.ca.HNAP/schema/gmd/citation.xsd#L87

Previously value was minOccurs="0" from NAP specification

I the sample "Harbour Porpoise Presence, Maritimes Region" is loaded. Then going to the view the sample record and attempting to "add to map" will produced the following error.

Biota002_MarineMammals_HarbourPorpoise_PH_SJ_Presence
The WMS getCapabilities request failed, Failed to parse GetCapabilities operation response. Try again later or contact service provider.

I believe a lot of the published Open Data records also have similar issues. So I don't believe it is just a matter of fixing the sample records. May need a goenetwork core modification?

Any suggestions?

Replace "bold" class name with WET standard "h6" class name to enable bold text in html template
Ref: https://wet-boew.github.io/wet-boew-styleguide/design/sizing-en.html

Before:

<div class="col-md-6">
  <span class="bold">Geographic Extent</span>
  <br>
  <span itemprop="spatial">SW: -141 41.68<br>NE: -52.62 83.11</span>
</div>
<div class="col-md-6">
  <span class="bold">Time Period</span>
  <br>
  From:
  <span itemprop="temporal">2001</span>
  <br>
  To:
  <span itemprop="temporal">2006</span>
</div>

After:

<div class="col-md-6">
  <span class="h6">Geographic Extent</span>
  <br>
  <span itemprop="spatial">SW: -141 41.68<br>NE: -52.62 83.11</span>
</div>
<div class="col-md-6">
  <span class="h6">Time Period</span>
  <br>
  From:
  <span itemprop="temporal">2001</span>
  <br>
  To:
  <span itemprop="temporal">2006</span>
</div>

Lines of code to change are in line 396 and 410:
/iso19139.nap/src/main/plugin/iso19139.nap/formatter/ec-view/view.xsl

If I upload the following file and go to the viewer (taken from FGP/Open Data) and go to the viewer

The English Title looks correct but the French title is incorrect. It also seems like it is a json array with duplicate values.

["Cartographie des débarquements de la pêche côtière du homard et de l'effort de pêche sur un quadrillage statistique de la région des Maritimes (2012 à 2014)","Cartographie des débarquements de la pêche côtière du homard et de l'effort de pêche sur un quadrillage statistique de la région des Maritimes (2012 à 2014)"]

I have attempted to debug this with no luck.

I included a sample of the xml file that was uploaded.

64f741d7-1129-49dd-9e5c-2b1de79024f0.zip

The record also exist on open data.
https://open.canada.ca/data/en/dataset/64f741d7-1129-49dd-9e5c-2b1de79024f0

This is affecting the 3.8 and 3.10 branch - did not test 3.6

The config-editor.xml configuration references several angular components that are not inlcuded by geonetwork-core:

• data-gn-country-selector-ec
• data-gn-organisation-entry-selector-ec

These need to be removed and / or replaced for the editor to function.

HNAP plugin currently does not support multilingual gmd:filename. It seems to be based on the older ECCC/FGP code.

We need to be able to upload multilingual thumbnails and other resources.

Here is what seems to be produced based on the samples.

      <gmd:graphicOverview>
        <gmd:MD_BrowseGraphic>
          <gmd:fileName>
            <gco:CharacterString>Ricker_Beam_Trawl_Transects_Map.JPG</gco:CharacterString>
          </gmd:fileName>
          <gmd:fileDescription>
            <gco:CharacterString>large_thumbnail</gco:CharacterString>
          </gmd:fileDescription>
        </gmd:MD_BrowseGraphic>
      </gmd:graphicOverview>
      <gmd:graphicOverview>
        <gmd:MD_BrowseGraphic>
          <gmd:fileName>
            <gco:CharacterString>Ricker_Beam_Trawl_Transects_Map.JPG</gco:CharacterString>
          </gmd:fileName>
          <gmd:fileDescription>
            <gco:CharacterString>large_thumbnail_fre</gco:CharacterString>
          </gmd:fileDescription>
        </gmd:MD_BrowseGraphic>
      </gmd:graphicOverview>

Problem
gmd:fileDescription is not a real description of the resource. Instead it is a code. Of which the code are very specific to ECCC/FGP.
There is no linkage between the first and second resource even though it is the same data in both French and English
Much more difficult to validate that both English and French exists.

Here is the changes that is suggested

      <gmd:graphicOverview>
        <gmd:MD_BrowseGraphic>
          <gmd:fileName>
            <gco:CharacterString>Ricker_Beam_Trawl_Transects_Map.JPG</gco:CharacterString>
             <gmd:PT_FreeText>
              <gmd:textGroup>
                <gmd:LocalisedCharacterString xmlns="" locale="#fra">Ricker_Beam_Trawl_Transects_Map.JPG</gmd:LocalisedCharacterString>
              </gmd:textGroup>
            </gmd:PT_FreeText>
          </gmd:fileName>
          <gmd:fileDescription>
            <gco:CharacterString>Resource description</gco:CharacterString>
             <gmd:PT_FreeText>
              <gmd:textGroup>
                <gmd:LocalisedCharacterString xmlns="" locale="#fra">Description de ressource</gmd:LocalisedCharacterString>
              </gmd:textGroup>
            </gmd:PT_FreeText>
          </gmd:fileDescription>
        </gmd:MD_BrowseGraphic>
      </gmd:graphicOverview>

If anyone has any extra information to supply in regards to this issue then please add some comments.

Extent Description field is and option HNAP field. If a value is entered from the advanced view, the value does not show up in the Basic View

As this is part of the HNAP fields, it should display in the basic view.

@ianwallen you recently added a snippet to add-schema.sh to help address this, it seems to no longer be working:

[ERROR] testExecListOne(org.fao.geonet.services.metadata.schema.SchematronCriteriaGroupServiceIntegrationTest)  Time elapsed: 0.001 s  <<< ERROR!
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 10 in XML document from URL [jar:file:/Users/jgarnett/.m2/repository/org/geonetwork-opensource/schema-iso19139.ca.HNAP/3.11.0-SNAPSHOT/schema-iso19139.ca.HNAP-3.11.0-SNAPSHOT.jar!/config-spring-geonetwork.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 10; columnNumber: 75; cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'context:component-scan'.
Caused by: org.xml.sax.SAXParseException: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'context:component-scan'.

Original comments bellow from Jose:

Related to duplicated gmd:supplementalInformation, I found the problem, the element cardinality is 0:1, but can be multiple napec:EC_CorporateInfo/napec:EC_Project elements

An option can be to add all in the same element i different lines, we add now:

gco:CharacterString
Project; PROJECT TEXT_1
</gco:CharacterString>

Can be added like:

gco:CharacterString
Project; PROJECT TEXT_1
Project; PROJECT TEXT_2
</gco:CharacterString>

Publication of new drafts:
Need re-confirmation – if an editor submits metadata for review, the CR can approve the metadata OR publish (which approves the metadata by default) correct? Or, if the metadata is submitted, the CR must approve THEN publish - separate actions?

• In testing, I can approve the review of the metadata, then publish.
• I cannot publish submitted metadata without approving first. I get an error message stating it must be approved
• I also tested direct publishing of valid metadata from the CR account – it would not let me publish metadata that was not approved.

Review submission/publish of working copies: problem.
As an editor;

• I can create a new working copy, make and save changes, and validate the metadata while in edit mode.
• Metadata shows as valid.
• I submit it to the CR for review.

As Content Reviewer;

• In the editor board the, the working copy of the metadata is gray (unknown validation) with a question mark icon (despite my validating as the editor).
• When I click to view the working copy, it takes me to the published version.
• I see the blue menu prompt on the metadata stating that a working copy exists and to click to view it. It won’t take me to the draft.
• The working copy isn’t counted in the stats in the left hand filter menu > not counted in the # of submitted records in workflow.
• Approval stalemate.
  
  Unpublish: appears ok.

2020-09-05: in testing this afternoon, I've submitted a metadata record "Publication in GeoCat Live - Test 7" (48fcb49e-993f-4eef-bb55-f4c65a50c191) as an editor successfully but the metadata fails to appear for approval/publish in the CR account. Can't account for why this would happen. Workspaces are a match.

for the online resource section of the editor the following behaviour is observed:

If you type the resource name info in English and/or French then choose or modify what was selected for protocol, the resource name information disappears and must be reentered. Is this the expected behaviour? Creates more work for user if they selected the wrong protocol to start.

@davidblasby comment - "It's a issue with the language management in the dialog, HNAP uses fra for french and seem not well managed."

I need some help in creating an xslt or XQuery that updates all gdm:voice to be bilingual. is there an existing sample maybe in this repo that could help me?

If not null Change from:

              <gmd:voice>
                <gco:CharacterString>819-997-2800</gco:CharacterString>
              </gmd:voice>

Changed to:

              <gmd:voice xsi:type="gmd:PT_FreeText_PropertyType">
                <gco:CharacterString>819-997-2800</gco:CharacterString>
                <gmd:PT_FreeText>
                  <gmd:textGroup>
                    <gmd:LocalisedCharacterString locale="#fra">819-997-2800</gmd:LocalisedCharacterString>
                  </gmd:textGroup>
                </gmd:PT_FreeText>
              </gmd:voice>

The following

Generates the following error.

There are a couple issues.
1 - The org name is correct and the error is incorrect.
2 - If the org name is not correct it should only show one error not 2. Once the first issues is corrected then we can show the second. They are sort of dependent.

The initial schema plugin is arriving as #1 against branch 3.8.x.

We should decide which active branches to maintain:

• 3.6.x is used by Federal Geospatial Portal (FGP)
• 3.6.x-ec used by Environment Climate Change Canada (ECCC)
• 3.8.x not in use
• 3.10.x stable branch of core-geonetwork (GeoCat Live, DFO)
• master not in use, available to track core-geonetwork master

I would like to understand the process needed to review and edit French and English update help tool. Where do are tips managed within the repository?

The readme.md does not seem to have the correct ./add-schema.sh command for all branches.

• The 3.6 branches indicates to use the 3.8 - but I think it should be 3.6
• In the 3.8 branch it has a line to link to the geocat repos which I don't think should be there.

May want to review this to see if it is correct.

There is an odd inconsistency preventing validating:

The check in schematrong-rules-common.sch for:

      <sch:let name="coreSubjectThesaurusExists"
               value="count(gmd:descriptiveKeywords[*/gmd:thesaurusName/*/gmd:title/*/text() = 'Government of Canada Core Subject Thesaurus' or
              */gmd:thesaurusName/*/gmd:title/*/text() = 'Government of Canada Core Subject Thesaurus']) > 0" />

Does not match with GC_Core_Subject.rdf has the following title:

<skos:ConceptScheme rdf:about="http://www.thesaurus.gc.ca/#CoreSubjectThesaurus">
      <dc:title>GC Core Subject</dc:title>

Forgive me if I word this wrong, I am pulling on a piece of string to describe a challenge.

The HNAP schema also has some keywords thesauri built-in

The HNAP schema (core-geonetwork) has a workflow for adding additional keyword thesauri:

1. Use the admin menu to upload a thesauri

2. Define a template that references the new thesauri

3. Any metadata record created from that template can access and use the keywords defined

I would like to discuss how we can manage additional optional thesauri in the plugin, perhaps as a build profile, in order to manage these under version control. Would such a proposal be welcome?

Security note currently contain the following options

The data is also stored in a free form text field and uses a helper for a suggested list of options.

We are looking to add a schematron validation for this list so that even though it is free form, we want a value from the list.

But we are also being asked to only limit the values to Unclassified, Protected A and Protected B as we are not allowed to store data with higher security classifications.

I was wondering what the suggestions for applying this change?

I plan on making a schematron that will check the values based on the same thesaurus that generates the helper list.

• Should we fork and have our own list of thesaurus for security classifications which would only contain Unclassified, Protected A and Protected B?
• We can probably update our custom thesaurus after each deploy?
• Everyone can agree that Protected B will be the highest classification allowed and we all use the same check Unclassified, Protected A and Protected B
• Created a custom overwrite process - I know @jodygarnett has had similar issues with different thesaurus

Comments?

Feature: Edit mode > Add associated resources > example: link an online resource.

Users can list both a resource name and a resource description.

In the metadata profile, users can see all information input as part of adding the resource. (ie. file name/resource name, resource description, link to resource (url), etc).

If a user searches the metadata, there are quick link options to access the resources that have been added to the metadata. This is where it appears we have problem. From the search results (grid or list), users can click on the link or map buttons to view the list of available resources.

In this view, the catalogue only lists the resources by their description NOT their name. It is difficult to determine which resource to access based on the resource description.

There is a new metadata template with French as the main language "Profile - HNAP Level 1 (French)"/"Profile - PNAH niveau 1 (française)"

When looking at the list of templates with the ui set to English - I would expect to see "Profile - HNAP Level 1 (French)" but I get the French.

Also the default display in English also show other French data.

Oddly, if I used the formatter such as "/formatters/xsl-view?root=div&view=default" then it displays as expected.

Sample data shows inconsistencies between date displayed in xml, default display mode, and full display mode.

Ocean Salmon Program b16db0fe-a4ec-4844-819e-630e33137b6d

• dateStamp: 2018-12-17T12:41:20 (displayed full Mon Dec 17 2018 12:41:20 GMT-0800)
• created: 1993 (display full: 1992 default 1992-12-31 )
• published: 2018 (display full 2017)
• revision: 2018-12 (display full: Dec 2018)
• begin: 1990-07-25 (display full: 25 Jul 1990)
• end: 1993-07-27 (display full: 27 Jul 1993)

Harbour Porpoise Presence, Maritimes Region 58ea48ab-f052-48ab-9c18-4353e51b8bea

• dateStamp: 2018-09-21T12:08:41 (displayed full Fri Sep 21 2018 12:08:41 GMT-0700)
• created: 2016-11-18 (displayed full: 18 Nov 2016 default 2016-11-18)
• published: 2018-06-21 (displayed full 21 Jun 2018)
• revision
• begin: 2018-05-09 (displayed full 09 May 2018)

Sites d'échouerie potentiels 3b0ccbbe-e0f7-4e6f-a6aa-8b327c69a169

• dateStamp: 2020-03-25T14:12:19 (displayed Wed Mar 25 2020 14:12:19 GMT-0700)
• creation: 2001 (display default: 2000-12-31 full:2000)
• publication: 2018 (display full: 2017)
• revision:
• begin: 1978 (displayed full 1978)
• end: 2000 (displayed full 2000 )

A couple things for a release, based on core-genetwork example:

• commit changing from 3.10.4-SNAPSHOT to 3.10.4
• tag the above commit
• mark the tag as a release in GitHub, fill in some release notes
• attach jar and zip to release
• commit changing from 3.10.4 to 3.10.5-SNAPSHOT to match the 3.10.x branch.
• Create a 3.10.4 milestone to collect a list of closed issues and pull requests

Known issues:

• Because the 3.10.4 branch uses schemas 3.7 as a parent there is no great way to enforce we are getting the correct transitive dependencies from core-geonetwork. As a workaround please be sure to build the core-geonetwork 3.10.4 tag locally. To avoid conflict all the transitive dependencies from core-geonetwork have been marked as scope provided.

Getting errors after installing the HNAP plugin.
Here are the steps to reproduce

git checkout tags/3.10.1

mvn clean install -Penv-dev -Djava.awt.headless=true -Dmaven.wagon.http.pool=false -Dhttp.keepAlive=false

mvn clean install -Penv-dev -Dtest=XmlValidationTest -pl core -e

All success
Now add the HNAP schema and perform the last test again.

./add-schema.sh iso19139.ca.HNAP http://github.com/metadata101/iso19139.ca.HNAP 3.10.x

mvn clean install -Penv-dev -Dtest=XmlValidationTest -pl core -e

It fails with the following error.

[INFO] Running org.fao.geonet.XmlValidationTest
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 11.381 s <<< FAILURE! - in org.fao.geonet.XmlValidationTest
[ERROR] testBasicValidation(org.fao.geonet.XmlValidationTest)  Time elapsed: 3.661 s  <<< FAILURE!
java.lang.AssertionError
	at org.fao.geonet.XmlValidationTest.testBasicValidation(XmlValidationTest.java:53)

It seems to fail because the test for file "kernel/valid-metadata.iso19139.xml autodetects" is being autodetected as being HNAP. The autodetect in in schema-indent.xml seems to be correct so I'm not sure why this error is being generated.

Can someone else test this to see if they can reproduce the same results.
I tried to debug this with Intellij but I can't seem to get my environment setup correctly to be able to step though the code.

Thank you

https://iso19139echnap.geocat.live/geonetwork/srv/eng/catalog.search#/metadata/2aa3a4ff-7c4c-44d0-a4e2-9ab1a1df0d56

2aa3a4ff-7c4c-44d0-a4e2-9ab1a1df0d56.zip

I imported this attached mef published ECCC I am unable to completely validate in the editor. see image. you can test with the mef attached.

As HNAP requires the main language the main language was added to the Simple editor form as read only. But it remained on the full view as editable.

It was done that way because changing the main language of the document without changing all the locales messes up the metadata record and confuses the editor. But it should work as expected for unilingual metadata - which I believe is used by others. So maybe making the field read-only was not a great idea.

I think we should look into seeing if there is a better way of applying the language change. Then we could allow users to change the language.

I'm not sure if this is possible in the editor?
Looking for some feedback on this issue.

when validating record the Thesaurus returns validation error.

Data Identification : Thesaurus cited responsible party role is required
HNAP validation rules for FGP publication (multilingual)
Data Identification : Thesaurus cited responsible party organisation is required

@

Does the editor add responsible party info for local and external Thesaurus?

@davidblasby has this already been addressed? and therefor just needs testing?

Geonetwork core has a metadata tab in the advanced view. But this does not exists in hnap

Looking to see the file identifier and other metadata information

I would expect the metadata standard fields to also be displayed as non editable field.

Odd that there is a language under identification and under metadata - not sure what the difference is?
I would also expect the metadata character set to be in metadata? But I see it in identification tab - should probably investigate this as well.

I would like to propose an hnap user manual, using the following structure:

• docs/ published as github pages (a repository settings change option rather than gh-pages branch)
• src/sphinx sphinx rst documentation
• pom.xml goals to to build docs if sphinx-build installed and publish into docs folder

user documentation

User focused documentation in contrast the the geonetwork user manual which combines information for visitors, editors and administrators (sometimes in a single paragraph!)

Working on writing guidelines to keep documentation consistent and gradually improve the geonetwork user manual.

runtime documentation

In addition to publishing using github pages above, this documentation could be included in the running geonetwork application (for example accessed by a link in the application footer.)

Note core-geonetwork already publishes documentation added to the geonetwork web app doc folder, we are just taking over this functionality to include a user manual rather than the community docs.

maintenance

Reviewing other active schema plugins:

iso19139: possibility to share general content such as how to search, view and download records. Large range of translations desirable. Long term it may be desirable to setup iso19139 documentation as a base, and use maven to carefully add additional pages.

dutch schemas: Likely only needed in the dutch language, make use of a specific dutch skin so unlikely to collaborate on iso19139 content above.

Draft structure

FILE               TITLE
index              User Manual
search/index       Catalogue
search/records     Search records
search/detail      View Record
search/download    Download record
editor             Editor
editor/edit        Edit
editor/validation  Validation
editor/publish     Publish
editor/create      Create
admin              Administration
admin/settings     Settings

The above draft structure is what I have on hand to start the activity. Adding additional pages to cover editor/workflow can be added over time. The docs use sphinx external links to link to the geonetwork docs as reference material.

Once Rdf's are renamed #53 @davidblasby if you are available can you add the following titles:

The language links are not consistently in the same order.

In the screenshot below we can see that the organization selector has Francais and then English but the other fields have English and then Francais.

Tried to create new record based on different templates. some are correct and some are not.

The screenshot above was done using the "FGP - Dataset Metadata - ISO19115 NAP (Default)" template.

Note that after I start editing the record - I switched from XML view and back to Basic view and then seems to have corrected itself. So it seems that the update-fixed-info.xsl fixes the problem But we should identify why they are in different order.

Getting the following error.

Thesaurus cited responsible party organisation is required

When I look at the xml I see the following.

<gmd:organisationName xsi:type="gmd:PT_FreeText_PropertyType">
                              <gco:CharacterString/>
                              <gmd:PT_FreeText>
                                 <gmd:textGroup>
                                    <gmd:LocalisedCharacterString locale="#eng">Government of Canada; Library and Archives Canada</gmd:LocalisedCharacterString>
                                 </gmd:textGroup>
                                 <gmd:textGroup>
                                    <gmd:LocalisedCharacterString locale="#fra">Gouvernement du Canada; Bibliothèque et Archives Canada</gmd:LocalisedCharacterString>
                                 </gmd:textGroup>
                              </gmd:PT_FreeText>
                           </gmd:organisationName>

It should be placing the english text into the gco:CharacterString instead of gmd:LocalisedCharacterString

We have a number of active branches, and would like to share:

• Tracking HNAP profile
• Collect "official" thesauri in use by different organizations
• Gradually collect example records that have caused validation problems

The discussion can work on the governance over time, the technical side is handled in other tickets.

Anyone know why we have the same dependency artifact twice in the pom.xml?

 <dependency>
     <groupId>${project.groupId}</groupId>
     <artifactId>core</artifactId>
     <version>${gn.project.version}</version>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>schema-core</artifactId>
      <version>${project.version}</version>
      <type>test-jar</type>
      <scope>test</scope>
    </dependency>
    <dependency>

Also anyone know why the plugin folder is marked as a test resource and why there is reference to a test resource folder that does not exists?

<testResources>
      <testResource>
        <directory>src/test/resources/org/fao/geonet</directory>
      </testResource>
      <testResource>
        <directory>src/main/plugin/iso19139.ca.HNAP</directory>
      </testResource>
    </testResources>

I noticed that these same issues exist in schema iso19139 - So may be they were copied over from there?

Looking at the following location I can see that there are still some English text that needs some French translation.

https://github.com/metadata101/iso19139.ca.HNAP/tree/3.10.x/src/main/plugin/iso19139.ca.HNAP/loc/fre

i.e.
https://github.com/metadata101/iso19139.ca.HNAP/blob/3.10.x/src/main/plugin/iso19139.ca.HNAP/loc/fre/schematron-rules-common.xml#L24

https://github.com/metadata101/iso19139.ca.HNAP/blob/3.10.x/src/main/plugin/iso19139.ca.HNAP/loc/fre/schematron-rules-common.xml#L29

https://github.com/metadata101/iso19139.ca.HNAP/blob/3.10.x/src/main/plugin/iso19139.ca.HNAP/loc/fre/schematron-rules-multilingual.xml#L14-L27

Someone will need to go over the files in this folder and ensure that translation has been done.

Metadata which includes French thumbnail (gmd:fileDescription = thumbnail_fre and large_thumbnail_fre) are not imported correctly.

Looks like thumbnail_fre and large_thumbnail_fre is being ignored when updating/adding http:// to the gmd:fileName during import.

Before import this is the xml

<gmd:graphicOverview>
        <gmd:MD_BrowseGraphic>
          <gmd:fileName>
            <gco:CharacterString>Preview_Image_s.png</gco:CharacterString>
          </gmd:fileName>
          <gmd:fileDescription>
            <gco:CharacterString>thumbnail</gco:CharacterString>
          </gmd:fileDescription>
          <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
            <gco:CharacterString>png; png</gco:CharacterString>
          </gmd:fileType>
        </gmd:MD_BrowseGraphic>
      </gmd:graphicOverview>
      <gmd:graphicOverview>
        <gmd:MD_BrowseGraphic>
          <gmd:fileName>
            <gco:CharacterString>Preview_Image.png</gco:CharacterString>
          </gmd:fileName>
          <gmd:fileDescription>
            <gco:CharacterString>large_thumbnail</gco:CharacterString>
          </gmd:fileDescription>
          <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
            <gco:CharacterString>png; png</gco:CharacterString>
          </gmd:fileType>
        </gmd:MD_BrowseGraphic>
      </gmd:graphicOverview>
      <gmd:graphicOverview>
        <gmd:MD_BrowseGraphic>
          <gmd:fileName>
            <gco:CharacterString>Preview_Image_s_fre.png</gco:CharacterString>
          </gmd:fileName>
          <gmd:fileDescription>
            <gco:CharacterString>thumbnail_fre</gco:CharacterString>
          </gmd:fileDescription>
          <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
            <gco:CharacterString>png; png</gco:CharacterString>
          </gmd:fileType>
        </gmd:MD_BrowseGraphic>
      </gmd:graphicOverview>
      <gmd:graphicOverview>
        <gmd:MD_BrowseGraphic>
          <gmd:fileName>
            <gco:CharacterString>Preview_Image.png</gco:CharacterString>
          </gmd:fileName>
          <gmd:fileDescription>
            <gco:CharacterString>large_thumbnail_fre</gco:CharacterString>
          </gmd:fileDescription>
          <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm" xsi:type="napm:napMD_FileFormatCode_PropertyType" codeList="http://nap.geogratis.gc.ca/metadata/register/registerItemClasses-eng.html#IC_115" codeListValue="RI_716">
            <gco:CharacterString>png; png</gco:CharacterString>
          </gmd:fileType>
        </gmd:MD_BrowseGraphic>
      </gmd:graphicOverview>

And this is what it is after the import.

<gmd:graphicOverview>
            <gmd:MD_BrowseGraphic>
               <gmd:fileName>
                  <gco:CharacterString>http://localhost/geonetwork/srv/api/records/97ab458b-44b2-4cb4-8e06-8351e1c745b3/attachments/Preview_Image_s.png</gco:CharacterString>
               </gmd:fileName>
               <gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
                  <gco:CharacterString>thumbnail</gco:CharacterString>
                  <gmd:PT_FreeText/>
               </gmd:fileDescription>
               <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
                             xsi:type="napm:napMD_FileFormatCode_PropertyType"
                             codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
                             codeListValue="RI_716">
                  <gco:CharacterString>png; png</gco:CharacterString>
               </gmd:fileType>
            </gmd:MD_BrowseGraphic>
         </gmd:graphicOverview>
         <gmd:graphicOverview>
            <gmd:MD_BrowseGraphic>
               <gmd:fileName>
                  <gco:CharacterString>http://localhost/geonetwork/srv/api/records/97ab458b-44b2-4cb4-8e06-8351e1c745b3/attachments/Preview_Image.png</gco:CharacterString>
               </gmd:fileName>
               <gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
                  <gco:CharacterString>large_thumbnail</gco:CharacterString>
                  <gmd:PT_FreeText/>
               </gmd:fileDescription>
               <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
                             xsi:type="napm:napMD_FileFormatCode_PropertyType"
                             codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
                             codeListValue="RI_716">
                  <gco:CharacterString>png; png</gco:CharacterString>
               </gmd:fileType>
            </gmd:MD_BrowseGraphic>
         </gmd:graphicOverview>
         <gmd:graphicOverview>
            <gmd:MD_BrowseGraphic>
               <gmd:fileName>
                  <gco:CharacterString>Preview_Image_s_fre.png</gco:CharacterString>
               </gmd:fileName>
               <gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
                  <gco:CharacterString>thumbnail_fre</gco:CharacterString>
                  <gmd:PT_FreeText/>
               </gmd:fileDescription>
               <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
                             xsi:type="napm:napMD_FileFormatCode_PropertyType"
                             codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
                             codeListValue="RI_716">
                  <gco:CharacterString>png; png</gco:CharacterString>
               </gmd:fileType>
            </gmd:MD_BrowseGraphic>
         </gmd:graphicOverview>
         <gmd:graphicOverview>
            <gmd:MD_BrowseGraphic>
               <gmd:fileName>
                  <gco:CharacterString>Preview_Image.png</gco:CharacterString>
               </gmd:fileName>
               <gmd:fileDescription xsi:type="gmd:PT_FreeText_PropertyType">
                  <gco:CharacterString>large_thumbnail_fre</gco:CharacterString>
                  <gmd:PT_FreeText/>
               </gmd:fileDescription>
               <gmd:fileType xmlns:napm="http://www.geconnections.org/nap/napMetadataTools/napXsd/napm"
                             xsi:type="napm:napMD_FileFormatCode_PropertyType"
                             codeList="http://nap.geogratis.gc.ca/metadata/register/napMetadataRegister.xml#IC_115"
                             codeListValue="RI_716">
                  <gco:CharacterString>png; png</gco:CharacterString>
               </gmd:fileType>
            </gmd:MD_BrowseGraphic>
         </gmd:graphicOverview>

This is a sample that can be used for testing the upload.
97ab458b-44b2-4cb4-8e06-8351e1c745b3.zip

I have an interesting challenge for non GC use, the file schematron-rules-non-multilingual.sch#L136-L151

    <!-- Use Limitation -->
    <sch:rule context="//gmd:identificationInfo/gmd:MD_DataIdentification
            |//*[@gco:isoType='gmd:MD_DataIdentification']
            |//*[@gco:isoType='srv:SV_ServiceIdentification']">

      <sch:let name="open-licenses" value="document(concat('file:///', replace(concat($thesaurusDir, '/external/thesauri/theme/GC_Open_licenses.rdf'), '\\', '/')))"/>

      <sch:let name="openLicense" value="count(gmd:resourceConstraints/gmd:MD_LegalConstraints/gmd:useLimitation[
               (normalize-space(gco:CharacterString) = $open-licenses//rdf:Description/ns2:prefLabel[@xml:lang=$mainLanguage2char])])" />


      <sch:assert
        test="$openLicense > 0"
      >$loc/strings/OpenLicense</sch:assert>

    </sch:rule>

The above rule wants to ensure the record is available as one of the approved open data licenses defined by GC_Open_licenses.rdf thesaurus.

Options:

a) Split this into a seperate schematron file, so the check can be disabled as a configuration option

b) Replace GC_Open_licenses.rdf with an appropriate license

c) change the $openLicense check

d) can we make a configuration options (to send into the schematron)

When importing data from FGP, we see that the resource descriptions seems to contains multiple fields.

The same field on FGP looks like the following

I don't believe that using structured data into a free text description fields is part of the ISO standard however it is what it is! If not using the same form as FGP, it makes it near impossible for people to know that formatted text needs to go into the description.

Any ideas on how this will be supported in this profile?

• 1 - The selected values is using the text instead of the code
• 2 - languages should be a multi-selection option
• 3 - The items should be semi-colon separated without space, values should be comma seperated.
• 4 - The formatting seems to be missing the indentations. It should be lined up with the other options.

Having trouble with updating an older hnap installation:

1. thesaurus file renames: #53

• 1. EC_Core_Subject -> GC_Core_Subject
• 2. EC_GC_Security_Classification -> GC_Security_Level
• 3. EC_Geographic_Scope -> GC_Geographic_Scope
• 4. EC_Government_Titles -> GC_Departments
• 5. EC_ISO_Countries -> ISO_Countries
• 6. EC_Program -> EC_Program_PAA
• 7. EC_WAF -> EC_Functions
• 8. regions -> NE_Global_Regions
• 9. EC_States -> GC_State_Province
• 10. EC_Resource_Formats -> GC_Resource_Format
• EC_Directorate -> unchanged
• EC_Branch -> unchanged

2. title updates: #74

Results in:

• Both EC_Core_Subject.rdf GC_Core_Subject.rdf files defined, with title "Government of Canada Core Subject Thesaurus"
• Codebase GC_Core_Subject.rdf has title "GC Core Subject"

The SchemaInitializerThesauri will copy any missing RDF files into the data directory:

• But it will not remove previously managed files, such as EC_Core_Subject.rdf above
• Expected OVERWRITE_EXISTING_THESAURI setting to update the contents GC_Core_Subject.rdf title above

If this kind of activity is common we may wish to build some additional automation.

Contact information for the supplied sample metadata record does not show the contact information correctly.

It shows the following for English (RI_418 needs to be converted)

Government of Canada; Fisheries and Oceans Canada; Oceans and Coastal Management Division
RI_418:

And for French, there is no contact information.

64f741d7-1129-49dd-9e5c-2b1de79024f0.zip

Record also exists on open data - https://open.canada.ca/data/en/dataset/64f741d7-1129-49dd-9e5c-2b1de79024f0

The following files

https://github.com/metadata101/iso19139.ca.HNAP/blob/3.10.x/src/main/plugin/iso19139.ca.HNAP/extract-ckan-dataset.xsl#L65-L70

contains the following

    ```

    <owner_org>49E2ADF4-AD7A-43EB-85C8-6433D37ED62C</owner_org>
    <creator>Environment and Climate Change Canada</creator>
    <subject>
      <value>economics_and_industry</value>
    </subject>

I believe this should be changed to customizable so that DFO can use different values.
I'm not sure on the best approach to do this.

Unable to import FGP data into catalog while checking the validate option.

I have uploaded a zip file containing 3 files.

• TestXmlll.html - test file used for testing.
• TestXmlll_stripped_downfails.xml - Stripped down version which still generated the JDOM error.
• TestXmlll_stripped_down_expected_errors.xml - Stripped down version with small change which gives the expected error.

The only difference related to the stripped down versions is that one points to
(JDOM error)http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/gmd/gmd.xsd
And the other point to
(Correct error)http://www.isotc211.org/2005/gmd/gmd.xsd

http://www.isotc211.org/2005/gmd/gmd.xsd is the official gmd source.

http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/gmd/gmd.xsd is the GMD created for NAP/HNAP

Almost seems like there is a bug in the NAP XSD.

Attachment: TestXmlll.zip

The http://nap.geogratis.gc.ca/ server is decommissioned making this issue now critical (ie actively broken) and the metadata documents being published cannot be used by external tools. We are looking for a 5 to 10 year decision here, although realistically documents like these do not expire and are useful as a historical record.

Some good ideas from meeting (following the OGC opengis and w3c example layout):

@bo-lu has access to geo.ca domain will request schemas.geo.ca:

• https://schemas.geo.ca/iso19139.HNAP/1.2
• https://schemas.geo.ca/iso19139.HNAP/2.3.1
• Backed by a Government of Canada github repository (for groups to update via pull request).

The other idea is to setup GitHub pages metadata101.github.io/schemas:

• https://metadata101.github.io/schemas/iso19139.HNAP/1.2
• https://metadata101.github.io/schemas/iso19139.HNAP/2.3.1
• repo would be available for use by any of the metadata 101 schemas

Original: Schema location for hnap version 1.2 and 2.3.1, replacing http://nap.geogratis.gc.ca/

The Metadata configuration configuration validation to remove schema location for validation is required to validate records that have been generated by geonetwork.

Notes:

• remove schema location for validation removes the schemaLocation from the xml editor when validation button is pressed
• exporting always generates schemaLocation information (when record is downloaded as xml)
• Records downloaded as xml, do not subsequently validate (when imported back into geonetwork again)

Generated schemaLocation information:

• http://www.isotc211.org/2005/gmd
• http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/gmd/gmd.xsd
• http://www.isotc211.org/2005/srv http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/srv/srv.xsd
• http://www.geconnections.org/nap/napMetadataTools/napXsd/napm
• http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/napm/napm.xsd

Example:

<?xml version="1.0" encoding="UTF-8"?>
<gmd:MD_Metadata xmlns:gml="http://www.opengis.net/gml/3.2"
                 xmlns:gco="http://www.isotc211.org/2005/gco"
                 xmlns:gmd="http://www.isotc211.org/2005/gmd"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:gfc="http://www.isotc211.org/2005/gfc"
                 xmlns:gmi="http://www.isotc211.org/2005/gmi"
                 xmlns:gsr="http://www.isotc211.org/2005/gsr"
                 xmlns:gmx="http://www.isotc211.org/2005/gmx"
                 xmlns:xlink="http://www.w3.org/1999/xlink"
                 xmlns:gss="http://www.isotc211.org/2005/gss"
                 xmlns:uuid="java:java.util.UUID"
                 xmlns:srv="http://www.isotc211.org/2005/srv"
                 xmlns:gts="http://www.isotc211.org/2005/gts"
                 xsi:schemaLocation="http://www.isotc211.org/2005/gmd http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/gmd/gmd.xsd http://www.isotc211.org/2005/srv http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/srv/srv.xsd http://www.geconnections.org/nap/napMetadataTools/napXsd/napm http://nap.geogratis.gc.ca/metadata/tools/schemas/metadata/can-cgsb-171.100-2009-a/napm/napm.xsd">

With remove schema location for validation true, when validating the xsi:schemaLocation above are removed (and geonetwork is able to validate the file).

``

Failed` to import MEF file '2aa3a4ff-7c4c-44d0-a4e2-9ab1a1df0d56.zip'. Check error for details.
XSD Validation error(s): <xsderrors> <error> <typeOfError>ERROR</typeOfError> <errorNumber>1</errorNumber> <message>cvc-complex-type.2.4.b: The content of element 'gml:TimePeriod' is not complete. One of '{"http://www.opengis.net/gml/3.2":endPosition, "http://www.opengis.net/gml/3.2":end}' is expected. (Element: gml:TimePeriod with parent element: gmd:extent)</message> 

<xpath>gmd:identificationInfo/gmd:MD_DataIdentification/gmd:extent[1]/gmd:EX_Extent/gmd:temporalElement/gmd:EX_TemporalExtent/gmd:extent/gml:TimePeriod</xpath> </error> <error> <typeOfError>ERROR</typeOfError> <errorNumber>2</errorNumber> <message>cvc-complex-type.2.4.d: Invalid content was found starting with element 'gmd:supplementalInformation'. No child element is expected at this point. (Element: gmd:supplementalInformation with parent element: gmd:MD_DataIdentification)</message> 

 <xpath>gmd:identificationInfo/gmd:MD_DataIdentification/gmd:supplementalInformation[2]</xpath> </error> </xsderrors> 

Is this actually a validation error for FGP or NHAP.... not clear to me what the actual issue is. See attached mef.

2aa3a4ff-7c4c-44d0-a4e2-9ab1a1df0d56.zip

Chris

It is quite hard to enter open license values by hand, to pass validation.

Updating config-editor.xml to use keyword picker recommend:

    <!--for name="gmd:useLimitation" use="textarea"/-->
    <for name="gmd:useLimitation"
             use="data-gn-keyword-picker">
          <directiveAttributes data-thesaurus-key="external.theme.GC_Open_Licenses" data-focus-to-input="false"/>
    </for>

The following has been provided by NrCAN FGP in discussion with @ianwallen:

The problem is that our geonetwork base is 3.8 and FGP geonetwork base is 3.6. In geonetwork 3.8 they applied a change to the language setting.
So prior to 3.8 if the languages were English(main) and French then they would have a English as the main language and French would be in the PT_LOCALE. So in this case there was only one PT_locale.
In 3.8, they still have English as the main language but they also have 2 PT_LOCALE – one for each language. The multiple PT_Locale is causing the issue.
I have included 2 version of the metadata that were attempted to be loaded. The 3.6 version loads correctly. The 3.8 version causes the issues – you test loading them if you want.

So we either need to apply a transformation prior to uploading the data to FGP so that it remove the main language PT_locale. Or the FGP system is updated to support multiple PT_LOCALE. I believe it may be easier for us to apply a transformation prior to submitting the data to FGP.

I have already applied a bunch of fixes related to this issue in the new metadata 101 for HNAP (https://github.com/metadata101/iso19139.ca.HNAP) for the 3.8 branch - so I presume that someday FGP will be upgraded to support this.

I believe we should rename the none ECCC specific thesaurus files to GC.

i.e. I don't believe the EC_States.rdf is specific to ECCC and can probably be applied to all government departments so it may be better to have it renamed to GC_States.rdf.

As I don't know all the uses for the thesaurus, I believe this should be an ECCC task.

Steps to reproduce.

Create a metadata record. (can use sample metadata records)
Ensure that it is valid

Use the following api to get the xml record.
/{portal}/api/0.1/records/{metadataUuid}/formatters/xml
http://localhost/geonetwork/doc/api/index.html#/records/getRecordAsXmlOrJSON_1

Now attempt to load and validate this record with FGP

In our example - when we attempted to load the record to FGP we got the following errors.

There seems to be some issues with the export - I see that the original data that passes validation contained the following. (when editing the metadata record in xml mode)

<gmd:distributionFormat>
            <gmd:MD_Format>
               <gmd:name>
                  <gco:CharacterString>DOC</gco:CharacterString>
               </gmd:name>
               <gmd:version>
                  <gco:CharacterString>1.32</gco:CharacterString>
               </gmd:version>
            </gmd:MD_Format>
         </gmd:distributionFormat>

And the one that was generated by the API looks like the following.

<gmd:distributionFormat>
          <gmd:MD_Format>
            <gmd:name xsi:type="gmd:PT_FreeText_PropertyType">
              <gco:CharacterString>DOC</gco:CharacterString>
              <gmd:PT_FreeText/>
            </gmd:name>
            <gmd:version>
              <gco:CharacterString>1.32</gco:CharacterString>
            </gmd:version>
          </gmd:MD_Format>
        </gmd:distributionFormat>

It is putting the gmd:PT_FreeText in the gmd:name and it should not

And it is also moving the gmd:locale was moved to the top before gmd:hierarchyLevel - not sure what that seems to be causing an error?

The drop down list is using the iso19139 dropdown list.

On FGP it is a help list.

I have fixed the issue where the wrong selection list was being selected and I added a REST API to the list.
#105

But I'm not sure how to convert the the drop down list to a helper list so that if we happen to get data that is not part of the dropdown list, it does not accidently remove the data.