Coder Social home page Coder Social logo

nasa-pds / mi-label Goto Github PK

View Code? Open in Web Editor NEW
3.0 10.0 1.0 8.71 MB

Metadata Injector for PDS Labels (MILabel) provides a command-line interface for generating PDS4 Labels using a user provided PDS4 XML template and input (source) data products.

Home Page: https://nasa-pds.github.io/mi-label/

License: Other

Java 97.10% Shell 1.93% Batchfile 0.98%
pds

mi-label's Introduction

Metadata Injector for PDS Labels (MILabel)

DOI πŸ€ͺ Unstable integration & delivery 😌 Stable integration & delivery

This is a software tool, formerly known as "Generate Tool", that provides a command-line interface for generating PDS4 Labels using a user-provided PDS4 XML template and input (source) data products to parse the metadata from the source and inject into the output PDS4 labels. The XML-like Templates use Apache Velocity variables and logic to enable the programmatic generation of the labels.

Documentation

The documentation for the latest release of MILabel, including release notes, installation and operation of the software are online at https://nasa-pds.github.io/mi-label/.

If you would like to get the latest documentation, including any updates since the last release, you can execute the mvn site:run command and view the documentation locally at http://localhost:8080.

πŸ‘₯ Contributing

Within the NASA Planetary Data System, we value the health of our community as much as the code. Towards that end, we ask that you read and practice what's described in these documents:

  • Our contributor's guide delineates the kinds of contributions we accept.
  • Our code of conduct outlines the standards of behavior we practice and expect by everyone who participates with our software.

Build

The software can be compiled and built with the mvn compile command but in order to create the JAR file, you must execute the mvn compile jar:jar command.

In order to create a complete distribution package, execute the following commands:

$ mvn site
$ mvn package

Operational Release

A release candidate should be created after the community has determined that a release should occur. Thanks to GitHub Actions and the Roundup Action, this can happen automatically. However, manual steps are given below should the need for a hand-made release arise.

Clone a Fresh Repository

Run:

git clone [email protected]:NASA-PDS/mi-label.git

Release with ConMan

For internal JPL use, the ConMan software package can be used for releasing software, otherwise the following sections outline how to complete these steps manually.

Manual Release

Manual release steps are detailed in this subsection.

Update Version Numbers

Update pom.xml for the release version or use the Maven Versions Plugin, e.g.:

$ # Skip this step if this is a RELEASE CANDIDATE, we will deploy as SNAPSHOT version for testing
$ VERSION=1.15.0
$ mvn versions:set -DnewVersion=$VERSION
$ git add pom.xml

Update Changelog

Update Changelog using Github Changelog Generator. Note: Make sure you set $CHANGELOG_GITHUB_TOKEN in your .bash_profile or use the --token flag.

$ # For RELEASE CANDIDATE, set VERSION to future release version.
$ github_changelog_generator --future-release v$VERSION
$ git add CHANGELOG.md

Commit Changes

Commit changes using following template commit message:

$ # For operational release
$ git commit -m "[RELEASE] mi-label v$VERSION"
$
$ # For release candidate
$ CANDIDATE_NUM=1
$ git commit -m "[RELEASE] mi-label v${VERSION}-rc${CANDIDATE_NUM}"
$
$ # Push changes to main
$ git push -u origin main

Build and Deploy Software to Sonatype Maven Repo.

$ # For operational release
$ mvn clean site deploy -P release
$
$ # For release candidate
$ mvn clean site deploy

Note: If you have issues with GPG, be sure to make sure you've created your GPG key, sent to server, and have the following in your ~/.m2/settings.xml:

<profiles>
  <profile>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <properties>
      <gpg.executable>gpg</gpg.executable>
      <gpg.keyname>KEY_NAME</gpg.keyname>
      <gpg.passphrase>KEY_PASSPHRASE</gpg.passphrase>
    </properties>
  </profile>
</profiles>

Push Tagged Release

$ # For Release Candidate, you may need to delete old SNAPSHOT tag
$ git push origin :v$VERSION
$ 
$ # Now tag and push
$ git tag v${VERSION}
$ git push --tags

Deploy Site to Github Pages

From cloned repo:

$ git checkout gh-pages
$ 
$ # Create specific version site
$ mkdir -p $VERSION
$ 
$ # Copy the over to version-specific and default sites
$ rsync -av target/site/ $VERSION
$ rsync -av $VERSION/* .
$ 
$ git add .
$ 
$ # For operational release
$ git commit -m "Deploy v$VERSION docs"
$ 
$ # For release candidate
$ git commit -m "Deploy ${VERSION}-rc${CANDIDATE_NUM} docs"
$ 
$ git push origin gh-pages

Update Versions For Development

Update pom.xml with the next SNAPSHOT version either manually or using Github Versions Plugin.

For RELEASE CANDIDATE, ignore this step.

$ git checkout main
$ 
$ # For release candidates, skip to push changes to main
$ VERSION=1.16.0-SNAPSHOT
$ mvn versions:set -DnewVersion=$VERSION
$ git add pom.xml
$ git commit -m "Update version for $VERSION development"
$ 
$ # Push changes to main
$ git push -u origin main

Complete Release in Github

Currently the process to create more formal release notes and attach Assets is done manually through the Github UI but should eventually be automated via script.

NOTE: Be sure to add the tar.gz and zip from the target/ directory to the release assets, and use the CHANGELOG generated above to create the RELEASE NOTES.

Snapshot Release

Deploy software to Sonatype SNAPSHOTS Maven repo:

$ # Operational release
$ mvn clean site deploy

Maven JAR Dependency Reference

This subsection lists reference URLs for the various releases.

Operational Releases

https://search.maven.org/search?q=g:gov.nasa.pds%20AND%20a:mi-label&core=gav

Snapshots

https://oss.sonatype.org/content/repositories/snapshots/gov/nasa/pds/mi-label/

If you want to access snapshots, add the following to your ~/.m2/settings.xml:

<profiles>
  <profile>
     <id>allow-snapshots</id>
     <activation><activeByDefault>true</activeByDefault></activation>
     <repositories>
       <repository>
         <id>snapshots-repo</id>
         <url>https://oss.sonatype.org/content/repositories/snapshots</url>
         <releases><enabled>false</enabled></releases>
         <snapshots><enabled>true</enabled></snapshots>
       </repository>
     </repositories>
   </profile>
</profiles>

πŸ“ƒ License

The project is licensed under the Apache version 2 license. Or it isn't. Change this after consulting with your lawyers.

mi-label's People

Contributors

dependabot[bot] avatar jordanpadams avatar jpl-jengelke avatar mcayanan avatar nutjob4life avatar pdsen-ci avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

jpl-jengelke

mi-label's Issues

Resource leaks and bad code identified per code scanning

Checked for duplicates

Yes - I've already checked

πŸ› Describe the bug

See static code analysis reports

πŸ•΅οΈ Expected behavior

I expected no resource leaks

πŸ“œ To Reproduce

πŸ–₯ Environment Info

N/A

πŸ“š Version of Software Used

Latest SNAPSHOT

🩺 Test Data / Additional context

No response

πŸ¦„ Related requirements

No response

βš™οΈ Engineering Details

No response

Need ability to remove boilerplate in mi-label

One of the awesome-est features of mi-label is that it will collapse out elements that have no children. This is what gives it much of its power.

But sometimes what happens is, you have very complex decisions about whether to include elements or not, and then some basic boilerplate. The boilerplate keeps the element in even though there are no other children. It’s not practical to replicate all those checks just to get rid of the boilerplate. So if we had some symbol that indicated β€œkeep this element if you have other siblings but don’t let it prevent you from collapsing out the element if not” that would be extremely helpful.

For example, this fails validation:

          <geom:Articulation_Device_Parameters>
            <geom:device_id>RSM</geom:device_id>
            <geom:device_name>REMOTE SENSING MAST</geom:device_name>
          </geom:Articulation_Device_Parameters>

because it doesn't have any substantive children (angles, temperatures, etc). in this case, all the children collapsed away because they were NULL in the input (it was some very weird data). If we could get the whole Articulation_Device_Parameters to collapse away too, that would be great... but the boilerplate name prevents that.

One way to do this might be to enclose such "weak" children in double <<>> or in [] or some such. So the template might look something like:

          <geom:Articulation_Device_Parameters>
            <<geom:device_id>>$ADP_ID<</geom:device_id>>
            <<geom:device_name>>$ADP_NAME<</geom:device_name>>
            #addDeviceAngles()
            #addDeviceComponentStates()
            #addDeviceTemperatures()
          </geom:Articulation_Device_Parameters>

etc. Or pick your own syntax. If any of the macros expand to something substanstive, the "weak" entries stay in (replaced by proper brackets of course). If they all go away, the weak children go away too, and along with them the rest of the Articulation_Device_Parameters group.

For simple cases, you can add the same conditional you use for the children, to the boilerplate. But even in the simple case that's a maintenance nightmare. In more complex cases like with ADP, where there are loops and many levels of nested macros, it really is not practical to repeat or remember the conditionals.

There are literally dozens of places we could use this in the current templates, and it would go a long way to making them much more readable and usable (and maintainable!!).

Transform error when calling Generator directly

Checked for duplicates

Yes - I've already checked

πŸ› Describe the bug

When I try to run Transform tool with example from Transform, examples/N1727539187_1.LBL, I get a NullPointerException when trying to execute generate() method.

πŸ•΅οΈ Expected behavior

I expected the tool to execute successfully.

πŸ“œ To Reproduce

$ transform-1.12.0-SNAPSHOT/bin/transform --target transform-1.12.0-SNAPSHOT/examples/N1727539187_1.LBL --format-type pds4-label
java.lang.NullPointerException: Cannot invoke "org.apache.velocity.VelocityContext.put(String, Object)" because "this.context" is null
	at gov.nasa.pds.imaging.generate.Generator.setPDSObject(Generator.java:688)
	at gov.nasa.pds.transform.util.Utility.generate(Utility.java:312)
	at gov.nasa.pds.transform.product.Pds3LabelTransformer.transform(Pds3LabelTransformer.java:107)
	at gov.nasa.pds.transform.product.Pds3LabelTransformer.transform(Pds3LabelTransformer.java:121)
	at gov.nasa.pds.transform.TransformLauncher.doTransformation(TransformLauncher.java:428)
	at gov.nasa.pds.transform.TransformLauncher.processMain(TransformLauncher.java:461)
	at gov.nasa.pds.transform.TransformLauncher.main(TransformLauncher.java:475)
outputs = [/Users/jpadams/proj/pds/pdsen/workspace/transform/n1727539187_1.xml]
PDS Transform Tool Log

Version                     Version 1.12.0-SNAPSHOT
Time                        Mon, Oct 02 2023 at 02:37:05 PM
Target                      [file:/Users/jpadams/proj/pds/pdsen/workspace/transform/transform-1.12.0-SNAPSHOT/examples/N1727539187_1.LBL]
Output Directory            /Users/jpadams/proj/pds/pdsen/workspace/transform
Index                       1
Format Type                 pds4-label

INFO:   [/Users/jpadams/proj/pds/pdsen/workspace/transform/transform-1.12.0-SNAPSHOT/examples/N1727539187_1.LBL] Transforming label file: /Users/jpadams/proj/pds/pdsen/workspace/transform/transform-1.12.0-SNAPSHOT/examples/N1727539187_1.LBL
INFO:   [/Users/jpadams/proj/pds/pdsen/workspace/transform/transform-1.12.0-SNAPSHOT/examples/N1727539187_1.LBL] Finished transforming PDS3 label '/Users/jpadams/proj/pds/pdsen/workspace/transform/transform-1.12.0-SNAPSHOT/examples/N1727539187_1.LBL' to a PDS4 label '/Users/jpadams/proj/pds/pdsen/workspace/transform/n1727539187_1.xml'

πŸ–₯ Environment Info

  • mi-label 1.4.0-SNAPSHOT

πŸ“š Version of Software Used

No response

🩺 Test Data / Additional context

No response

πŸ¦„ Related requirements

Refs NASA-PDS/transform#36

βš™οΈ Engineering Details

No response

[SECURITY] Update Reference equality test on strings

Vulnerability

Determine if the findings about string object comparison issues do exist and determine the potential consequences if they are.

File Warning Line Warning Text Notes
mi-label/src/main/java/gov/nasa/pds/imaging/generate/label/JsonLabel.java 536 Reference equality test on strings (CWE-597) Confirm the intention is to compare the object may need to test with .equals
mi-label/src/main/java/gov/nasa/pds/imaging/generate/label/JsonLabel.java 538 Reference equality test on strings (CWE-597) Confirm the intention is to compare the object may need to test with .equals
mi-label/src/main/java/gov/nasa/pds/imaging/generate/label/JsonLabel.java 638 Reference equality test on strings (CWE-597) Confirm the intention is to compare the object may need to test with .equals
mi-label/src/main/java/gov/nasa/pds/imaging/generate/label/JsonLabel.java 643 Reference equality test on strings (CWE-597) Confirm the intention is to compare the object may need to test with .equals

update software to better handle and cleanup non-UTF8 characters

Archive.zip

% generate -version

Generate Tool
Version 0.18.2
Release Date: 2019-10-29 18:09:03

Copyright 2012-2017, by the California Institute of Technology.
ALL RIGHTS RESERVED. United States Government Sponsorship acknowledged.
Any commercial use must be negotiated with the Office of Technology Transfer
at the California Institute of Technology.

This software is subject to U. S. export control laws and regulations
(22 C.F.R. 120-130 and 15 C.F.R. 730-774). To the extent that the software
is subject to U.S. export control laws and regulations, the recipient has
the responsibility to obtain export licenses or other export authority as
may be required before exporting such information to foreign countries or
providing access to foreign nationals.

% generate -p genImgJNC.LBL -t genImgJNC.vm
[Fatal Error] :1:1: Content is not allowed in prolog.

Error applying XSLT to output XML. Verify label and template are correctly formatted.
Content is not allowed in prolog.
Outputting without formatting.

% head -1 genImgJNC.xml

%

pds-generate crashes when runs out of heap memory

PDS4_SPACECRAFT_1H00_1000.sch.gz
PDS4_SPACECRAFT_1H00_1000.xsd.gz
Template_NH_Pepssi_data.vm.gz
pepssi_example.sh.gz

πŸ› Describe the bug

When trying to run pds-generate on 1274 PDS3 lbl files that are more than 2 MB each
it crashes with the below error output

pds-generate -p data//.lbl -t Template_test.vm
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.antlr.runtime.ANTLRReaderStream.load(ANTLRReaderStream.java:78)
at org.antlr.runtime.ANTLRInputStream.(ANTLRInputStream.java:68)
at org.antlr.runtime.ANTLRInputStream.(ANTLRInputStream.java:52)
at org.antlr.runtime.ANTLRInputStream.(ANTLRInputStream.java:48)
at org.antlr.runtime.ANTLRInputStream.(ANTLRInputStream.java:40)
at gov.nasa.pds.tools.label.parser.DefaultLabelParser.parseLabel(DefaultLabelParser.java:265)
at gov.nasa.pds.tools.label.parser.DefaultLabelParser.parseLabel(DefaultLabelParser.java:249)
at gov.nasa.pds.tools.label.parser.DefaultLabelParser.parseLabel(DefaultLabelParser.java:154)
at gov.nasa.pds.imaging.generate.readers.ProductToolsLabelReader.parseLabel(ProductToolsLabelReader.java:302)
at gov.nasa.pds.imaging.generate.label.PDS3Label.setMappings(PDS3Label.java:463)
at gov.nasa.pds.imaging.generate.GenerateLauncher.query(GenerateLauncher.java:225)
at gov.nasa.pds.imaging.generate.GenerateLauncher.main(GenerateLauncher.java:293)

It should be noted that no PDS4 xml labels were produced.
When test were run using small numbers of products (1, 2, 5, and 10) it was noted that the percentage of memory
used from the beginning increased with the number of products included, increased as processing time passed,
and was not seen to decrease.

πŸ“œ To Reproduce

Steps to reproduce the behavior:

  1. Go to 'https://pdssbn.astro.umd.edu/holdings/nh-a-pepssi-3-kem1-v5.0/dataset.shtml'
  2. Click on 'DOWNLOAD (1010.3 MB)'
  3. Create a working directory
  4. Download the checksum and tgz file
  5. Place them in the working directory
  6. Confirm the checksum and open the tgz file
  7. Create a link from the working directory to the nh-a-pepssi-3-kem1-v5.0/data directory
    ln -s nh-a-pepssi-3-kem1-v5.0/data data
  8. Download the 4 attached files and unzip them.
  9. Place the PDS4_SPACECRAFT_1H00_1000.sch and PDS4_SPACECRAFT_1H00_1000.xsd files in the working directory
  10. Place the Template_NH_Pepssi_data.vm file in the working directory
  11. Update the PDS4_SPACECRAFT_1H00_1000.sch and PDS4_SPACECRAFT_1H00_1000.xsd lines in Template_NH_Pepssi_data.vm with the correct local path
  12. Place pepssi_example.sh in working directory (I put the command in this small script since this form appears to be corrupting the command line)
  13. Update pepssi_example.sh with the correct local path
  14. run pepssi_example.sh Note: suggest running the script via cron since it may take a while
  15. Be sure to note if there are any xml files in the same data subdirectories as the lbl and fit files
  16. See error17.
  17. To see the behavior with increased memory use, you can create new directories subset1, subset2, subset5, and subset10. Based on the number in the directory name, copy subdirectories from nh-a-pepssi-3-kem1-v5.0/data/ to the subset directories. Redirect the data link. Watch the information provided by top.

πŸ•΅οΈ Expected behavior

PDS4 xml labels to be produced for each PDS3 lbl label and the program to not crash.

πŸ“š Version of Software Used

/sbnops/lcltools/bin/pds-generate -V

gov.nasa.pds:mi-label
Version 1.2.2
Release Date: 2022-04-14 05:41:49

🩺 Test Data / Additional context

See Reproduce section. Note: that is the latest public version of the dataset we are having issues with. It only has 847 products versus 1274 products in the version in review. The public version can run on the server we are using for the migration work, but will crash on a laptop. The in review version is half again larger and crashes on the server. Specifications of the computer will matter, so it is suggested to not use your best computer for the testing.

🏞Screenshots

πŸ–₯ System Info

  • OS: [e.g. iOS] RHEL 3.10.0-1160.66.1.el7.x86_64 number 1 SMP Wed May 18 16:02:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  • Browser [e.g. chrome, safari] n/a
  • Version [e.g. 22]

πŸ¦„ Related requirements

βš™οΈ Engineering Details

Windows version will not run because of error in generate.bat

Describe the bug
The Windows version of mi-label version 1.1.2 does not work out-of-the-box. It is currently necessary for the user to manually edit line 52 of mi-label-1.1.2\bin\generate.bat, replacing the erroneous path in that line with "%LIB_DIR%\mi-label-*.jar", before any call to generate will complete successfully.

To Reproduce
Steps to reproduce the behavior:

  1. On a machine running Windows 10, follow the installation and configuration steps specified on this page to install the mi-label version 1.1.2. (I downloaded it from https://github.com/NASA-PDS/mi-label/releases/download/v1.1.2/mi-label-1.1.2-bin.zip.) For the purposes of these steps, the installation directory is assumed to be C:\pds\mi-label-1.1.2.
  2. Ensure that the user's path does not contain the bin directory of any previous install of mi-label or generate, so that the generate command will run the file C:\pds\mi-label-1.1.2\bin\generate.bat.
  3. Open an instance of Powershell and run the command generate -V.
  4. See the error Cannot find Generate Tool jar file in C:\pds\MI-LAB~1.2\bin\..\lib, after which the program halts without completing the request.

Expected behavior
The version of the mi-label package should be displayed.

Version of Software Used
gov.nasa.pds:mi-label
Version 1.1.2
Release Date: 2020-05-11 20:38:25

Desktop (please complete the following information):

  • OS: Windows 10
  • Version 1.1.2

VICAR reader not working with portion of FITS label

Jordan,

I updated from mi-label-1.2.1 to mi-label-1.2.2. No I got the following bug when trying to migrate some (but not all) PDS3 to PDS4 labels.

Unfortunately I can't attach one of thePDS3 label here.

java.lang.NullPointerException
at gov.nasa.pds.imaging.generate.label.PDS3Label.setMappings(PDS3Label.java:437)
at gov.nasa.pds.imaging.generate.GenerateLauncher.query(GenerateLauncher.java:225)
at gov.nasa.pds.imaging.generate.GenerateLauncher.main(GenerateLauncher.java:293)

Engineering Details

After initial triage, it appears this issue is specifically related to the PDS3LabelReader (VICAR), and, more specifically, the EXTENSION_RALPH_HK_TABLE portion of the label.

As noted below, a workaround has been identified per the product tools library.

unable to run generate.bat after switchover to mi-label

Describe the bug
Issue when trying to run generate.bat

I just downloaded the mi-label tool (version 1.1.2) and think there is a bug in the generate.bat file:
 
:: Check for dependencies.
if exist "%LIB_DIR%\generate-*.jar" (
set GENERATE_JAR=%LIB_DIR%\mi-label-*.jar
) else (
echo Cannot find Generate Tool jar file in %LIB_DIR%
goto END
)
 
I think the "If exist" statement should have 'mi-label-*.jar' and not 'generate-*.jar'.
 
Editing my downloaded copy of the *.bat file gets the tool to work. I have only tried 'generate -V' so far. But I wanted to let you know before I forgot.

found by @edguinness and @sslavney

Change default reader from VICAR to PDS3 Product Tools

πŸ’‘ Description

After encountering numerous bugs related to VICAR's ability to read PDS3 labels, it appears the PDS3 Product Tools library may be a more stable solution for reading PDS3 and ODL labels. This task will switch the default reader to be PDS3 Product Tools instead of VICAR.

Product-Tools parsing mode cannot parse units

πŸ› Describe the bug

The Product-Tools parser mode is unable to return the units of any quantities in a PDS3 label.

It matters to us at PDS-GEO that Product-Tools mode be able to handle units because it supports some PDS3 features, like {}-delimited arrays, which the default VICAR parsing mode does not. There are many PDS3 labels that have both those features and units, but currently no mode of MILabel that can be used to handle all these features at once.

πŸ“œ To Reproduce

Steps to reproduce the behavior:

  1. Create a file problem_illustration.lbl with these contents:
PDS_VERSION_ID                    = PDS3

PROPERTY_WITH_UNIT = 29.7245241213 <DEGREE>
  1. Create a file template.vm with these contents:
The property with a unit:
The unit: $label.getUnits('PROPERTY_WITH_UNIT')
The value: $label.get('PROPERTY_WITH_UNIT')
  1. Switch MILabel to Product-Tools mode via the instructions at "Advanced Usage"/"Changing the Underlying Parser" of this document.
  2. Run MILabel with the above files with pds-generate -t template.vm -p problem_illustration.lbl
  3. Observe the following output in the file problem_illustration.xml:
The property with a unit:
The unit: none
The value: 29.7245241213

πŸ•΅οΈ Expected behavior

The desired contents of problem_illustration.xml are:

The property with a unit:
The unit: DEGREE
The value: 29.7245241213

πŸ“š Version of Software Used

Version 1.2.0 (latest stable version at this time)

🩺 Test Data / Additional context

N/A

🏞Screenshots

N/A

πŸ–₯ System Info

  • OS: Windows 10 Pro build 19043.1237
  • Java Runtime Environment: 1.8.0_301

πŸ¦„ Related requirements

βš™οΈ Engineering Details

imageInputStream is null, giving file size of .LBL instead of .IMG

Describe the bug
When running generate on a .LBL file in debug mode, from FileSize.java see this line:

Debugger.debug("file="+this.file+" imageInputStream="+this.imageInputStream);

(see internal ticket for exact location)

that the imageInputStream is null. This results in $generate.file_size returning the size of the .LBL file instead of its corresponding .IMG file.

To Reproduce
Steps to reproduce the behavior:

  1. Go to https://github.jpl.nasa.gov/PDSIMG/pdsimg-mi
  2. Clone the repo
  3. Checkout branch nsyt-ida
  4. Install generate 0.15.0 or 0.18.2
  5. From the repo root, run
generate -d -t schemas/juno/sample_products/juno_jnc_template.xml -p schemas/juno/sample_products/JNCE_2011238_00A00002_V01.LBL

Expected behavior
It is expected that the imageInputStream not be null so that the file size of the .IMG file be returned.

Version of Software Used
0.15.0 and 0.18.2

Screenshots
See internal ticket

Desktop (please complete the following information):

  • OS: macOS 10.14.6 Mojave

@decesare

Unable to read in ODL label metadata when a paren is in a long description field

πŸ› Describe the bug

From user:

Hey Jordan,

For some reasons there is a bug when there is a close parenthesis within a set of double quote.

I attached the label. In this case the problem is within the NOTE:

NOTE                           = "                                            
 a)                                                                           
   MAGNETIC_COORDINATE_SYSTEM : INSTRUMENTCOORDS                              
 b)                                                                           
   THE VALUES OF THE KEYWORDS SC_SUN_POSITION_VECTOR,                         
   SC_TARGET_POSITION_VECTOR AND SC_TARGET_VELOCITY_VECTOR,                   
   ARE RELATED TO THE ECLIPJ2000 REFERENCE FRAME.                             
   SUB_SPACECRAFT_LATITUDE AND SUB_SPACECRAFT_LONGITUDE                       
   ARE NORTHERN LATITUDE AND EASTERN LONGITUDE IN THE STANDARD                
   PLANETOCENTRIC IAU_<TARGET_NAME> FRAME. ALL VALUES ARE COMPUTED            
   FOR THE TIME T= START_TIME.                                                
   DISTANCES ARE GIVEN IN <KM> VELOCITIES IN <KM/S>, ANGLES IN <DEG>          
 c)                                                                           
   LBL & TAB FILE HAVE BEEN GENERATED BY S/W: GEN_CAL_DATA, VERSION V20180123 

Because of the a) b) c) when I generate the PDS4 label, it doesn’t read/find the next line after the double quote.
In this case ^TABLE             = "RPCMAG080901T0002_RAW_HK.TAB"  

The PDS4 label will return:

<File>
      <file_name>$label.PTR_TABLE</file_name>
 </File>

If I delete the close parentheses in the NOTE. Then the PDS4 label will return:

    <File>
      <file_name>RPCMAG080901T0002_RAW_HK.TAB</file_name>
    </File>

What can I do to fix this problem?

Attached is the PDS3 label.

Thank you in advance
Adeline

πŸ“œ To Reproduce

$ ./pds-generate -p ../rpcmag080901t0002_raw_hk.lbl -t ../examples/example1/template_example.vm -d

output looks like this:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>


<Product_Array_2D_Image xmlns="http://pds.nasa.gov/schema/pds4/pds/v03" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://pds.nasa.gov/schema/pds4/pds/v03 Product_Array_2D_Image_0311B_MPF_IMP_EDR.xsd ">
  <!-- Scenario 1 - Hard-coded values  -->
  <Data_Standards>
    <dd_version_id>0311B_20110709</dd_version_id>
  </Data_Standards>
  <!-- Scenario 2 - Base element  -->
  <Subject_Area>
    <target_name>a)</target_name>
  </Subject_Area>
  <!-- Scenario 7 - Generated Values  -->
  <File_Area>
    <md5_checksum>$label.PTR_TABLE</md5_checksum>
  </File_Area>
</Product_Array_2D_Image>

πŸ•΅οΈ Expected behavior

πŸ“š Version of Software Used

generate -V,--version

gov.nasa.pds:mi-label
Version 1.1.2
Release Date: 2020-05-11 20:38:25

🩺 Test Data / Additional context

rpcmag080901t0002_raw_hk.lbl.txt
template_example.vm.txt

🏞Screenshots


πŸ¦„ Applicable requirements

Engineering Details

This issue has been replicated with mi-label v1.2.0 and is a problem with the VICAR ODL reader. A workaround has been identified to use the product tools library:

In the bin/pds-generate script, update the following line:
Was:

${JAVA_CMD} -jar ${GENERATE_JAR} "$@"

Is:

${JAVA_CMD} -Dpds.generate.parser.type="product-tools" -jar ${GENERATE_JAR} "$@"

Re-running generate gives expected output:

<Product_Array_2D_Image xmlns="http://pds.nasa.gov/schema/pds4/pds/v03" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://pds.nasa.gov/schema/pds4/pds/v03 Product_Array_2D_Image_0311B_MPF_IMP_EDR.xsd ">
  <!-- Scenario 1 - Hard-coded values  -->
  <Data_Standards>
    <dd_version_id>0311B_20110709</dd_version_id>
  </Data_Standards>
  <!-- Scenario 2 - Base element  -->
  <Subject_Area>
    <target_name>a)                                                                           
   MAGNETIC_COORDINATE_SYSTEM : INSTRUMENTCOORDS                              
 b)                                                                           
   THE VALUES OF THE KEYWORDS SC_SUN_POSITION_VECTOR,                         
   SC_TARGET_POSITION_VECTOR AND SC_TARGET_VELOCITY_VECTOR,                   
   ARE RELATED TO THE ECLIPJ2000 REFERENCE FRAME.                             
   SUB_SPACECRAFT_LATITUDE AND SUB_SPACECRAFT_LONGITUDE                       
   ARE NORTHERN LATITUDE AND EASTERN LONGITUDE IN THE STANDARD                
   PLANETOCENTRIC IAU_&lt;TARGET_NAME&gt; FRAME. ALL VALUES ARE COMPUTED            
   FOR THE TIME T= START_TIME.                                                
   DISTANCES ARE GIVEN IN &lt;KM&gt; VELOCITIES IN &lt;KM/S&gt;, ANGLES IN &lt;DEG&gt;          
 c)                                                                           
   LBL &amp; TAB FILE HAVE BEEN GENERATED BY S/W: GEN_CAL_DATA, VERSION V20180123</target_name>
  </Subject_Area>
  <!-- Scenario 7 - Generated Values  -->
  <File_Area>
    <md5_checksum>RPCMAG080901T0002_RAW_HK.TAB</md5_checksum>
  </File_Area>
</Product_Array_2D_Image>

Executing `pds-generate` does not output anything

Checked for duplicates

Yes - I've already checked

πŸ› Describe the bug

I tried executing the tool on some LOLA data and there was no log information or output files.

πŸ•΅οΈ Expected behavior

I expected the tool to output something...

πŸ“œ To Reproduce

pds-generate --template lola_900Crdr_template.xml LOLARDR_230741421.LBL

πŸ–₯ Environment Info

No response

πŸ“š Version of Software Used

3.5.0-SNAPSHOT

🩺 Test Data / Additional context

mi-label-bug.zip

πŸ¦„ Related requirements

No response

βš™οΈ Engineering Details

No response

.toLowerCase mi-label-1.2.1

Hey Jordan,

In the vm file I use .toLowerCase(). For example
$label.TARGET_TYPE.substring(0,1)$label.TARGET_TYPE.substring(1).toLowerCase()

In the previous version it would modify for example COMET to Comet.

In the new mi-label-1.2.1 it doesn't understand the .toLowerCase() command:
$label.TARGET_TYPE.substring(0,1)$label.TARGET_TYPE.substring(1).toLowerCase()

Adeline

update docs with more complex keyword queries

Is your feature request related to a problem? Please describe.

I have a question about the apache Velocity tool

In PDS3 I have: ROSETTA:ROLIS_ISB_FLAGS

I tried
rolis:flags$label.ROSETTA:ROLIS_ISB_FLAGS</rolis:flags>

I got the error : Null reference [template 'template_rolis.vm', line 95, column 42] : $label.ROSETTA cannot be resolved.

The problem seems to be the :

I can’t find a way around it.

Do you know who I could ask for help?

Thank you in advance

Adeline

Describe the solution you'd like
Solution:

<rolis:flags>$label.get("ROSETTA:ROLIS_ISB_FLAGS")</rolis:flags>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.