dmtf / redfish-tools Goto Github PK
View Code? Open in Web Editor NEWDMTF-produced tools to support the Redfish Specification
License: Other
DMTF-produced tools to support the Redfish Specification
License: Other
JSON schema draft-06 renamed "id" to "$id". Redfish schemas (except for the redfish-extensions) did not populate the "id" property.
This property is useful for providing the schema name/version within the JSON document, and is now considered "best practice" for JSON schemas.
Add a config option to specify a folder that contains JSON payload examples, and use a filename format to automatically match examples with schemas and Action definitions. The current process uses individual entries in the Input file to specify examples - this would remove that burden to keep those entries up to date.
Suggest a simple naming structure that ties to Schema Name (open to easier / better suggestions):
e.g.
This style also makes updates to the examples simple by only using major version.
Obviously the current tag/include method should be retained as well.
If the json output directory doesn't exist the tool should just create it.
In the Property Tables and Parameters tables for Actions, it would be good to have those properties/parameters that are Required to be tagged in the document.
Required parameters will contain: "requiredParameter": true
Required properties will be listed in the schema within the "required"[] array at the object/resource level.
The units of measure as shown in the data type column come from the schemas (as designed). But as these appear fairly infrequently in the resulting document, the abbreviations can be confusing without context.
Adding a simple string-replacement table for common units would improve readability:
Examples
Current: number (s)
Replace: number (seconds)
Current: number (Mb/s)
Replace: number (Mbits/second)
For better usability to debug why an XML file failed the validator, the output should contain a file name and line number to identify where to start looking for the issue.
This can be seen with the Swordfish Capacity schema. The Capacity schema contains multiple complex types as well as an entity type definition.
The doc generator is currently extracting the entity type "CapacitySource" and presenting it as the namespace "Capacity".
In this particular schema, the Capacity structure is a complex type (which is currently ignored by the doc generator, a separate issue to address), so the CapacitySource entity is incorrectly presented as Capacity instead.
(It is unclear how the doc generator behaves if there are multiple entity types defined within the same namespace and/or if the complex type inclusion here is triggering this behavior. Algorithmically this should be evaluated for correctness, or incorrectness.)
Hi DMTF,
Could you design a centralized "Brokers" redfish API to consolidate the sensor readings?
If I want to get power consumption, thermal, Intel ME readings... etc., I have to invoke the Systems or Chassis API subcomponent many times. Is it possible to design a "broker" settings to deliver sensor reading value for management software in the future? It will be more convenient to monitor servers.
Sorry for that I don't know how to give my feedback, so I leave message here.
Sincerely,
John
No default value used in get() for "enumDescriptions".
Previously, enumerations that had been deprecated were not detectable programmatically via JSON schemas. The support was added for a "enumDeprecated" schema annotation. The generator can use this to determine both that an enum has been deprecated, and also what version that occurred.
From section 4.5 of the OData CSDL spec:
Vocabulary terms can, in addition, use
· Edm.AnnotationPath
· Edm.PropertyPath
· Edm.NavigationPropertyPath
as the type of a primitive term, or the type of a property of a complex type that is exclusively used as the type of a term.
Org.OData.Capabilities.V1.xml has the following code
<ComplexType Name="NavigationPropertyRestriction">
<Property Name="NavigationProperty" Type="Edm.NavigationPropertyPath">
<ComplexType Name="NavigationRestrictionsType">
<Property Name="Navigability" Type="Capabilities.NavigationType">
<Annotation Term="Core.Description" String="Supported Navigability"/>
</Property>
<Property Name="RestrictedProperties" Type="Collection(Capabilities.NavigationPropertyRestriction)"/>
The key being that NavigationPropertyRestriction is a ComplexType which uses Edm.NavigationPropertyPath as the type of a property. The last line of the CSDL snippit states in this case the complex type is exclusively used as the type of a term. In the same file the NavigationPropertyRestriction ComplexType is used as the type of a property which seems to violate the exclusive use as a type of a term rule.
Per conversation with @jautor and @mraineri , the current design excludes inclusion of ComplexTypes with the rationale that these are used pervasively through the system and that it would not be ideal to have the type definitions replicated throughout the documentation.
One option suggested is to manually add the definitions to a common header section to address this. However, in the Swordfish schema set, we have 8 - 10 complex types that are used throughout the system, and both manually documenting these, as well as ensuring that the documentation for these are kept up-to-date is an onerous prospect.
I'd like to request that the complex type definitions be extracted programmatically , and to address the concerns about replicating them, that a new section be added for "Common Structures" (or pick a name you like better) to host the definitions of the common structures.
The list of missing types for the JSON schema generator:
• Edm.Duration
• Edm.NavigationPropertyPath
• Edm.PropertyPath
• Edm.SByte
• Edm.TimeOfDay
Hello,
When trying to convert a set of csdl files generated by Yang-to-Redfish into json we get some errors:
$ python csdl-to-json.py --input ./interfaces/output_dir --output ./output_dir
Generating JSON for: SessionService_v1.xml
Generating JSON for: MessageRegistry_v1.xml
Generating JSON for: openconfig_if_ethernet.ethernet_top_v1.xml
-- ERROR: Could not resolve reference to "ethernet" for "ethernet.ethernet"
-- Errors detected while generating ./output_dir/openconfig_if_ethernet.ethernet_top.v1_0_0.json; not creating file
...
I am attaching a tar.gz with the CSDL files.
I believe that the problem is that the function 'csdl_type_to_json_type' in file csdl-to-json.py can't handle types that are named using an alias, like for example in the error above, the 'ethernet' alias is defined as
<edmx:Reference Uri="http://redfish.dmtf.org/schemas/v1/openconfig_if_ethernet.ethernet_top.ethernet_v1.xml">
<edmx:Include Namespace="openconfig_if_ethernet.ethernet_top.ethernet" Alias="ethernet"/>
</edmx:Reference>
Could you confirm that?
Thanks,
andre
When converting output of yang to csdl converter tool to json schema, all of the Property fields are missing in the resultant json . These errors are shown when running:
Generating JSON for: openconfig_interfaces.interfaces.interface.hold_time.config_v1.xml
-- ERROR: Missing "Type" attribute for tag "Property"
-- ERROR: Could not resolve reference to "UNKNOWN_ATTRIB" for "UNKNOWN_ATTRIB"
-- ERROR: Missing "Type" attribute for tag "Property"
-- ERROR: Could not resolve reference to "UNKNOWN_ATTRIB" for "UNKNOWN_ATTRIB"
-- Errors detected while generating ./interfaces_out/openconfig_interfaces.interfaces.interface.hold_time.config.v1_0_0.json; not creating file
Generating JSON for: openconfig_interfaces.interfaces.interface.state_v1.xml
...
There are a few instances in the Redfish schema where an object is defined that includes no Redfish-defined 'properties', but rather, a pattern for the names of properties that are defined by the implementation. This occurs in the Message Registry and Attribute Registry.
Because no "properties" are defined in the JSON schema, the doc generator doesn't report any properties in the output for that object. However, in the case of Message Registry, the pattern property is also an object, and there are Redfish-defined properties within (which are not included in the output).
Suggest that when the doc generator finds a "patternProperties" of type 'object', it should produce an entry for the property with a generic name like "(pattern)" and include the Pattern specified in the description. Then include any properties defined for that object as usual.
The example for this is in MessageRegistry.json where "Messages" is an object that includes a $ref to "MessageProperty", which defines the "patternProperites" that contains the properties for a Message.
CSDL and JSON use different formats for the strings. So far, we've only added escaping slashes. We need to double check if there are any other potential conversion issues.
Hi there,
I'm working on a Proof of Concept solution which is using Swordfish + Redfish CSDL files to simulate some SNIA Swordfish scenarios. This PoC is based on Olingo Library.
We're trying to use some DMTF Redfish tools validate our PoC.
One of them is OData CSDL Validator. When we input our $metadata response, we're getting this error:
Reference: http://docs.oasis-open.org/odata/odata/v4.0/errata03/csd01/complete/vocabularies/Org.OData.Core.V1.xml->Namespace Org.OData.Core.V1 has already been included
We have this name in the reference and then in the schema. But I think this error is a false positive.
I ran it thorugh OData Validation Tool (https://github.com/OData/ValidationTool/) and I didn´t get this issue.
I've posted the metadata XML here:
https://gist.github.com/fera2k/c3a6b30cf6917bd73f9452f53aa0abf8
The Schema URI Mapping directive only allows translation of local-path schema objects to URI-path objects. It doesn't allow URI-path schema to be supplemented by local-path schema. For example:
Schema URI Mapping
Local-repo: http://redfish.dmtf.org/schemas/swordfish/v1 ../../SSM/json_schema/
Will not expand the parsed schema to be the union of the remote definitions (found via URI) and the local defintions (found by file path).
When parsing an entity property, if precision is present and scale is not, there is a NullType python error thrown. this is because of a check "if self.scale > self.precision:" that does not check if scale is present prior to the evaluation. NullType > int
OData 4.0 spec allows precision, scale, both or neither. 6.2.4 If no value is specified, the Scale facet defaults to zero
if (self.precision != None and self.scale != None): if self.scale > self.precision: raise SchemaError("Scale facet {} > Precision facet {}".format(self.scale, self.precision))
There are a complex set of requirements and recommendations around specification of EntityContainers and EntitySets (including emplied EntitySets for NavigationProperties with ContainsTarget=True.
Validation should include the following rules:
The tool needs to be extended to support http://redfish.dmtf.org/schemas/"customer" (e.g., swordfish) namespace or a completely customizable namespace, that can also refer to the http://redfish.dmtf.org/schemas/ as appropriate.
The SNIA swordfish CSDL schemas are available for testing the feature enhancement.
Expand to include extracting content for more complex / hierarchical schema. Swordfish can provide examples (class of service hierarchy) and inclusion of complex types that are currently excluded from the doc generator output.
Property table should do case-insensitive sorting of property names (we have only a few exceptions where the first letter of a property is lower-case).
Example in NetworkDeviceFunction with "iSCSI" object coming at the end of the table, instead of with the "I" properties.
These are present in CSDL (Validation.Minimum and Validation.Maximum), and the equivalent definitions "minimum" and "maximum" exist in JSON schema v4.
Hello there.
I'm getting this issue when validating Redfish CSDL in my metadata response
DataServices->Schema:Resource.v1_0_0->EntityType:ReferenceableMember->Key->PropertyRef:MemberId->segment MemberId in path Resource.v1_0_0.ReferenceableMember/MemberId not found
In the namespace "Resource.v1_0_0" there is the EntityType "ReferenceableMember".
It defines a Key, with a PropertyRef poiting to a Property defined in the baseType "Resource.ReferenceableMember".
Isn´t it suppose to consider also the properties on the base type ?
please, consider this metadata gist: (line 46561)
https://gist.github.com/fera2k/c3a6b30cf6917bd73f9452f53aa0abf8
from OData CSDL spec section 11.1.3
"Annotations MAY be applied to a type definition, and are considered applied wherever the type definition is used. Applying the same annotation to a property whose type definition already defines that annotation is an error."
Org.Odata.Core.V1.xml defines the Tag TypeDefinition and attaches a Description annotation to it
<TypeDefinition Name="Tag" UnderlyingType="Edm.Boolean">
<Annotation Term="Core.Description" String="This is the type to use for all tagging terms"/>
</TypeDefinition>
There are usages of that TypeDefinition which also apply the Core.Description annotation
<Term Name="DereferenceableIDs" Type="Core.Tag" DefaultValue="true" AppliesTo="EntityContainer">
<Annotation Term="Core.Description" String="Entity-ids are URLs that locate the identified entity"/>
</Term>
<Term Name="ConventionalIDs" Type="Core.Tag" DefaultValue="true" AppliesTo="EntityContainer">
<Annotation Term="Core.Description" String="Entity-ids follow OData URL conventions"/>
</Term>
The issue is now that the Core.Description Annotation is applied twice to the Terms that are of type Core.Tag
Need a separate section in the document output to provide documentation for 'referenced' schemas that come from other sources.
This would essentially be an ability to insert schema docs from additional sources (not just the json-schema directory). Best if this ability is added as #include tags within the Input file, as these references should be file-specific, not "everything in this folder".
Issue 1995 in the spmf repo showed Attribute Registry had a duplicate property name in a complex type, but no error was reported.
I’m getting an error when generating from the latest OpenAPI spec online via openapi-generator as below.
I can see the GenerateCSRResponse section, but it doesn’t seem to be under any structure called definitions.
[asill@redfish ~]$ openapi-generator generate -g python -o redfish-test-python -i https://redfish.dmtf.org/schemas/openapi.yaml
[main] WARN io.swagger.v3.parser.OpenAPIV3Parser - Exception while reading:
java.lang.RuntimeException: Could not find definitions/GenerateCSRResponse in contents of http://redfish.dmtf.org/schemas/v1/CertificateService.v1_0_0.yaml
at io.swagger.v3.parser.ResolverCache.loadRef(ResolverCache.java:140)
at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:51)
at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema(SchemaProcessor.java:202)
at io.swagger.v3.parser.processors.SchemaProcessor.processSchema(SchemaProcessor.java:37)
at io.swagger.v3.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:52)
at io.swagger.v3.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:67)
at io.swagger.v3.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:84)
at io.swagger.v3.parser.OpenAPIResolver.resolve(OpenAPIResolver.java:49)
at io.swagger.v3.parser.OpenAPIV3Parser.readLocation(OpenAPIV3Parser.java:53)
at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:19)
at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:561)
at org.openapitools.codegen.cmd.Generate.run(Generate.java:353)
at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:62)
This seems to be referred to by this line in the openapi.yaml file:
$ref: http://redfish.dmtf.org/schemas/v1/CertificateService.v1_0_0.yaml#/definitions/GenerateCSRResponse
Typo in internally-coded text for Action Details table that refers to the "target" property as "Target" (capitalized).
The CSDL Validator displays the following message.
"Could not open http://docs.oasis-open.org/odata/odata/v4.0/errata03/csd01/complete/vocabularies/Org.OData.Core.V1.xml"
Despite the message
The validator is reporting errata on the schema file
The URI path directly from a Chrome browser.
Reproduction notes: Windows 7, Python 3.5, using validator on a single CSDL file.
Debug statements show that abs_local_path (line ~483) is properly set, and refers to a valid directory, but the isdir() test fails, causing a parsing error.
# Schema URI Mapping
## Local-repo: http://redfish.dmtf.org/schemas/swordfish/v1 ../../SSM/json_schema/
OS: Windows 7 (current patches)
Python: 3.6.1
Cygwin bash: 4.4.12(3), x86_64
If the tool doesn't find a local copy of a file it should just be able to use the online copy.
If the " is used in a Long Description annotation the it is included as simply a " (double quote) and not a " (escaped double quote) in the produced JSON.
While rare in the Redfish schema, a simple JSON array of strings which specifies enumerations does not render properly (using the HTML output, at least). "SupportedEthernetCapabilities" property in the "NetworkPort" schema is an example (and the only one I could find!).
In the published json-schema, the enum string type is included as a $ref to a type (of the same name). This produces an entry that shows object braces, and breaks this into multiple lines with the property name repeated (this may be the Type exposed, which in this cases is the same name as the property).
name | type | RW | description |
---|---|---|---|
SupportedEthernetCapabilities [ { | array | read-only | The set of Ethernet capabilities that this port supports. |
SupportedEthernetCapabilities | string(enum) | read-write(null) | See SupportedEthernetCapabilities in Property Details, below, for the possible values of this property. |
} ] |
Moving the string/enum definition inline with the property shows the entry differently, and also causes the Property Details (enum) section to show an empty ":" value for the property name.
JSON schema:
"SupportedEthernetCapabilities": {
"type": "array",
"items": {
"anyOf": [{
"type": "string",
"enum": [
"WakeOnLAN",
"EEE"
],
"enumDescriptions": {
"WakeOnLAN": "Wake on LAN (WoL) is supported on this port.",
"EEE": "IEEE 802.3az Energy Efficient Ethernet (EEE) is supported on this port."
}
},
{
"type": "null"
}
]
}
},
This produces an output like:
name | type | RW | description |
---|---|---|---|
SupportedEthernetCapabilities [ { | array | read-only | The set of Ethernet capabilities that this port supports. |
string(enum) | read-write | See in Property Details, below, for the possible values of this property. | |
read-write(null) | |||
} ] |
The output for this property should look like this:
name | type | RW | description |
---|---|---|---|
SupportedEthernetCapabilities [] | array, string(enum) | read-only | The set of Ethernet capabilities that this port supports. See SupportedEthernetCapabilties in Property Details, below, for the possible values of this property. |
tested validating schemas that do this on github under Intel RackScale
The CSDL validator should be enhanced to check that there are annotation terms in all new/existing schema mode so that the documentation can be generated.
Originally filed by @J2Hilland in the other tools repo; moved the issue since the tool has moved.
Properties that are nullable (CSDL term) are shown to have 'null' as a type in JSON schema. It the final user display, though, this complicates the type column (normally a single type) and negatively impacts readability.
Remove the 'null' type and instead, add "(null)" to the read/write, read/only column ("attributes"?), and we will add descriptive text to explain these columns - which is needed regardless.
Before:
Thingy string, null read-write This is the description for the thingy.
After:
Thingy string read-write (null) This is the description for the thingy.
This happens when an abstract ComplexType is defined in a versioned namespace. Currently observable with the Manager resource with the "ManagerService" definition.
While the doc generator lists the URIs for each resource (gathered from the schema annotations), it doesn't show the URIs for Actions, which are also specified by the v1.6 specification, but don't appear as annotations. But they can be constructed from the URIs in the schema and the Action definitions.
These should be added to the Action Details section for each action found when the URIs annotation is present.
Allow insertion of a page break tag that will render invisible in viewed HTML but would create physical page breaks when printed or converted to PDF.
I found I could insert this tag into the generated HTML:
<p style="page-break-before: always">
But I couldn't include it in the source .MD file without affecting the resulting output. (or is there a Markdown tag equivalent?).
Regex pattern for Edm.Guid does not match the OData definition. The OData Edm.Guid definition includes both upper and lower-case hex digits, which matches the definition from RFC 4122.
Enumerations that have been defined in the un-versioned JSON schema files and appear in v1.0.0, have an invalid "version added" notation: (vunversioned+)
This condition should be suppressed, but this issue will likely be addressed when support for the Version annotations is added.
"@odata.etag" annotation property is called out in the Redfish Specification but is not present in the generated json schema files (but @odata.type, id, etc. are). Separate issue opened against the odata.4.0.x schema to have it added there as well.
Hi,
Good to see a tool to validate odata xml. Now i am looking for odata json schema validate tool for schema's written in json format. Please suggest if you know any tool. Thanks
The CSDL for all of our resources contains Capabilities annotations to give guidance on things being Insertable, Updateable, or Deleteable. These terms should be converted over to JSON Schema as well for completeness.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.