Stack overlfow by converting FHIR about x2xmldatabinding

[GeritFreericks]

Some new Info:
The Error occour, when compiling as 32-Bit App.
When I use 64-Bit as output then it eats my 16 GByte RAM.
The Procedure XMLDataBindingGenerator.pas:GenerateShemaObjects
run through near a half of the XSD-Files before it stops.
Maybe a machine with 128 GByte RAM will take it, but it seems,
that there is gerenally an issue when you use so much RAM.

Greetings from germany
Gerit Freericks

[MvRens]

Hi Gerit,

Thanks for the report! I think the 16 Gb RAM usage is a consequence of the stack overflow / endless loop, not the cause. There should never be a case where that much RAM is actually required.

I do know of a few files which fail to import, usually enterprise specifications with lots of redirection generated by some framework. Unfortunately this also makes them very time-consuming to debug. If you only need a relatively small subset of the data, the way we've dealt with it at our company is to create a simplified XSD which only describes those elements. Definitely not ideal.

Considering the size of the XSD's you've linked I'm not even going to attempt to debug this. Perhaps you can reproduce the issue in a small subset of the files? Not an easy task I'm sure. If you do find the cause I would be interested though!

[GeritFreericks]

Hi MvRens,

a subset would not have the problem, because there are too many files included.
Why do I think so:
For a test I add a log-file-funktion (simply write the Schema-Names into a file)
next to the first "OutputDebugString" in XMLDataBindingGenerator.pas:GenerateSchemaObjects.
If you run the "fhir-all.xsd" it writes the name of the current object in the log-file.
The XSD-Converter writes a file and continues. By the time it consumes more and more RAM
but there are also more an more files written in the log-file.
I think you should realy just try it and look what happend.
I believe if you are using a machine with 64 GByte it is possible that it comes to an end.
But if you have 16 GByte you would see it work file after file but with so much RAM needed
that it will stop at one point or another.
XMLDataBindingGenerator.zip
You can use this file to test it very easy.

Greetings from germany
Gerit Freericks

[GeritFreericks]

Short info:
A friend with a gaming PC had made it to the end with 23 GByte RAM usage.

Greetings from germany
Gerit Freericks

[GeritFreericks]

Next short Info:
The Problem is somehow complex.
When split it up to many files (export to folder) then you can't use it because there are cyclic dependencies.
When merge in one file, then you can't use it because there are some symbols twice (maybe case-problems).
Also there are some type-definitions which are sets with items of the same name.
For example:

  TXMLQuestionnaireItemOperatorlist = (QuestionnaireItemOperatorlist_exists,
                                       QuestionnaireItemOperatorlist_,
                                       QuestionnaireItemOperatorlist_,
                                       QuestionnaireItemOperatorlist_,
                                       QuestionnaireItemOperatorlist_,
                                       QuestionnaireItemOperatorlist_,
                                       QuestionnaireItemOperatorlist_);

I don't even have an idea how to solve one of the problems.

The next problem is, that when a new version comes out, we have to change to this one
and can not manipulate 150 files until it compiles again. I felt totaly screwed.

Greetings from germany
Gerit Freericks

[GeritFreericks]

Found in fhir-single.xsd:

  <xs:simpleType name="QuestionnaireItemOperator-list">
    <xs:restriction base="code-primitive">
      <xs:enumeration value="exists">
        <xs:annotation>
          <xs:documentation xml:lang="en">Exists</xs:documentation>
        </xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="=">
        <xs:annotation>
          <xs:documentation xml:lang="en">Equals</xs:documentation>
        </xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="!=">
        <xs:annotation>
          <xs:documentation xml:lang="en">Not Equals</xs:documentation>
        </xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="&gt;">
        <xs:annotation>
          <xs:documentation xml:lang="en">Greater Than</xs:documentation>
        </xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="&lt;">
        <xs:annotation>
          <xs:documentation xml:lang="en">Less Than</xs:documentation>
        </xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="&gt;=">
        <xs:annotation>
          <xs:documentation xml:lang="en">Greater or Equals</xs:documentation>
        </xs:annotation>
      </xs:enumeration>
      <xs:enumeration value="&lt;=">
        <xs:annotation>
          <xs:documentation xml:lang="en">Less or Equals</xs:documentation>
        </xs:annotation>
      </xs:enumeration>
    </xs:restriction>
  </xs:simpleType>

[MvRens]

For such enums with untranslatable values you can use a ".hints.xsd" file. Since it is a separate file you can still use it when the XSD updates. See: https://x2xmldatabinding.readthedocs.io/en/latest/hints.html#renaming-enumeration-values