dgm9704 / xoxo

reading, writing and comparing XBRL reports

Breaking changes!

After starting to add support for xBRL-JSON it became obvious that the code needed restructuring.

So 3.0.0 comes with breaking changes to the naming and organizing of code.

Different flavors are now in their namespaces Diwen.Xbrl.Xml, .Inline, .Csv, .Json,

Diwen.Xbrl.Instance is now Diwen.Xbrl.Xml.Report and so on.

Test code was also similarly shuffled.

Functionally everything is still the same, performance wasn't affected etc.

Please get in touch if you can't find something, or if have questions or comments.

With regards to xBRL-JSON, I would be grateful for any actual "real world" samples!

(of course it must be either publicly available or anonymized/fuzzed etc)

Branched!

The library code is now in two branches:

• main: .NET 5+ version that will get new features etc. (version 2 upwards)
• legacy: .NET Standard 2.0 version that will only get critical bug fixes (versions 1.x)

Xoxo

Container for the Diwen.Xbrl library and any related stuff

Diwen.Xbrl

A .NET library for reading, writing, comparing and converting XBRL reports Supports:

• EBA and EIOPA ITS reporting (eg COREP, FINREP, AE, FP, SBP, Solvency II, Pension Funds, etc.)

• Australian and Finnish SBR message structures

format	read/write	convert to
xBRL-XML	read+write	xBRL-CSV, xBRL-JSON
xBRL-CSV	read+write	xBRL-XML
xBRL-JSON	read+write	xBRL-XML
iXBRL	read	xBRL-XML

License:

GNU Lesser General Public License v3.0

http://www.gnu.org/licenses/gpl.txt

http://www.gnu.org/licenses/lgpl.txt

TLDR;

You can use the compiled library with your application without it affecting the licensing of your other source code.

(Just remember to link back here ie. "attribution")

If you modify the source code and distribute it, you need to license the software containing the modified version accordingly.

Diwen.Xbrl.Tests

Tests for exercising Diwen.Xbrl code, also serving as documentation and samples

Test code license

Free Public License 1.0.0

My environment

Code is written in C#,

targeting .NET 8

Test framework is xUnit

I use Visual Studio Code on Arch Linux

Ways get in touch

If you have any questions, comments, suggestions, problems, bugreports/fixes, etc

create an issue

It's always ok to create one even if you're not sure

https://github.com/dgm9704/Xoxo/issues/new

send a message through the NuGet package page

This might be the best way if you have something you don't want share publicly

https://www.nuget.org/packages/Diwen.Xbrl/2.0.0/ContactOwners

twitter: @DiwenXbrl

https://twitter.com/DiwenXbrl

mastodon: @[email protected]

https://techhub.social/@diwen

bluesky: @[email protected]

https://bsky.app/profile/diwen.bsky.social

xoxo's Issues

Can't add comments to instance w/o going through XmlDocument

Instance.FromFile throws if xbrl file has incorrect encoding (not UTF-8)

XML declaration and character encoding missing when writing to XmlDocument

Exception thrown when comparing facts and unitRef is missing

System.NullReferenceException: Object reference not set to an instance of an object.
at Diwen.Xbrl.Fact.Equals(Fact other)
at System.Collections.Generic.GenericEqualityComparer1.Equals(T x, T y) at System.Collections.Generic.LinkedList1.Find(T value)
at Diwen.Xbrl.ListExtensions.ContentCompareReport[T](IList1 left, IList1 right)
at Diwen.Xbrl.InstanceComparer.FactComparison(Instance a, Instance b)
at Diwen.Xbrl.InstanceComparer.Report(Instance a, Instance b, ComparisonTypes comparisonTypes)
at XbrlCompare.Program.CompareAndReport(String inputReport, String outputReport, String comparisonReport, ComparisonTypes comparisonTypes)
at XbrlCompare.Program.Main(String[] args)

Unused Dimension/TypeDomainNamespace is written to xbrl file

Missing Instance.AddFilingIndicator overload for providing value for 'filed'

Comparing FilingIndicator collections doesn't consider filed attribute

When comparing FilingIndicator collections, filed=false should match missing filing indicator

Scenario with no members in some cases serialized as empty element

ie: <xbrli:scenario />
instead it should not be serialized at all

If unit is not specified ( = null), metric gets unit -attribute with empty content

Filingindicator handling by attribute is not supported

No way to select what to compare in InstanceComparer.Report

Instance.TaxonomyVersion is null after reading from file

So basically need to implement reading of processing instructions when deserializing

No overload for InstanceComparer.Report that takes file paths of reportfiles

InstanceComparer should be able to ignore Decimals when comparing facts

Facts can be equivalent in other ways and have different values for Decimals. In some cases these facts should be seen as being same.

InstanceComparer.Report does not compare facts

RemoveUnusedUnits throws when unit of a fact is null

Fact.ToString() throws when fact has null properties

Assert.Fail failed. System.NullReferenceException: Object reference not set to an instance of an object.
at Diwen.Xbrl.Fact.ToString()
at Diwen.Xbrl.InstanceComparer.FactComparison(Instance a, Instance b)
at Diwen.Xbrl.InstanceComparer.Report(Instance a, Instance b, ComparisonTypes comparisonTypes)

-> In Fact.ToString() check for Metric, Unit and Context being null

Can't add fact without unit when CheckUnitExist is true

Instance.AddFact(Scenario, ...) doesn't allow for Scenario to be null

Scenario argument should perhaps be allowed as null

No way to choose basic comparisons

In some cases it's know beforehand that instances are going to differ eg by entity or period, but should be otherwise identical. These cases are now reported as different. Since the basic comparisons are already separated in logic, there should/could be switches for them also.

Facts are compared by unitRef and not measure of unit

ie. these should be equal cases and are not atm.
a)

  <xbrli:unit id="uEUR">
    <xbrli:measure>iso4217:EUR</xbrli:measure>
  </xbrli:unit>
<eba_met:mi290 unitRef="uEUR" decimals="-3" [...]

b)

  <unit id="u-01">
    <measure>iso4217:EUR</measure>
  </unit>
<eba_met:mi290 unitRef="u-01" decimals="-3" [...]

Comparison report doesn't contain information about compared files

Sgen fails because of non-public setters for serialized collections

Not fixed, still need to do the same for Scenario.ExplicitMemebrs and .TypedMembers

Namespace declaration for enumerated fact value missing from written xbrl in some cases

Exception thrown when comparing facts when other has Unit==null

Comparing instances fails when one has entity and other doesn't

Object reference not set to an instance of an object.
at Diwen.Xbrl.InstanceComparer.CheckEntity(Instance a, Instance b)

Other instance is empty (has no contexts or facts), other has no facts but has a context (for filing indicators)

Exception thrown from Instance.ToFile if dimension / typeddomain namespaces not set

Scenario.Instance return null when it should return Instance reference

ExplicitMemberCollection.Instance returns null when it should return instance reference (Same for TypedMemberCollection)

ContextComparison throws NullReferenceException

Seems to come from trying to get Scenario.ToString() when scenario is null

Failed equality comparison doesn't provide any useful info about differences

There should be some way to tell where the differences(s) are, like reading a property after comparison, or giving an optional out parameter, logging the differences to a stream, throwing an exception with details or something. At the moment it's just a true/false which doesn't help much with any substantial instance.

TypedMember with no value (=null) does not get xsi:nil -attribute

add "instance-generator" processing instruction

Contexts are considered different when entity is different

This results in every context in compared instances being "different", which is true from a purely technical point of view. However it would better if the entity is compared only once per comparison (plus of course checked to be consistent across the instance) and contexts compared only by scenarios. The above holds also for period.
This would result in a situation where two otherwise functionally identical reports with differing entity (or period) would not report each and every context (and fact!) as different. Instead, the report would simply state that the entity (or period) is different and leave it at that.

instance-generator attributes are delimited by single quotes

... and everything else use double quotes

InstanceComparer compares taxonomyversions even when one instance doesn't have one

... this results in difference being reported when functionally same instance.
The comparison should be skipped when other instance (or both) doesn't have taxonomyversion,
since it isn't mandatory.

InstanceComparer.Report does not compare contexts

InstanceComparer.Report does not give message for all differing facts

Two cases with facts differing by value, only first is reported. Facts have same context but different metric. If one is fixed then other is reported.

Reading instance throws if no facts

example instance:

<?xml version='1.0' encoding='utf-8'?>
<?taxonomy-version 2.0.3?>
<xbrli:xbrl xmlns:xbrli="http://www.xbrl.org/2003/instance" xmlns:link="http://www.xbrl.org/2003/linkbase" xmlns:xlink="http://www.w3.org/1999/xlink">
  <link:schemaRef xlink:type="simple" xlink:href="http://www.eba.europa.eu/eu/fr/xbrl/crr/fws/corep/its-2013-02/2014-07-31/mod/corep_con.xsd"/>
</xbrli:xbrl>

link to xsd:
https://www.xbrl.org/2003/xbrl-instance-2003-12-31.xsd

excerpt:

<element name="xbrl">
<annotation>
<documentation>XBRL instance root element.</documentation>
</annotation>
<complexType>
<sequence>
<element ref="link:schemaRef" minOccurs="1" maxOccurs="unbounded"/>
<element ref="link:linkbaseRef" minOccurs="0" maxOccurs="unbounded"/>
<element ref="link:roleRef" minOccurs="0" maxOccurs="unbounded"/>
<element ref="link:arcroleRef" minOccurs="0" maxOccurs="unbounded"/>
<choice minOccurs="0" maxOccurs="unbounded">
<element ref="xbrli:item"/>
<element ref="xbrli:tuple"/>
<element ref="xbrli:context"/>
<element ref="xbrli:unit"/>
<element ref="link:footnoteLink"/>
</choice>
</sequence>
<attribute name="id" type="ID" use="optional"/>
<anyAttribute namespace="http://www.w3.org/XML/1998/namespace" processContents="lax"/>
</complexType>
</element>

RemoveUnusedContexts() needs performance improvement

RemoveUnusedContexts() gets IEnumerable and calls Contains multiple times. Should probably call ToList() first

InstanceComparer.Report does not understand datatypes

eg. "0.0000" and "0" are considered different when both are possible and correct representations of fact value.

instance-generator processing instruction missing last single quote character

Can't create streamable instance

http://www.xbrl.org/specification/streaming-extensions-module/pwd-2013-03-06/streaming-extensions-module-pwd-2013-03-06.html#term-streamable-instance

http://specifications.xbrl.org/work-product-index-streaming-extensions-streaming-extensions-1.0.html

InstanceComparer should be able to ignore trailing decimals zeroes when comparing facts

Two facts can be equivalent but have values differing only by trialing decimal zeroes. In some cases these facts should be seen as being same.

If decimals is not specified (=null), metric gets decimals -attribute with empty content

Unused domain namespaces aren't removed

InstanceComparer.Report throws when comparing nulls in Basic comparison

InstanceComparer should flush results after each comparison

...so when there are problems, you would at least get some output

Implement property or method that returns member xbrlCode

InstanceComparer should display differences in domain namespaces