awoken-bible / usfm Goto Github PK

New in USFM 3.0 are "milestone" tags, see https://ubsicap.github.io/usfm/milestones/index.html

I don't think this would be too hard to add, simply update the lexer to read the new syntax, then parser can track currently open milestone regions in same way as character markers.

Before USFM 3.0 and the introduction of arbitrary attributes associated with markers, \fig markers used a strict ording of attributes, as follows:

\fig_DESC|FILE|SIZE|LOC|COPY|CAP|REF\fig*

To quote the spec:

Compatibility with USFM 1.x and 2.x: If a USFM parser encounters a list of un-named attributes using a vertical bar separator within \fig ...\fig* these should be interpreted according to the strict USFM 2.x order.

https://ubsicap.github.io/usfm/characters/index.html#usfmc-fig-attr

This will require changing the lexer to not throw errors when multiple | (TokenType.VBar) are encountered in the body of a single marker. We could just keep this change within the lexer and have a special version of parseAttribute which simply implicity knows the names of the attributes based on their position

The main parser.ts file is getting very big and complicated.

We've split out parsing for footnote and cross reference environments into their own files - but its becoming clear that we are needing to duplicate code in these and the main parser file (eg, for generic text styling markersv (https://ubsicap.github.io/usfm/characters/index.html#special-text)

It would probably be possible to make the code more readable (and also remove duplication) by refactoring these "subparsers" into a resuable modular system, then each parser could be made up of multiple subparsers that can be combined depending on what symbols are allowed in a given environment.

The USFM spec has ways of encoding additional study material such as prefaces, side bars, etc.

https://ubsicap.github.io/usfm/notes_study/index.html

No thought has been given to implementing this thus far