dekarrin / fairyscript

Create manuscripts for Ren'Py in a human-readable format, then export to .rpy and .docx

FairyScript

Write manuscripts in screenplay style and export them to various formats.

Description

FairyScript provides a human-readable yet still regular format for writing manuscripts in plaintext. These manuscripts can then be compiled with fairyscript to various other formats, either for consumption by other programs or for ease-of-use by humans.

There are three main values that FairyScript provides:

• Storing manuscripts as plaintext rather than as binary or XML-based files (such as those used by Microsoft Word) consumes less space.
• Using the .FEY format allows the manuscript to be compiled to other formats when necessary.
• The .FEY format provided by FairyScript is easier to read than formats that are directly used by manuscript execution systems, such as Ren’py.

Supported Formats

FairyScript provides the .FEY format for files. It is capable of interpreting manuscripts written in this format, and compiling them to the following formats:

• .FEY -> .RPY (Ren’py Script)
• .FEY -> .DOCX (Word Office)

Installation

To install FairyScript to the system, do pip install fairyscript. This will make the fairyc command available on the system.

Requirements:

• Python (2.7, or 3.3 or later)
• lxml, which can be installed from pip.
• ply, which can be installed from pip.

To execute the FairyScript compiler from a local download without installing it, run the local fairyc.py (and substitute executing this file for the fairyc command found in the documentation). Note that the above dependencies must first be resolved.

FairyScript also requires python-docx (for writing to Word documents). However, the original codebase has been modified to support additional features that FairyScript requires, and so it is included in the FairyScript codebase. For license information on this package, please see the appropriate LICENSE file.

Compiler Usage

The FairyScript compiler (fairyc) is used to compile .FEY format manuscript files into other file types. It is invoked by execution from the command line.

The typical usage of fairyc is to either compile .FEY files to prettier, more human-readable formats or to compile .FEY files to executable scripts.

To compile a FEY manuscript to Ren’Py:

$ fairyc renpy input_file.fey -o renpy_script.rpy

To compile a FEY manuscript to Microsoft Office:

$ fairyc renpy input_file.fey -o my_script.docx --word

When invoked with no arguments, fairyc will read manuscript statements from stdin, compile them to Ren’Py script format, and then output them to stdout:

$ generate_fey | fairyc renpy | process_output

Input files are specified by passing each as an argument; to specify multiple input files, pass in multiple arguments. All input files are read in the order that they are given. If no input files are given, fairyc will read from stdin.

In this example, three files are compiled to a single Ren’Py script:

$ fairyc renpy main_path.fey branch1.fey branch2.fey -o my_script.rpy

The output file is specified with the -o option. Only one output file may be specified. If the -o is not given, fairyc will write the output to stdout. Note that due to the inherent limitations of the DOCX format, writing to stdout is not permitted when compiling to Microsoft Word format.

Without any additional specification, fairyc will compile the input to Ren’Py script format. This can be changed by changing the subcommand given. renpy is used to specify Ren’Py, docx specifies DOCX format output, lex specifies lexer symbols only, ast specifies outputting the abstract syntax tree without compiling, and analyze performs static analysis on the code and outputs the results as plain text.

By default, fairyc expects input to be in FEY manuscript format, but it can also parse files that have already been lexed as well as compile abstract syntax trees. Use the -f (long version --format) option to set the type of file being processed. The argument to -f option must be fey for processing FEY manuscript format, lex for processing lexed symbols directly, or ast for processing abstract syntax trees.

The following example parses a file containing pre-lexed symbols and then compiles the result to a Ren’Py format script:

$ fairyc renpy -f lex script_symbols.lex -o my_script.rpy

In addition to the options listed above, there are many options that are specific to the compiler for a particular format. For a full list of options, invoke fairyc with the -h option:

$ fairyc -h

FEY: The FairyScript Language

The FairyScript Language is documented in full in the file fairyscript.md in the docs directory of fairyscript.

fairyscript's Issues

Complete notes on Ren'Py compilation

Complete Notes on DOCX compilation

preproc_chars not saving characters from within branches

Possible bug. Analysis of code shows that preproc_chars seems to not be saving the results of recursion into branches. Proper investigation and repro would be to write a test that includes a CHARACTERS annotation in IF/WHILE block; proper behavior is that the characters are then included in the final output, improper is that they are skipped.

Add test for new LEX format with stdin as file

Need to test that using stdin makes source file information get set to None.

Do not overwrite files on error

Currently the files to output to are opened immediately, regardless of whether there are any errors. They should only be opened on successful compilation of one of the inputs.

New format for LEX output

Create a new format for LEX output; for FEY input, include the source filename of each symbol, but should preserve original order. For LEX input, pass through unchanged.

New format for AST output

Create a new format for AST output. Make behavior default but turned off via the --no-debug-symbols option.

For FEY and LEX input, include the line number and original source filename for each node of tree. For AST input, pass through unchanged unless --no-debug-symbols option is given, then remove symbols before outputting.

Take input of two different files and check file sources in AST

Create a test that inputs multiple source files and outputs a single AST with sources set correctly.

Split non-PLY lexing and parsing into own modules

Lexing and parsing functionality in fairyscript package __init__ module has grown extensively. Need to abstract both into own modules.

Add mode to condense AST debug file references

Add a new mode that will assemble AST source file references into list at beginning and use only the index to reference them in each mode. Make default but can disable via --inline-sources option.

Fix broken stdin detection

Better detection of stdin rather than just via possibly-valid filename.

Make tests fail-fast

Currently, tests that fail a step in their execution are allowed to continue. While other TESTS should certainly run after a test has failed, any command within a test that fails should cause instant failure of the test, rather than trying to execute the remaining commands.

Allow multi-word keyword sequences to have any whitespace in between

Compilation occurs even when parsing fails

Compiler should not attempt compilation step if parsing has failed.

Update expected output for new LEX format

Current expected output fails with new lex format (which includes lines). Update expected to use new format.

Add FOR TARGET parameter to Include annotation

Include annotation should be able to specify that the inclusion happens only when the file is being compiled for a particular set of target languages. Could specify multiple with "AND".

Support Unicode character files

Because of the limitations of python's csv module, character files are currently only supported if they are written using UTF-8 or printable ASCII characters only. This should be changed to allow any input.

Add test for consuming LEX into other formats

Add a new test which checks that consuming lex input is working properly for outputting other formats.

Update expected output for new AST format

Current test fails because expected output mismatches new AST format (with debug symbols). Update test to not fail, and add additional test for --no-debug-symbols.

Include targets of CAMERA in ANA output

ANA output should include a listing of the camera targets in its output; currently only includes the number of references.

Update ANA tests

Update ANA test case to use new format defined by implementation of #23. Should test both new format, as well as add a new case for extended formats.

Add test for stripping AST debug symbols

Add a new test which consumes AST input with debug symbols and outputs AST without debug symbols.

Add test for new AST format with stdin as file

Ensure that AST sets stdin file information properly in debug symbol output, and can handle both getting stdin directly and stdin as used from a series of lex symbols.

Better naming for test cases

Test case sources and expected output currently do not match the name of their test, and are used by multiple tests. To keep tests independent, each should use its own expected output/source input files, and they should match the name of the test (substituting expected_ and input_ for the expected and input files, respectively)

Allow specification of ren'py visual effect bindings

Compiling visual effects to Ren'Py requires knowing what the effect binds to. As of right now, there is no way to inform the compiler of the bindings, meaning it always assumes a binding to the scene. There should be a way of annotating Ren'Py inclusions so that bindings can be located.

Split AST data object into own module

AST has now progressed to level of sophistication where it makes sense to give it its own class in a new module. Allow mutating methods to combine other ASTs with it, so there is less overhead in processing large ASTs (right now we do a lot of memory x2 space copy operations, want to eliminate those).

Document new command-line switches

Add documentation for new switches for --no-debug-symbols in new AST mode implemented in #19, --no-source-info implemented in #23, and --inline-sources implemented in #26.

Create stdin distinction

Have a scheme for distinguishing between multiple stdin sources in AST and consuming compilers.

Test case would be to load multiple .lex files, each of which came from stdin, and test that the resulting stdins are distinguished and consumed properly

Add verbose mode

Add a new verbose mode -v to common command line options that causes exceptions to fully report.

Also include the values of command line arguments in debug output.

Add 'other' field to Character File format

Allow for the specification of other attributes of a character with the use of an 'other' field in the character file.

Create an onstage count for characters in ANA

Include functionality for an onstage count for characters in static analysis

Add test for consuming AST to other formats

Add test which checks that AST can be consumed as input and output to other sources

Use new debug symbol info for ANA output

Use new debug symbols to show extended information on references and definitions in static analysis output. Make behavior default but can disable via command-line switch --no-source-info.

Add CHARACTERS annotation

Annotation CHARACTERS specifies a file that lists the characters in the script.

Characters file is CSV format, and includes the name of the character as referenced in Scrappy, the name of the character as it should appear after compilation, the color of the character's label, and a field for all other specifications.

Create scene count for character in ANA

Add name of exception to generic Critical Error

Currently, critical error shows only the message of the exception. This is fine for the specific except clauses, but for the generic one it should at least show the type of error.

Add WITH PARSING parameter to INCLUDE annotation

Include annotation should be able to specify whether the included file should be parsed. If not, it is included directly in the output.

WITH PARSING, WITH PARSING ON, or the lack of a WITH PARSING parameter indicates that the included file is to be parsed and its AST added to the including file at that point.
WITH PARSING OFF indicates that the included file is not to be changed; its contents will be output exactly at that point during compilation.

Split character file reading into own module

Character file reading is separate functionality that should be split from main.

Make analysis of references in CHOICE items more specific

CHOICE directives include line breaks by design, and so the individual lines should include their own line number rather than that of the CHOICE directive parent in analysis output

Include distinction between character names and IDs in ANA output

Currently, analysis does not distinguish between a character name and a character ID for lines. These should be distinguished.

Complete Language Reference

The Language Reference for scrappy must be complete.

Switch all errors/warnings to use logging framework instead of saving them for later retrieval

Add functionality to process INCLUDEs separate from compiling

Currently, processing INCLUDE annotations only happens as an automatic process directly before compilation, and there is no way to output ASTs include the contents of referenced files. Add this as new functionality. Could add new mode, or just a switch to AST processing.