unforbidable / patcher Goto Github PK

Implement ALCH forms.

Once the general layout of the GUI has been decided on and all the functionality is in place, I'll go back and add some finishing touches to make it nicer to look at :)

When multi-line messages get printed in the console, the scrolling gets messed up. Please fix! (:

Among other things, notably the following will be removed in favor of the new data model DLL:

• The whole Patcher.Rules.Compiled project
• Most of Patcher.Data.Plugins namespace
• The whole Patcher.Data.Plugins.Content namespace
• The whole Patcher.Rules.Proxies namespace

As discussed, for various purposes of the GUI side of patcher, some model objects need to be extended to include display name.

Not all model objects need this property, but it seems to make sense for model objects that follow

• RecordModel - Name for the selected forms type
• FieldModel - Labels for controls (covers fields that are field groups and structures)
• MemberModel - Labels for controls
• EnumMemberModel - Lables for values in combo boxes
• FunctionParamModel - Labels when entering function parameters

Just in case this wasn't implemented yet, as discussed:

• Display name will automatically adopt the value of name if not provided

(Note: Included reason to make sure we aren't missing anything)

Ideally, for the releases all currently supported form types should still be supported with the new XML data model. Form types to support:

• ALCH
• AMMO
• ARMO
• COBJ
• FLST
• GLOB
• GMST
• IMAD
• IMGS
• KYWD
• PROJ
• WEAP
• WTHR

And the list of unfinished fields that will need their own XML file:

• virtual-machine (scripting data)
• body-template (for armor-related records)
• destructible (destruction data)
• actor-value

Also the function index (functions/functions.xml) including function parameters should be completed:

• Functions

While the big picture diagram went into a broad overview of how each half would fit together, I think it would also be important to have more in-depth diagrams for each individual half of the project to lay out their class and method structure, so that we can both follow the exact implementation more easily :)

Now I'd like to start cleaning up the layout of the prototype and align it more closely to the implementation ideas that we've come up with and outlined in the big picture diagram thus far. Of course, this is also the time to start brainstorming more ideas and improvements to the current layout to make the GUI as user-friendly as possible, and get everything figured out and agreed upon before we start implementing the functionality :)

Just implementing the actual control functionality for now, so that buttons act like buttons, drop-down menus will switch screens, etc. Then we can worry about integrating the classes for the real functionality of the tool :)

While the big picture diagram goes into a broad overview of how each half fits together, I think it is also important to have more in-depth diagrams for each individual half of the project to lay out their class and method structure, so that we can both follow the exact implementation more easily :)

This is not a high priority, of course, and is something we can update or add to as we move forward and continue to change things.

After downloading and installing "Patcher", i ran the program from the ".bat" file and everything was working fine until i received this error

2019-06-17 18:24:14.227506 ERROR System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentException: Illegal Global Variable type: None
at Patcher.Data.Plugins.Content.Records.Glob.OnTypeChanged(GlobalVariableType oldType)
at Patcher.Data.Plugins.Content.Records.Glob.set_Type(GlobalVariableType value)
--- End of inner exception stack trace ---
at Patcher.Data.Plugins.FormLoader.get_IsBusy()
at Patcher.Data.Plugins.FormLoader.WaitForCompleted()
at Patcher.Data.Plugins.FormLoader.Dispose()
at Patcher.Data.Plugins.Plugin.LoadForms(Boolean lazyLoading, Func2 predicate) at Patcher.Data.DataContext.LoadForms(Boolean lazyLoading, Func2 predicate)
at Patcher.App.Main(ProgramOptions options)

Log file is attached.
Patcher-demos.esp.log

Implement ARMO forms.

XML structure need to be able to represent the data model as well as allow minimal redundancy, which will be done by allowing files to be included in others.

The structure will define both the physical data layout in ESP/ESM files but also the form used to represent the data to the user. Adapters (that will not be generated) will be used to map raw data to more user friendly form, i.e. a uint with color data will be mapped to a structure.

This will be the tricky part: writing classes to generate record lists and iterate through them, writing methods for the GUI to know when and where to trigger these classes, etc.

This might alleviate the need to use the console altogether. Basically user's could save the settings like file path, form type, and "where" parameters as a template that they can select from a dropdown menu upon first running the GUI. This would make repeated editing of the exact same record lists in the exact same plugin files extremely easy and fast, which would be great for repetitive tweaking and creating lots of plugins :)

Currently there is need for a "big picture" diagram combining both the GUI navigation and main Patcher engine functionality in one place, in order to plan methods of implementation in how to integrate the two together.

An idea to include a small output section where errors will be displayed when rules are executed by the Patcher. Within that output box, user's would be able to click on individual rules and be pointed towards the exact line of code that caused the error within the rule editor window.

This will take quite a lot of time, and there are definitely some of these that we will choose not to provide support for.
**I will include a full list eventually, once we start working on this.

When there is a lot (thousands) of log entries displayed, adding more entries visibly slows down printing new text. The current workaround is that only last 800 entries is shown. I suspect the component used to display the log entries, i.e. ItemsControl, is not suitable for this purpose.

With the way the Patcher is currently being redesigned, it will be extremely good at editing records but not that great at actually viewing them, so you would most likely use it alongside another tool that is better for record viewing, like xEdit.
This is definitely not a feature that would be implemented until sometime after official release, but to make the process even smoother it might be nice to have a custom record viewer or some kind of record navigation built into the Patcher on the side, so that users could view and understand the types of records they wanted to edit in advance. We'll see! :)

Handle a situation when the specified path includes a trailing backslash. The tool will crash when a a path that includes a trailing slash is specified.

Add long, ulong and bool types to be supported by MemberType and TargetType models.

I just wanted to set up a place where we could make notes for small things we wanted to add to the models, as we have added quite a few things up to this point and we will most likely want to add more in the future once we start working on new form types :)

• fields should inherit the name of their structs/field groups if a name is not provided for them
• field types can now be read from the as-type attribute as well
• added conditional loading attributes if to fields
• the union tag can be used for fields that can contain values of multiple different types
• support for long, ulong, bool, lstring, ilstring, and dlstring types for members and as types
• update target model type to allow for more than one form reference type (ref(WEAP|ARMO))
• display-name will automatically take on the value of name if not provided
• display-name element added for records, fields, members, function params, enum members, and enums, to hold the names of as they will appear within the GUI.
• support non-printable characters as Unicode literals (e.g. \u0001).
• field and member defined as inline structures can be arrays with attribute array.
• allow automatic syncing of numeric fields with the count of items in a specified array field with attribute sizeof.

Just the bare bones of the GUI for now :) I'll post a few screenshots of a simple outline so that we can decide on the basic color scheme and layout that we like best. No functionality for now.

A small host of C# types will be generated based on the loaded data model:

• A record class for every record to maintain record data.
• A sub-record class for every structure and field group to maintain sub-record data, used by record classes.
• A class for every target (as) structure, such as Color.
• Two interfaces IForm for FormReference and IFormCollection for FormReference[].
• A single static class with extension methods for casting references from any form type to a specific form type, and for filtering a list of references from any form to a specific form type, a set of two for every form type.
• A class derived from the Variable class for each struct union.
• An enum for every enumeration type.
• A proxy class for every record and sub-class, for access control.
• An interface for every record and sub-record class, for isolation of implementation.
• An interface for every adapter to handle conversion between fields and properties. This includes so called default adapters that sort of make sense, although they are not currently used just yet.
• A single interface to serve as provider of adapter implementations. (The patcher implements this interface to provide adapter implementations for the data model.)
• A single class to serve as the function index and provider.
• An enum with containing all functions.
• A single static class that will store various information, such as the patcher version.
• A set of classes Variable for variable types (strongly typed unions), supporting 2 through 10 types.
• Additionally the serialized data model will be embedded as a binary resource.

Mod Organizer will be supported. The user will specify the path to a profile and optionally paths to the Mods and Override folders in case they are not located in the parent folder of the profiles folder.

Currently when a new form is inserted based on another form the data of the original form is not automatically copied to the new form even if the forms are of the same kind. There will be an option to "copy" the form content in case the original and the new form are of the same kind.

When data model is loaded from XML files, it should be validated, before it can be used and compiled. The validation not only ensures that the compilation will not fail but also that it is complete. The following points will be verified (so far, there will be more):

• Names must be legal C# type names.
• Names must be unique in their scope (e.g. names of fields in the same structure).
• Key is required for fields (unless the field represents a field group).
• Name is required for all models except fields and members that aren't going to be used.
• Key is exactly 4 chars long.

No archived files are used currently but at some point the tool will need to extract the new BA2 archives introduced by Fallout 4.

Once loaded the complete data model will be represented as a tree of objects. There objects will be used to generate the class files for records and subrecords, but also enums, function index, and so on. The model tree will also be used to by the GUI and the Rule Engine. Reference manual pages will be also generated based on the model tree (i.e. instead of System.Reflection)

• Records
• Fields
• Field Groups
• Structures
• Enumerations
• Functions

Method that can be used to dump information about any variable, property or form in debug mode will be added.

Methods Debug.Dump are not stripped when debug mode is disable with results in compiler error because the Debug helper is not available then.

Fix the following typos in XML generated documentation:

Engine.CreateMaterial: "Create" -> "Creates"
Functions: "Methods" -> "Methods such as"
Functions: "any Papyrus function" -> "any Papyrus function that returns a value"

This is a 'chicken and egg' sort of thing. The following expectations must be met:

• The patcher executable will statically reference the data model DLL and so it must already exist during
  compile time. This is because patcher needs to see all the interfaces in order to be able to implement them, which means the data model DLL cannot be referenced lazily like a compiled rule DLL is.
• The data model DLL will be created during the run-time of the patcher executable.

The current plan is as follows:

1. Initially there is no data model DLL, so the patcher executable creates one, e.g. Patcher.Data.Models.dll.
2. New DLL is added as a static reference, new interfaces are implemented, and the patcher executable is recompiled.
3. When the patcher starts up it will check XML model files to see whether the data model DLL is up to date. When the DLL is not up to date (e.g. user made changes to the XML model files) the patcher will simply inform the user that the data model DLL needs to be updated.
4. The user may choose to recompile the data model DLL by running patcher with a specific command line option e.g. --updatemodel. After that, when patcher is run again the new data model DLL will be used.
5. Depending on the sort of changes to the data model, new interfaces might need to be implemented. For a developer it means going back to step 2. For the user this means having to suffer some error messages.

Ideally, to avoid errors on the user side, the patcher should make sure it implements all generated interfaces before it actually replaces the data model DLL. When new interfaces are detected, the new data model DLL will not replace the original data model DLL and will be saved with an alternate name instead, e.g Patcher.Data.Models-new.dll. Also, when replacing a data model DLL a backup copy of the original should be created, e.g Patcher.Data.Models-backup.dll.

The problem of missing adapter implementations will be the rarer the more forms are supported and the more adapters exist, as the existing adapters will be reused, but the problem will never be eliminated.

A small cave-at regarding replacing of the data model DLL file. The file will be locked by the file system, because it will be in-use when the patcher executable is running. To physically replace the file, a new process will be started, which will wait until the patcher executable process ends and only then replace the file.

The tool should ensure the value passed via option -r, --rules contains only characters that are allowed in a folder name and a file name. If not, e.g. when the full path to rules is specified, an error should be displayed if not.

XML model definition files can be changed freely but they won't be actually used by patcher until the data model DLL is recompiled. The patcher will always load the XML model definition files and from them build a new model internally in order to check whether or not the data model DLL is up to date with the XML model definition files, however, for consistency, the patcher needs access to the data model tree that was used to create the data model DLL.

The solution is to bake the data model tree into the data model DLL when the classes are generated i.e. a static public member of a static class. embedded as a binary resource.

This requires the ability to serialize the data model tree into a byte array, and back again. We will use JSON format and sort of an XPath scheme for references (enums, structs and field groups), custom written to avoid dependencies on additional libraries.

• Serialization
• Deserialization

As an alternative to iterating through a list of records and applying a single rule set all of them, this would be an option to pause on each individual record in the list and apply a different rule to each.

A great example of a scenario in which this might be useful would be if a mod user wanted to quickly look through all of the female NPCs in the game and change their hair style. The Patcher would pause on each NPC record (which could still be specified using a set of parameters, e.g only female NPCs) and prompt for a new rule, in which case the user would only change the singular line relating to the NPC's hair and hit "Next."