ortham / libloadorder Goto Github PK
View Code? Open in Web Editor NEWA cross-platform library for manipulating the load order and active status of plugins for the Elder Scrolls and Fallout games.
License: GNU General Public License v3.0
A cross-platform library for manipulating the load order and active status of plugins for the Elder Scrolls and Fallout games.
License: GNU General Public License v3.0
I just noticed that if eg. Skyrim/plugins.txt
is missing, libloadorder reports that no plugins are active. This is because LoadOrder::loadActivePlugins()
is only called if the active plugins file exists, and it also adds the implicitly active plugins.
At least for Fallout 4, it is possible to load a non-master (or presumably a master) in the FE index while light masters are also present, so they end up sharing FE at runtime. In this case, if two records have the same runtime FormID, then the non-ESL record replaces the ESL record.
This behaviour is undesirable, and may lead to strange issues due to records being silently replaced by records that may be completely unrelated. libloadorder should probably disallow activating a normal plugin in the FE slot when light masters are also present.
See http://forums.bethsoft.com/topic/1459682-relz-wrye-bash-thread-97/?p=22860364 and onwards.
At the moment, I think the issue is to do with the path being returned by SHGetFolderPath being encoded in Windows/ANSI and not being converted to UTF-8, or something like that. I need to investigate more, really.
Now that Plugin::name()
no longer allocates, it's possible to avoid copying strings in plugin_names()
, plugin_at()
and active_plugin_names()
.
The .ccc files are text files with \r\n line separators and one CC plugin per line, in their load order. These should be read and cached as implicitly active plugins instead of hardcoding the CC plugins, as the game executables don't hardcode them any more.
There are several cases where .par_iter()
and other rayon-provided methods are used when sorting or searching, and these may no longer provide any performance boost since the recent optimisations of things like string comparisons were made. It would be good to re-test each of these and make any that don't provide any boost serial again.
Boost.Locale is used to convert between Windows-1252 and UTF-8, but UTF-16 uses the same small bytes for its first 255 codepoints as Windows-1252, so it's possible to convert between UTF-8 and Windows-1252 using the standard library's support for UTF-8 and UTF-16.
It's also used by libespm, so resolution of WrinklyNinja/libespm#4 is a precondition for this.
There's a few copy/pasted functions in the testing code. It might be possible to have a module that can be imported by unit tests and benchmarks alike, but it's more likely that I'll need to create a testing crate.
To complement my rewrite of libespm2, I want to rewrite libloadorder's backend using TDD:
LoadOrder
and ActivePlugins
classes to eliminate the need to synchronise the two._lo_game_handle_int
struct so that all game settings are inherited from a GameSettings
class. The struct will then only add the C string/array memory storage members.LoadOrder
constructor store a GameSettings
reference to be used instead of having to pass a _lo_game_handle_int
object for most of its member functions.The end result should be that libloadorder has a C++ API that gets wrapped by a C API, which only handles the memory management and exceptions, and contains no logic otherwise.
I would also like to make a few behavioural changes:
lo_set_game_master()
and its C++ equivalent, making them do nothing, as the game master will hold no significance for timestamp-based games, and the filename for the game master is hardcoded in textfile-based games.While API behaviour change is a breaking change, it's breaking safely, because it means that clients will no longer have to expect weirdness when reading data, and the API will be more permissive. Otherwise, maintaining API compatibility is also a goal.
Libloadorder doesn't currently do this, and there's a crash in LOOT that I can't replicate but which is probably due to a panic in libloadorder or esplugin.
There should be some method by which a client can be told they need to use lo_fix_plugin_lists
before they try try setting the load order themselves. See wrye-bash/wrye-bash#162 for an example use-case.
E.g. when the client loads, if the existing load order is invalid, it should be fixed straight away. If libloadorder makes a change, then another utility makes a change that results in an invalid change, when libloadorder updates its cache, it should also warn then.
A call to CheckValidity
after each Load
should be enough, and to catch exceptions thrown in it and treat them as a warning. A new warning return code should be added.
When setting an incomplete load order for timestamp-based games, the timestamps of plugins not included in the load order are unchanged, and the earliest timestamp used is the earliest timestamp of the specified plugins, so it's possible for those other plugins to end up loading before, amongst or after the specified plugins.
In timestamp-based games, any plugins that aren't specified get loaded in an undefined load order at the end. This is the correct behaviour, and in v6 timestamp-based games would do the same as unspecified plugins would be appended to the given load order. It looks like that needs to be re-implemented.
Needed for loot/loot#816.
ESLs have been patched into Fallout 4 and Skyrim Special Edition, so they're an update to the Asterisk load order method for the purposes of libloadorder's implementation.
Libloadorder needs to:
.esl
files as plugins.esl
files, treat them like master-flagged plugins.As of Fallout 4 v1.10.26.0.0, the implicitly active ESLs are:
ccBGSFO4001-PipBoy(Black).esl
ccBGSFO4002-PipBoy(Blue).esl
ccBGSFO4003-PipBoy(Camo01).esl
ccBGSFO4004-PipBoy(Camo02).esl
ccBGSFO4006-PipBoy(Chrome).esl
ccBGSFO4012-PipBoy(Red).esl
ccBGSFO4014-PipBoy(White).esl
ccBGSFO4016-Prey.esl
ccBGSFO4017-Mauler.esl
ccBGSFO4018-GaussRiflePrototype.esl
ccBGSFO4019-ChineseStealthArmor.esl
ccBGSFO4020-PowerArmorSkin(Black).esl
ccBGSFO4022-PowerArmorSkin(Camo01).esl
ccBGSFO4023-PowerArmorSkin(Camo02).esl
ccBGSFO4025-PowerArmorSkin(Chrome).esl
ccBGSFO4038-HorseArmor.esl
ccBGSFO4039-TunnelSnakes.esl
ccBGSFO4041-DoomMarineArmor.esl
ccBGSFO4042-BFG.esl
ccBGSFO4043-DoomChainsaw.esl
ccBGSFO4044-HellfirePowerArmor.esl
ccFSVFO4001-ModularMilitaryBackpack.esl
ccFSVFO4002-MidCenturyModern.esl
ccFRSFO4001-HandmadeShotgun.esl
ccEEJFO4001-DecorationPack.esl
As of Skyrim SE v1.5.3.0.8, the implicitly active ESLs are::
ccBGSSSE002-ExoticArrows.esl
ccBGSSSE003-Zombies.esl
ccBGSSSE004-RuinsEdge.esl
ccBGSSSE006-StendarsHammer.esl
ccBGSSSE007-Chrysamere.esl
ccBGSSSE010-PetDwarvenArmoredMudcrab.esl
ccBGSSSE014-SpellPack01.esl
ccBGSSSE019-StaffofSheogorath.esl
ccMTYSSE001-KnightsoftheNine.esl
ccQDRSSE001-SurvivalMode.esl
In LoadOrderTest::SetUp()
, an initial active plugins file is written, but in input plugin names are supplied as an unordered set, so duplicate entries is not tested (the set is initialised with them, but only one is stored because it's a set), and the order of entries is unknown, which affects synchronisation tests.
As of the availability of the Survival Mode beta for Fallout 4, the load order mechanism has changed. plugins.txt
now lists all installed plugin filenames in load order. Active plugins have their filenames prefixed by an asterisk *
, for example:
*Fallout4.esm
ArmorKeywords.esm
Armorsmith Extended.esp
Field Scribe Army - AE Comp.esp
*EasyHacking.esp
*Test_WEAP.esp
*Simple Intersection.esp
In the above, Fallout4.esm
, EasyHacking.esp
and Test_WEAP.esp
are all active, and the other plugins are not active.
This new mechanism obsoletes loadorder.txt
, so libloadorder should no longer write that file for Fallout 4. I'll call the new mechanism the "asterisk" load order method (so the constant will be LIBLO_METHOD_ASTERISK
).
I'll need to update the test suite to check that data is written correctly for the new method, then update the corresponding code accordingly.
The Creation Kit wiki page on the file formats says that
A maximum of 4096 .esl files can be loaded by the engine at once, but more Form IDs in an .esl file means that less .esl files can be loaded.
Using .esl files that each have 2048 Form IDs, the limit drops to about 300 loaded .esl files.
libloadorder currently just uses a static 4096 as the max, because I don't understand the relationship between number of FormIDs and the maximum active light masters, but filing this issue in case it gets figured out or more is revealed in future.
It's possible to create a master that depends on a non-master, and the game will load it, but in doing so the non-master will be moved so that it loads before the master (unverified). Libloadorder doesn't currently handle this case, as it doesn't check plugin masters at all.
I'm hoping this behaviour gets classified as a bug and fixed by Bethesda, since (IIRC) the analogous behaviour with a master depending on a non-master is a crash on load, but filing this issue to ensure I don't forget about it.
Edit: Removed mentions of light masters, it's actually about ESMs, which all light masters were when this issue was originally filed.
Reported by pStyl3 here.
To replicate, download ASIS and SUM, and run SUM through Mod Organiser, choosing to sort with LOOT. During the process, LOOT will launch with the error:
Error: Game-specific settings could not be initialised. libloadorder failed to create a game handle. Details: ASIS.esp : File is empty.
This is because a zero-byte ASIS.esp file is created before SUM runs LOOT. Ideally, the ASIS.esp created would have a TES4 header record, but libloadorder shouldn't fail when that isn't the case, it should just ignore the file, just like all other non-plugin files. In fact, it usually doesn't fail, but for some reason Mod Organiser's involvement breaks something.
It's worth noting though that this behaviour probably breaks why the ASIS.esp file is being created in the first place, so I should discuss this with whoever is responsible.
Right now all that lo_fix_plugin_lists
does is make sure all the plugins listed are present. Arthmoor has identified (wrye-bash/wrye-bash#162) that having more than 255 plugins active also causes problems in Bash, and being able to call this function to fix it would be neat.
The checks are given at the top of activeplugins.h
and loadorder.h
.
Would be good to have. Need to read up on what this entails.
The pluginComparator class should drop the esm tests. The way it is now a 'corrected' load order is returned to the client but the actual load order (the modification times) stay as they were
EDIT: the reason I want this is twofold - first Bash and liblo do double job in reading the timestamps - if liblo would just return me the timestamps I could easily check for master status. But I see now this will get very tricky - essentially a _cache_and_return_timestamps should be needed. I guess this is too much.
The second reason would be to have more control when setting load order of masters between them - for instance when I flip esm status.
So this probably should be renamed to "expose timestamps cache" cause that is what really is about. Will edit as I get more insight into the API.
After following the workaround in #32 (Build fails if LaTeX and Doxygen are installed)
The build continues through to the linking phase and spits out the message below.
This might also be related to #31 (Remove Boost.Locale dependency)
Link to file on dropbox -- because github can't properly display the file:
https://dl.dropboxusercontent.com/u/104054522/skyrim_mods/libloadorder/libloadorder_undefined_references_error.txt
Using libespm to get whether a plugin is a master or not would be preferred to the current method of libloadorder reading the master flag itself, as libloadorder shouldn't concern itself with the file format of plugins.
It would also be good for libloadorder to enforce the requirement that a plugin's masters all load before the plugin, which would require the reading of each plugin's master list, another job for libespm.
MSVS can't no longer find libespm due to recent changes on master.
Plugin validity checks are pretty much the only expensive operation in libloadorder, as they involve reading the whole plugin file, which can be quite large and so reading can be slow. The performance of actual plugin reading is up to libespm, but there are a couple of things libloadorder can do to improve its performance:
Plugin
object is constructed, and if an exception is thrown then it is handled as an invalid plugin, and if not that constructed object is stored.Unit testing of the API and internals would be good. I'm not sure how to accomplish it though, since libloadorder has a lot of filesystem interactions...
I'm going to try implementing the tests using Google Test, since I've seen examples of it being used in CMake + Travis environments, and it's obviously going to be of good quality.
Currently timetamp comparisons check if the file timetamp is newer than the cached timestamp, but it is possible for the timestamp of a file to be changed so that it is older, as identified by @Utumno in wrye-bash/wrye-bash#195.
The simplest fix for this is to use a not equals comparison operator rather than a greater than operator.
Debian GNU/Linux 8 (jessie) / Kernel 4.7.0-0.bpo.1-amd64
Boost: v1.55
Compiling libloadorder says that rbegin and rend were not decaled in this scope and
that they should be boost::rbegin / boost::rend.
This is the commit I was building against:
ae2512d
after perfixing all occurrences of rbegin / rend with boost:: the above error went away;
however, now it is complaining about codecvt not being found.
This is the output from make:
[ 46%] Building CXX object CMakeFiles/loadorder.dir/src/backend/LoadOrder.cpp.o
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp: In member function ‘void liblo::LoadOrder::loadActivePlugins()’:
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp:462:40: error: ‘rbegin’ was not declared in this scope
for (auto it = rbegin(loadOrder); numActivePlugins > maxActivePlugins && it != rend(loadOrder); ++it) {
^
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp:462:40: note: suggested alternative:
In file included from /usr/include/boost/range/functions.hpp:23:0,
from /usr/include/boost/range/iterator_range_core.hpp:28,
from /usr/include/boost/range/iterator_range.hpp:13,
from /usr/include/boost/iostreams/traits.hpp:39,
from /usr/include/boost/iostreams/detail/dispatch.hpp:17,
from /usr/include/boost/iostreams/flush.hpp:17,
from /usr/include/boost/iostreams/close.hpp:18,
from /usr/include/boost/iostreams/device/mapped_file.hpp:20,
from /home/william/temp/libloadorder/build/external/src/libespm/include/libespm/Plugin.h:30,
from /home/william/temp/libloadorder/src/backend/Plugin.h:32,
from /home/william/temp/libloadorder/src/backend/LoadOrder.h:29,
from /home/william/temp/libloadorder/src/backend/LoadOrder.cpp:26:
/usr/include/boost/range/rbegin.hpp:46:1: note: ‘boost::rbegin’
rbegin( const C& c )
^
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp:462:102: error: ‘rend’ was not declared in this scope
for (auto it = rbegin(loadOrder); numActivePlugins > maxActivePlugins && it != rend(loadOrder); ++it) {
^
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp:462:102: note: suggested alternative:
In file included from /usr/include/boost/range/functions.hpp:24:0,
from /usr/include/boost/range/iterator_range_core.hpp:28,
from /usr/include/boost/range/iterator_range.hpp:13,
from /usr/include/boost/iostreams/traits.hpp:39,
from /usr/include/boost/iostreams/detail/dispatch.hpp:17,
from /usr/include/boost/iostreams/flush.hpp:17,
from /usr/include/boost/iostreams/close.hpp:18,
from /usr/include/boost/iostreams/device/mapped_file.hpp:20,
from /home/william/temp/libloadorder/build/external/src/libespm/include/libespm/Plugin.h:30,
from /home/william/temp/libloadorder/src/backend/Plugin.h:32,
from /home/william/temp/libloadorder/src/backend/LoadOrder.h:29,
from /home/william/temp/libloadorder/src/backend/LoadOrder.cpp:26:
/usr/include/boost/range/rend.hpp:46:1: note: ‘boost::rend’
rend( const C& c )
^
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp: In lambda function:
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp:519:38: error: ‘rbegin’ was not declared in this scope
return *rbegin(timestamps) + 60;
^
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp:519:38: note: suggested alternative:
In file included from /usr/include/boost/range/functions.hpp:23:0,
from /usr/include/boost/range/iterator_range_core.hpp:28,
from /usr/include/boost/range/iterator_range.hpp:13,
from /usr/include/boost/iostreams/traits.hpp:39,
from /usr/include/boost/iostreams/detail/dispatch.hpp:17,
from /usr/include/boost/iostreams/flush.hpp:17,
from /usr/include/boost/iostreams/close.hpp:18,
from /usr/include/boost/iostreams/device/mapped_file.hpp:20,
from /home/william/temp/libloadorder/build/external/src/libespm/include/libespm/Plugin.h:30,
from /home/william/temp/libloadorder/src/backend/Plugin.h:32,
from /home/william/temp/libloadorder/src/backend/LoadOrder.h:29,
from /home/william/temp/libloadorder/src/backend/LoadOrder.cpp:26:
/usr/include/boost/range/rbegin.hpp:46:1: note: ‘boost::rbegin’
rbegin( const C& c )
^
In file included from /usr/include/c++/4.9/algorithm:62:0,
from /usr/include/boost/smart_ptr/shared_ptr.hpp:42,
from /usr/include/boost/shared_ptr.hpp:17,
from /usr/include/boost/filesystem/path.hpp:29,
from /usr/include/boost/filesystem.hpp:16,
from /home/william/temp/libloadorder/build/external/src/libespm/include/libespm/Plugin.h:27,
from /home/william/temp/libloadorder/src/backend/Plugin.h:32,
from /home/william/temp/libloadorder/src/backend/LoadOrder.h:29,
from /home/william/temp/libloadorder/src/backend/LoadOrder.cpp:26:
/usr/include/c++/4.9/bits/stl_algo.h: In instantiation of ‘_OIter std::generate_n(_OIter, _Size, _Generator) [with _OIter = std::insert_iterator<std::set<long int> >; _Size = long unsigned int; _Generator = liblo::LoadOrder::saveTimestampLoadOrder()::<lambda()>]’:
/home/william/temp/libloadorder/src/backend/LoadOrder.cpp:520:10: required from here
/usr/include/c++/4.9/bits/stl_algo.h:4325:11: error: no match for ‘operator=’ (operand types are ‘std::insert_iterator<std::set<long int> >’ and ‘void’)
*__first = __gen();
^
/usr/include/c++/4.9/bits/stl_algo.h:4325:11: note: candidates are:
In file included from /usr/include/c++/4.9/bits/stl_algobase.h:67:0,
from /usr/include/c++/4.9/bits/char_traits.h:39,
from /usr/include/c++/4.9/string:40,
from /home/william/temp/libloadorder/src/backend/Plugin.h:29,
from /home/william/temp/libloadorder/src/backend/LoadOrder.h:29,
from /home/william/temp/libloadorder/src/backend/LoadOrder.cpp:26:
/usr/include/c++/4.9/bits/stl_iterator.h:639:7: note: std::insert_iterator<_Container>& std::insert_iterator<_Container>::operator=(const typename _Container::value_type&) [with _Container = std::set<long int>; typename _Container::value_type = long int]
operator=(const typename _Container::value_type& __value)
^
/usr/include/c++/4.9/bits/stl_iterator.h:639:7: note: no known conversion for argument 1 from ‘void’ to ‘const value_type& {aka const long int&}’
/usr/include/c++/4.9/bits/stl_iterator.h:647:7: note: std::insert_iterator<_Container>& std::insert_iterator<_Container>::operator=(typename _Container::value_type&&) [with _Container = std::set<long int>; typename _Container::value_type = long int]
operator=(typename _Container::value_type&& __value)
^
/usr/include/c++/4.9/bits/stl_iterator.h:647:7: note: no known conversion for argument 1 from ‘void’ to ‘std::set<long int>::value_type&& {aka long int&&}’
/usr/include/c++/4.9/bits/stl_iterator.h:588:11: note: std::insert_iterator<std::set<long int> >& std::insert_iterator<std::set<long int> >::operator=(const std::insert_iterator<std::set<long int> >&)
class insert_iterator
^
/usr/include/c++/4.9/bits/stl_iterator.h:588:11: note: no known conversion for argument 1 from ‘void’ to ‘const std::insert_iterator<std::set<long int> >&’
/usr/include/c++/4.9/bits/stl_iterator.h:588:11: note: std::insert_iterator<std::set<long int> >& std::insert_iterator<std::set<long int> >::operator=(std::insert_iterator<std::set<long int> >&&)
/usr/include/c++/4.9/bits/stl_iterator.h:588:11: note: no known conversion for argument 1 from ‘void’ to ‘std::insert_iterator<std::set<long int> >&&’
CMakeFiles/loadorder.dir/build.make:123: recipe for target 'CMakeFiles/loadorder.dir/src/backend/LoadOrder.cpp.o' failed
make[2]: *** [CMakeFiles/loadorder.dir/src/backend/LoadOrder.cpp.o] Error 1
CMakeFiles/Makefile2:130: recipe for target 'CMakeFiles/loadorder.dir/all' failed
make[1]: *** [CMakeFiles/loadorder.dir/all] Error 2
Makefile:137: recipe for target 'all' failed
make: *** [all] Error 2
[ 51%] Building CXX object CMakeFiles/loadorder.dir/src/api/libloadorder.cpp.o
/home/william/temp/libloadorder/src/api/libloadorder.cpp:31:19: fatal error: codecvt: No such file or directory
#include <codecvt>
^
compilation terminated.
CMakeFiles/loadorder.dir/build.make:238: recipe for target 'CMakeFiles/loadorder.dir/src/api/libloadorder.cpp.o' failed
make[2]: *** [CMakeFiles/loadorder.dir/src/api/libloadorder.cpp.o] Error 1
CMakeFiles/Makefile2:130: recipe for target 'CMakeFiles/loadorder.dir/all' failed
make[1]: *** [CMakeFiles/loadorder.dir/all] Error 2
Makefile:137: recipe for target 'all' failed
make: *** [all] Error 2
See output at https://gist.github.com/Freso/a7f1c2fee5445dec881a - I don't know whether it is anything to be concerned about, but figured I'd report back in case it's an easy fix. (Maybe it's a Boost 1.59 vs. 1.55 thing, if Travis doesn't complain about it?)
Wrote a wiki article that you may want to incllude in your readme/wiki:
https://github.com/Utumno/libloadorder/wiki/Build-on-windows
For people new to CMake, boost etc this can be handy.
Still I am using MSVC 2013 and when building I get a libloadorder.lib - when I flip the BUILD_SHARED_LIBS flag build fails with:
1>LINK : fatal error LNK1104: cannot open file 'libboost_filesystem-vc120-mt-sgd-1_56.lib'
So:
Thanks
The stack overflow I reported: wrye-bash/wrye-bash@b79a4c2
is still present in your latest iteration: ddb53c7
Reason:
Calling the method again in the except block is asking for trouble anyway
EDIT: second SO:
In this case it is a misordering of mods and masters - especially in this case the mod was inactive
EDIT: edited the title to reflect the liblo rather than the Bash issue. Real fix should be to add to get_** methods an output parameter which would return the fixed list - the list being fixed while being checked in checkvalid() rather than repeating the ifs in a different method. Moreover the invalid lists are of interest to the client (warn the user, make decisions on how to fix etc) - so the model of the library binding should be return valid, invalid
and let the user decide (or even return invalid, warning
- where invalid should mean actual - more on this later) - rather than 'throw on error then call fix'. Especially the masters test should be dropped from LO as it was dropped from active - it even fails on inactive mods and that's not what the library should worry about anyway - it is the utilities role to display such inconsistencies to the user and let him decide
afaict there is no docs/latex/make.bat
and none is ever created.
If DLC .esm
files are not listed in plugins.txt
, FO4 will load them anyway. It seems that which .esm
files are DLC plugins are hardcoded into the game, as the game doesn't seem to use DLClist.txt or any other file to determine this.
It's worth noting that the DLC plugins don't appear alongside other plugins in the in-game load order settings UI, so modifying the load order there will remove them from plugins.txt
if they were previously added.
If they are not listed in plugins.txt
, the DLC plugins will load after all other master plugins, in the order of their release. In this case, the DLC do not load before any master files that have them as masters.
It seems like the DLC loading behaviour is either badly designed (requiring game executable / load order utility updates to support each new release, breaking master files having them as masters) or heavily bugged, so it's worth keeping an eye on this as the behaviour will hopefully change.
Changing this is a breaking change, but it's a potentially unexpected side-effect, which should be avoided. It's done so that the user doesn't end up with a partial (and so possibly invalid) load order, but a better way of handling this would be to just error if passed an incomplete load order.
Fresh off the back of the backend code overhaul in #20, some improvements can also be made to the "frontend" API code:
_lo_game_handle_int
.LoadOrder
class.Reported here.
The error message is from libespm, but libloadorder shouldn't be attempting to open an empty .esm or .esp file unless it's to check if it's a valid file in the first place, and that should have exceptions caught for evaluation, rather than propagating the error.
Skyrim SE doesn't include the official plugins in its plugins.txt, so libloadorder shouldn't either. It also currently (as of 1.2.39) loads the DLC plugins in timestamp order, which makes no sense whatsoever, so I'm in favour of just changing the hardcoded order to use the same as NMM as it's also the order that is most commonly assumed for the DLC in the original Skyrim. That order is:
Dawnguard.esm
Hearthfires.esm
Dragonborn.esm
@DuskDweller, will NMM be changing how it handles the DLC plugins' load order for Skyrim SE?
For Skyrim. Reported by DuskDweller. This should be fixed so that the function returns an error code without making any changes.
Reported here. The load order should be:
Fallout4.esm
DLCRobot.esm
DLCworkshop01.esm
DLCCoast.esm
DLCworkshop02.esm
DLCworkshop03.esm
DLCNukaWorld.esm
ccbgsfo4024-pacamo03.esl
DLCUltraHighResolution.esm
Unofficial Fallout 4 Patch.esp
but libloadorder will accept:
Fallout4.esm
DLCRobot.esm
DLCworkshop01.esm
DLCCoast.esm
DLCworkshop02.esm
DLCworkshop03.esm
DLCNukaWorld.esm
DLCUltraHighResolution.esm
Unofficial Fallout 4 Patch.esp
ccbgsfo4024-pacamo03.esl
though ccbgsfo4024-pacamo03.esl
is a Creation Club plugin and so always loads after the DLC plugins (excluding the ultra high resolution DLC plugin) and before all other plugins.
While libloadorder does limited validation (it doesn't check that a plugin loads after all its masters, for example), it would be good to improve it wherever possible without introducing too much complexity, and this might be one of those cases.
Implementing this is almost as simple as checking that the start of the list matches the list of implicitly active plugins, ignoring any of the latter that are missing, up to the end of the former if it is shorter than the latter. However, the original Skyrim has Update.esm
as an implicitly active plugin, but it can load after other master files, so that will have to be handled as a special case.
In more recent versions of Skyrim / Fallout 4, the .esl
file extension just forces the ESL flag at runtime, but a plugin that has it set with a different file extension still (for the purposes of the function) acts like a light master.
Therefore libloadorder needs to load all the plugins before it can count the different types.
I've currently got a stashed changeset in my local repo that provides a test that runs load()
on a load order with 1000 plugins and 300 of them listed in plugins.txt/loadorder.txt, doing so a few times, taking an average and printing to stdout. This works for one-off testing, and roughly replicates what I could be doing with nightly Rust's #[bench]
. However, it's not ideal because it's not run regularly (it's messy so I don't commit it).
Recently I found that somewhere between 10.0.0 and 10.1.0, performance of loading a load order halved. I tracked most of it down to upgrading WalkDir (which it turns out libloadorder doesn't need anyway...) but there's still a ~ 20% drop in performance even if I go back and run 10.0.0 code, so my PC isn't good for measuring performance trends (could it be due to meltdown/spectre mitigation, or just something happening in the background?). It would be good to have a history of performance results available on CI.
Criterion offers benchmarking on stable, and bencher is a stable port of #[bench]
, either could potentially be used.
A file write is necessary if plugins.txt
/loadorder.txt
does not contain entries for all the plugins they need to, in the order they need to be. This adds some overhead, but gets around write permissions issues that can happen if the read-only flag is temporarily set for the file (as has been happening for FO4).
This is for loot/loot#562.
As @Utumno pointed out, libloadorder only deals with valid plugins, but what it considers to be a valid plugin isn't documented.
Primarily as a learning exercise (more FFI, docs, libespm's ergonomics, a step up in complexity, etc.), but the library is a bit messy and could do with a refactor anyway. I'd like to avoid API breakage.
TODO:
C API Breakage:
LIBLO_ERROR_TIMESTAMP_READ_FAIL
is unused as read failures are reported as I/O errors. This change could potentially be avoided.LIBLO_WARN_BAD_FILENAME
. This could be avoided somehow, but it is due to invalid input, so an error is probably more correct.LIBLO_ERROR_NO_MEM
is unused.If API breakage is embraced, then replacing the error codes with equivalents that map more cleanly to the Rust error types would be good.
See here.
tknikita's plugins.txt and loadorder.txt.
Now that libloadorder should be support being built as a native Linux library, it would be good if its unit tests (to be added as in #8) could be run automatically for each commit. Travis integrates well with GitHub, and I have experience with it.
See here.
Until v11.2.2, Error::GameMasterMustLoadFirst
was only used for indicating when the game's main master file was in the wrong position. It then got used to also indicate invalid positions for other hardcoded plugins, to avoid the breaking change of adding another error variant, but it would be useful to be able to provide the name of the plugin causing an issue, so next time breaking changes are made, also replace the variant with something like Error::InvalidHardcodedPluginPosition(String)
.
If lo_set_active_plugins()
is called for Skyrim and passed an array that contains a plugin that hasn't previously had its load order position set, it will not be activated. This behaviour should be changed.
Probably the best fix would be to silently append any such plugins to the load order, but another possibility would be to return an error code if such a plugin was encountered.
DuskDweller also suggested a addIfNotPresent
flag input to control the behaviour, but this would break the ABI, and I'd rather avoid that.
ATM the getters for load order and active plugins don't do any checking to see if what they're getting makes sense, so if the plugin list contains files that aren't installed any more, they don't get removed.
The workaround is to have a function that gets the load order, removes any missing plugins, and sets it again. Since this is likely a fairly often occurance though, a function in libloadorder would be useful. Same for the active plugins list. In fact, might as well fix them both at the same time...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.