Merges alpha -> master

This branch will be used to do the primary development.

A lot of textures used in marker packs have a bunch of empty transparent pixels, just to pad out the empty areas to fit into a rectangle png image. using texture compression will probably give us a lot of savings in vram space.

This will be tracker issue for all the tasks that need completion before we can release a 0.2 version.

Checklist:

• Jokolink.exe to get MumbleLink info. and gw2 window id.
• Getting the size/position of gw2 window and keeping track of it periodically to resize Jokolay
• Transparency so that we can see the gw2 window in the background
• Input passthrough so that we can click through the jokolay window
• getting global mouse/keyboard status to keep checking whether we want to focus jokolay or let input passthrough
• being able to start with gw2 when lutris launches it via pre-launch scripts
• #12
• Draw egui
• Draw Markers static
• Draw Trails static
• Marker Manager crate
• #8
• global theme file for jokolay

This is a big list, but having atleast this much will make it clear the scope of what we want to do and also set the expectations for future version. having a flexible architecture will help the future versions be much more smooth to tinker with.

Xml Marker Packs

xml_files: xml files are placed anywhere in the pack folder and they will all need to be parsed to get a complete pack.
the format is roughly like this:
OverlayData is the root tag in xml file
optional MarkerCategory tag which is recursive and represents the category menu along with marker attributes which are supposed to serve as "inheritance" properties.

optional POIs tag which contains POI and Trail tags

optional POI tag which represents a marker. two attributes need special mentioning here.

first is the "type" attribute which is an xpath (representing the path to category this marker belongs to).

second is the iconFile attribute which is a relative path from the pack root to a png image.

optional Trail tag which represents a trail. it also has the "type" attribute and its image tag is called "texture".

also has a special attribute called trailData to refer to the path of trail binary data just like png image.

Advantages of using xml packs:

1. compatibility with other Overlays.
2. can add support for attributes slowly one step at a time.

Disadvantages of using xml packs:

1. rust has no mature xml library .
2. editing a marker pack is hard as moving a category menu affects all the markers that refer to them via xpath.
3. the packs in particular have so many errors that blish had to give up and make their own tmllib to have manual parsing with workarounds as they find errors. custom parser means we just lost the whole point of using a popular known format.
4. No defined structure of files. has no reproducibility. once we parse the pack, we will have to remember way too much data like where the marker category tree is in which file, which markers are in which file, what is the order of them etc.. to be able to completely reproduce a pack properly. same goes for order of attributes within the xml tags.
5. case insensitive (atleast in xpath or image/trail binaries paths) so extra steps to normalize case
6. uses UUIDs which are encoded in base64 and no overlay actually ERRORS when there's a duplicate UUID. there's no UNIQUE in the UUID here.
7. having Markers and Trails in the SAME POIs tag, especially when they are interleaved makes deserializing much harder by default in quick xml. having them both in separate tags would have been much better.
8. No separation of markers which are just static and which are dynamic (like having behavior attributes or info attributes which need special care like activation times, text rendering etc..). we will have to check for all of those attributes separately to check if something needs to be done.

In the above issues, some of them are due to lack of documentation/specification of the pack format, some of them due to manual editing of xml files and not checking for errors using linters, others due to no easy editors (which means manual editing) leading to inconsistencies between packs and more or less anyone actually enforcing the pack makers to care about these issues. if a trail binary is missing, there's no error.. so nobody ever knows about it unless they check for it specifically.

Most of the above issues in Xml Marker packs can be condensed into three categories:

1. the lack of a standard file structure + sorting of attributes + sorting of tags like POI or Trail affecting reproducibility.
2. Uses Inheritance of attributes which makes editing harder, as well as dealing with marker packs complicated.
3. Lack of a single source of truth. to look at things like whether color attribute is pre-multiplied or straight, we need to look at the implementation of taco/blish, and if they differ, RIP.

the main issue is that these problems require existing overlays to also support a stricter version of marker packs with the above issues fixed. I doubt they want to do that and break workflows for existing users. So, it is better for us to go with json packs.

I have spent way way way way way too much time on marker packs, and kinda starting to regret this. one of the main reasons to make Jokolay instead of porting taco was that I didn't want to learn specific functionality like taco's UI toolkit which won't be useful to me in the future compared to learning something like Qt or imgui in c++ or egui in rust. Yet, I spent way too much time wasting on this custom format. if it was gltf or some other open format, it might have been okay. we still learnt a bunch of stuff regarding all kinds of formats (text and binary) or the issues regarding representation of data like premultiplied colors or referring to external files like images/trail binaries. but, not enough to justify the amount of time invested.

Why JSON

1. it has support EVERYWHERE. lots of langauges support it natively or via popular defacto standard libraries.
2. has a LOT of tooling/features like jsonpath, json schema, json diff-patch and other RFCs. can be stored in a lot of databases too smh..
3. lots of rust libs like reqwest support it, json-rpc is a thing which is used by vscode for extensions, so obviously supported well enough.
4. easier to edit, format, validate etc.. in all popular editors like vscode or atom. e
5. compared to binary formats, its not bound by specs like protobuf and easily editable by editors.

size difference between xml and json packs was roughly 2 MB for Tekkit's pack.

Databases

just for completion, one more option we considered was databases like sqlite.
Advantages:

1. Very Mature and stable. can be used over network if being shared by people
2. can be edited live by multiple people with atomic transactions. (or multi boxers). can be used by multiple overlays too at once.
3. has god tier tooling for stuff like rollbacks, backups etc..
4. overlays like Burrito/Blish/Electaco which have a spike in ram usage when they parse a marker pack, they can now just connect to database and query only the required markers and cache them. no more splitting marker pack into map_named files like blish does to optimize.
5. literally anyone an just make an overlay without having to worry about parsing or caring about marker formats. universal indeed.

Disadvantages:

1. people need to learn sql now, but most can use abstract ORM libraries to ease things out.
2. migrations can be a bit of a headache when changing the marker format schema
3. no git like data version control.

Result

honestly, databases are an attractive option (especially with sqlite). but I don't want to completely commit myself to a schema yet, especially without consulting with other overlays. so i will choose json based pack. eventually we can use sqlite internally, but when we want to export a pack, we can choose to do so in json. once we get enough experience, we can reconsider databases.

Hey there! I see that this repository has previously contained an GLFW backend for bevy_window, before being removed in d7bcf60.

I ask for permission to use that old code as a base for an actual bevy_glfw crate, where my aim is to improve and maintain it. I am planning to license the fork under MIT or Apache 2.0, as per the Bevy Third Party Plugin Guidelines. I will credit your name.

As an aside, my motivation behind this is that I personally find it inconvenient to use winit due to its lack of proper keyboard input handling, as I'm using a non-qwerty keyboard layout, Colemak, which scrambles the key mapping in games, making it very difficult to use without making an effort to remap all keys to their correct locations. GLFW, on the other hand, always assumes a qwerty layout, no matter where the keys are physically located on the keyboard, which is the preferred behaviour in games. There is work underway to fix this in winit, but it's not done yet (rust-windowing/winit#1806).

Merges alpha -> master

Closes #7 . Tracking branch for alpha release of jokolay

Windowing

Winit on web. Glfw on Desktop.

GUI

egui

Rendering

There's only two 3D renderers in rust at the moment. rend3 and three-d. but rend3 cannot work on web unless webgpu is on stable (and it is very much not stable right now. neither chrome nor firefox on linux can render wgpu samples properly except the very basic ones). webgpu could be in chrome in february 2023.

so, if three-d gets a wgpu backend, then we an use that right now. lets try contributing for now.
fortunately, for the forseeable future, we can easily work with just wgpu as we will only render egui + basic markers / trails (which are custom rendering anyway).

Window Manager is what deals with the layout/position of the windows , decorations, window frames/borders, focus, stacking order etc..
Compositor deals with the drawing part. it draws the decorations on a offscreen buffer, copies the window's framebuffer into the decorations, deals with the translucency and other color stuff before finally rendering it onto the monitor.
First, we must understand the 3 states that Guild Wars 2 can be in.

1. Window: in this mode, it acts just like a normal window with decorations. especially useful if you want to put this on left side of monitor and another app like firefox/video player on the right side to watch a movie. or when you want to keep an eye on it, but not let it occupy the whole screen. very similar in windows or linux, but there is a chance of crashing if using arcdps on linux.
2. FullScreen: you can select various resolutions, and gw2 will render in that resolution by taking over the desktop. the game draws directly on to the monitor's framebuffer instead of the copying that compositor usually makes in window mode. when you alt-tab, game will be minimized and your desktop resolution restored to normal to draw the windows as the control is back to compositor now which does the framebuffer copy into a offscreen texture thingy for the visible windows. back in the day, this was the best way to get full performance as all resources are given to game, and compositor doesn't do much at all.
3. Windowed FullScreen: it is a hybrid mode where the game's decorations are stripped off, game resized to the monitor, but it is still a window. the compositor is still copying the game's window into a seperate texture where it builds the who used to have "slightly" reduced performance in a game in past, but it is almost as performant as fullscreen these days due to compositors d letting the game draw directly on to the framebuffer of monitor unless there's another window visible, at which point, it will take draw that window on top of the game.

On Windows, we use the TOP_MOST attribute. refer window style hints and setting it dynamically . this will set the window to be above windowed fullscreen gw2.

The main issue we will be dealing with is that linux does not differentiate between windowed fullscreen and fullscreen. most window managers will try to "emulate" the behavior of windows. when using windowed fullscreen, they will allow you to alt-tab without minimizing game window and the game's resolution must still match that of the desktop/compositor. so, if gw2 is running on a desktop with FHD resolution, then the windowed fullscreen mode is running at FHD. otoh, if we use fullscreen with different resolutions in gw2, then the desktop also changes resolution just like windows and will even minimize gw2 when alt-tabbing.

But it is just an emulation. the spec does not care about it at all. what the spec recommends is the stack order https://specifications.freedesktop.org/wm-spec/wm-spec-1.4.html#STACKINGORDER which says that a "always on top" window must be BELOW a "focused fullscreen" window. it means that the overlay will be pushed into background when gw2 window is focused (by clicking on it for example) either in fullscreen or windowed fullscreen mode. the key word is "focused", so overlay can be technically on top of windowed fullscreen gw2 as long as it is focused. but its still a recommendation and wm like i3 do not implement that order.

On Linux, we have to use window mode for now. even if overlay stayed above windowed fullscreen mode gw2 in any particular DE, it is accidental and cannot be relied upon as standard behavior. we have a temporary workaround to get "almost" the desired behavior. It must be remembered that there's a lot of window managers and we must target the spec rather than a particular WM to make sure the most people can run it.
Workaround:
We can resize gw2 window to be borderless (decoration less) and resize it to the monitor. this will feel like a fullscreen window except for the taskbar panel being visible. we can set taskbar panel to autohide mode. now, overlay can stay on top of the gw2 window while also feeling like its in fullscreen. one caveat is that resizing without decorations is not fun at all.

A more permanent solution would be to use the "Transient_For" hint and set gw2 as the parent window of our overlay. i have not looked much into this because it would mean adding the dependency for X11 and sending manual commands.
future reference: https://stackoverflow.com/questions/31225743/x11-non-clipped-child-windows/31436071#31436071
The stacking order in netwm spec recommends that the child windows that have "transient_for" set to a parent window must appear on top of it. this also means that we now need to find the gw2 window on the linux side for the particular mumble link we are dealing with.

there's also the option to completely skip the window manager by using override_redirect hint. then, we can avoid place jokolay wherever we want and on top of all the other applications. we would need to experiment and find out whether window manager can place their focused windows on "top" of our override_redirected overlay window.

do u just do cargo run?

when i compiled and ran i got this

�[2m2022-12-24T15:04:23.661712Z�[0m �[31mERROR�[0m crashing: The application panicked (crashed).
Message:  failed to create marker manager: 
   0: markers path is not a directory

Location:
   src/jmf/manager/mod.rs:119

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
Location: src/lib.rs:33

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
�[2m2022-12-24T15:04:23.661863Z�[0m �[32m INFO�[0m Dropping Global    
�[2m2022-12-24T15:04:23.662319Z�[0m �[32m INFO�[0m Destroying 0 command encoders    

Need to decide on a simple format to store the activation data. the structure can just be simple, but will need to decide on a file format. with xml, we don't add new dependency, but it means we need to still deal with xml. otoh, we can take this chance to use a simpler format like toml.

Not sure where logs are saved (if they are), or I'd post them. Essentially switching workspaces in X11 Gnome 45 causes the Jokolay app to close/crash. I've replicated this with alt-tab switching spaces, using a defined hotkey, and also using the new scroll over feature from Gnome 45.

Here's a video clip of it:

at the moment, we use a binding wrapper over rapidxml to filter out any xml errors that we can including deduplicating attributes.

MarkerCategory and POI can actually share MarkerTemplate with a [#serde(flatten)] attribute so that we can simplify deserializing xml packs. but it doesn't work correctly due to quick-xml and serde_xml_rs having issues. stuck on
RReverser/serde-xml-rs#137
tafia/quick-xml#226

once we get that done, we can simplify xml -> json packs easily.

Merges gh-pages -> master

allows for deploying jokolay to github pages

The info attribute needs us to show text. we can do this in two ways.

1. Add a glyph texture dependency and start rendering opengl text. this is more complicated, but we might eventually need text rendering anyway.
2. just use a simple egui area/window to display text. we can keep it to a side, or make a notification-ish floating and fading away thingy or just keep a seperate dedicated window for this which gets a text message of the info string.

right now, we have an assets directory (which we get from env) and have everything like marker packs as well as theme, cache, jokoconig etc.. all in that directory.

eventually we might split these as config, data and cache directories. if we make a linux deb/aur package.

right now, in mumble link, we use the xid provided by jokolink as the identification for an individual instance of gw2. but it means using x11 which is not exactly the fastest means of knowing whether gw2 is still alive or dead.

if we had the linux pid of the gw2 process, we could also be using that instead. it will also match the windows version where pid is the identification factor for a gw2 instance. especially for separating different gw2 instances writing to the same MumbleLink.

but wine doesn't provide any methods to get the linux pid mapped to the abstract wine pid that gw2 puts in the mumblelink.

this issue tracks a bug report made by us https://bugs.winehq.org/show_bug.cgi?id=52724 to wine and also check frequently if there's any progress.

Merges marker_manager -> alpha

We have a pretty good working of marker pack parsing and exporting. still needs polishing, but we need more info about our vulkan rendering and pipeline setup to optimize the structure of marker packs.

This merge will let us start working on vulkan and then later, we can complete the rest of the marker pack manager design.

gui is the heaviest component in terms of resource usage during runtime.

we have two options that are stable / mature enough for use.

1. imgui-rs

Advantages:
mature toolkit. so, a lot of the rough edges are already taken care of. eg: menus and their placement
ffi. if we wanted to share gui with scripts, we could simply pass the imgui context over and pretty much all scripting languages have some sort of bindings to imgui.
features. docking and multi-window are HUGE HUGE improvements and it would be very powerful to have them.

disadvantages:
written in C. so, errors might be harder to debug. and there's always the playing catchup with upstream for bug fixes as well as latest features.
api: the api is still in flux and not complete in rust. had weird hacks like imstr or other stuff.

2. egui

Advantages:
pure rust: easier to debug, build and all the safety advantages because egui doesn't use unsafe.
api: egui api is not ideal, but atleast clean and simple.

Disadvantages:
still new. lots of changes happening and some very rough edges like menus and such going out of window.
lacks momentum of imgui. there's only one core developer and could be abandoned if they burn out. imgui-rs OTOH, has changed hands of multiple devs and still going strong.
features. any complex features like docking or multi-window will take a long while to complete.
ffi. I have no idea how to pass egui context to mods/scripts as it uses a bunch of rust structs like Mutex/Arc.

For now, we will just try to keep UI logic separate from the other parts so that we can swap out the UI based on whichever toolkit becomes more reliable to support. we can even have feature gates to choose which UI to use and run trials with users.

Other alternatives which could be possible options in future:

1. thyme. https://github.com/Grokmoo/thyme . highly theme-able, similar to egui.
2. kayak-ui.https://github.com/StarArawn/kayak_ui . primarily designed for bevy, but could be really useful
3. kas-gui. https://github.com/kas-gui/kas . retained mode option, uses wgpu for backend.

could look into projects like iced, raui etc... to see if they are easy to integrate into a game-like app like ours.

ofcourse, if we shift to either godot or bevy in future as our base, we might reach different conclusions.

Merges alpha -> master

This merge deals with completing the glfw part of jokolay.

This will Deal with Loading, Reading, Writing, Storing, Editing, Creating and Management of Marker Packs from filesystem to providing filtered markers suitable for rendering.

Marker Manager Design

Filters for rendering

after we load marker packs, on a reset event (at startup or map changes etc..), we will generate marker Quads (mesh which has all the required data/defaults set for rendering) on a per player/account basis in a buffer and store the indexes in a map.
optional: we can create a secondary buffer that can be used by MDI or just as alpha multiplier to disable/enable markers based on whatever filters we decide (achievement done, triggered behavior, disabled cat, deleted marker, compute culling, map changed so no need to draw this marker from previous map etc..).

while multiboxing, there can be per account/profile buffers for only drawing markers relevant to that game window. (like each account only enabling certain set of categories, or being in a different map, achievement status etc..) and per player buffers can be regenerated only for that player.

1. where will we store data specific to gw2/markers like achievements (per character or account)? and when will we filter these markers? can this be disabled when the data is actually invalid? can this be manually set instead of relying on api keys or servers?
   Ans: External data like achievement status has nothing to do with marker packs themselves, so they will be dealt with by some other manager like player class. instead marker manager can have a per player hashmap<achievement_id: u32, done: bool> for the ones that it actually needs filters with the default being false (achievement not done, so show the markers). this can be set/updated by the overlay when it wants to and the changes will propagate to filters/renders.

2. in situations of Rendering/Editing Markers:
   what happens on map change? Ans: rebuild buffer
   what happens when user logs into a different character? Ans: same as map change
   what happens when user logs out and logs in with a different account? Ans: rebuild player buffer and other metadata like achievements + activation times.

3. Design a way to specify which textures are needed in gpu (alive textures). Ans: textures union of al player buffers
   what to do with markers when the texture is not loaded yet? Ans: a loading texture or default texture coords could be used
   preferably when to unload textures from gpu? Ans: only when it is not used in player buffers
   What happens on texture manager reset? Ans: you regenerate all the meshes again with the new coords.

4. Add ways to reset the triggers of markers or to ignore certain packs/categories/markers from triggering. Ans: triggers are player specific, so they can be exposed via overlay
   ways to only check triggers for markers which are NOT already activated (or not awake) OR ignored explicitly? Ans: to avoid duplication of checking sleeping (already triggered) markers, we will keep a list of awake markers and will only make them sleep (apply trigger ) IF they are not ignored explicitly (which is a separate hashset maintained by player).

5. add a Activation Data file, either globally or per pack with options for save intervals instead of writing the file at every change to filesystem. Ans: activation data should be based on player, and changes can just set a dirty flag, and a custom timer can write the file if it is dirty at its ticks.
   how will we store data for individual characters or individual accounts or map instances as some behavior are specific to characters/accounts/instances (map IP).Ans: Activation of a marker will include the necessary metadata to store these appropriately.
   what will we do when we do not know if the character logged in belongs to a certain account when we don't have api key or api servers are not working?

6. Add any configuration to MarkerConfig struct which can be part of JokoConfig.

7. add more typesafety with units like pixels, meters, inches for marker attributes. or decide what units we will use for JSON markers and what units are default for the XML markers.

8. add a way to mark a pack as "partially" loaded when we are only loading markers from a particular map. and when user wants to, he can explicitly upgrade to a fully loaded pack seamlessly.

9. Create a Marker Manager that handles the everything from loading to exporting and also filtering/rendering markers (atleast export the relevant glsl-compatible types) so that the core jokolay can handle the drawing part.

10. check if the markers exported by jokolay actually work in taco and blish.

11. Simplify XML to JSON packs conversion, and viceversa. make the code more readable and see if we can save it
    in a map based XML pack like Multi JSON so that blish can also use it for optimized loading.

12. Design a MarkerQuad struct (using crevice for glsl compatibility std430) to be able to be sent to gpu. this could be what we send in for rendering in batches based on texture.

13. Make Single JSON Pack be written to filesystem as a Multi JSON Pack.
    MultiJSONPack will just be markers saved as a HashMap<CateogryID, HashMap<MarkerID, Marker> in a json file named by the MapID.
    this will make possible the loading of markers in readonly packs that are strictly necessary. (or when you only want to add/delete/edit markers to the present map.)

14. Write predicates to filter markers from packs for rendering and dynamic trigger checks

jokolay is divided into core and client threads.

1. core: deals with windowing/rendering/gathering inputs etc.. that must happen on main thread.
2. client: deals with everything else. for this issue, it deals with loading markers from filesystem, filtering them out based on map/enabled categories and finally sending the required marker data for core thread to render them. and ofcourse, update them as we change maps, active categories, activation/triggers etc..

there's two kind of markers:

1. static: which just exist when category is enabled, but don't do anything. easy to deal with and only change on map/category selection change.
2. dynamic: which have triggers with activation times and other actions, they appear and disappear and need to be constantly checked. handling the buffer updates is a challenge with these.

there's three kind of scene contexts:

1. in world: the markers placed at their respective places in the normal game world. need to clear pixels on chatbox/minimap or other UI elements to not interfere with user. we use depth test.
2. main map: when user opens the full screen map, we have to stop rendering the markers in world and instead do a render from top perspective. there's attribute which affects which markers can be rendered. no idea about depth test.
3. mini map: draw markers from top perspective like a map. attribute affects which markers rendered.
   there's basically 2D markers like taco. these are just normal flat rectangles.no idea about depth test.

there's two kinds of rendering:

1. 2D: how taco draws flat rectangles on screen. easier to deal with min/max sizes which are in 2d pixels. depth test is still a little shaky.
2. 3D: how burrito/blish draw them in 3d contexts. painful to deal with min/max sizes. could be a way to use min/max sizes as factors of overall size and figure something out. depth testing is also easier.

Textures:
we do texture atlassing with 2048x248 sized texture arrays which dynamically resize. this could be a hit for performance at higher textures volume. only time will tell.
there's a question about whether we batch the markers by texture. this is absolutely essential if atlassing doesn't workout.
we have no idea on how to deallocate textures or how to keep track of active/alive/needed textures.
also, compression would be cool if we could figure that out.

synchronizing between core and client threads:
the hard part is to know when to update, how to update the gpu buffers on core thread and how to communicate changes.

With lots of distros coming in wayland nowadays we should target wayland too and make it run there.

Also maybe a List of working and todo Stuff could be added to the README.md or in https://github.com/coderedart/jokolay/projects so that ppl who want to contribute can do so more easily.
Is this even still developed?

We will delay the multi-boxing support for now.

Note: although we consider two windows, there's not much difference between two or five windows for any of the windows. the same problems apply.
There's 3 use cases that we need to consider for multi-boxing:

1. two guild wars 2 windows snapped side by side like a tiling window manager, with no overlap. this is the easiest to support, we can just have a huge (almost fullscreen) window and set viewports respectively for different player contexts when they are rendering. OR we can support having two windows and draw markers to each respective window, and draw egui to the window in focus or as selected manually by user.
2. two gw2 windows overlapping on the same monitor. still doable though with one huge window, we now need to know the stacking order, so we can just render the lowest window markers first and go up to the top while clearing the viewport of the next window to render. this will help only display the top level window's markers, as its visible on top. we can do this with multiple windows too, BUT we now need to use scissoring to make sure that the lower windows don't draws don't overlap with the top windows's space, as one window's clears don't clear the overlapping bottom window's colors.
3. two gw2 windows on separate monitors. this is the hardest problem to solve. if monitors have different refresh rates (they probably will), we have to consider the swapchain present rate. this affects multiple issues like egui's refresh rate, how often will a player's ctx be updated, how often will we draw/present the swapchain for all the windows etc.. and ofcourse, the worst case is what if the window is dragged to a position where half of the screen is on one monitor, and the other half is on second monitor.

Finally, egui itself is single window at the moment. multi window is only cool if egui's windows could be dragged between (or out of ) windows.

so, we will just not support the multi window / multiboxing for now.

This is to ensure that issue numbers in GitLab and GitHub are the same

We can have a static website on gitlab pages, as well as our own server.

Jokolay was imagined as a linux native port (cross platform technically) of Taco, primarily intended to be used by RolePlayers to enable them to create custom "Quests" and other players being able to "play" them.

We intended to use MumbleLink + Gw2 Api + arcdps combat data along with a scripting langauge which will enable them RolePlayers to create "Rooms", have a host be the "Dungeon Master", select a "Quest" and start playing with other players. they can assign tasks like "go kill this enemy" and we will use arcdps to track how many enemies they killed to report to DM (dungeon master), or "go to this spot with this item in your inventory" and we will use gw2 api to get items in inventory while mumblelink provides us with the location to track player to emit a successfully completed task event for that player.

now that we are done with the basic overlay on native, i wonder if i really made the right decision. I have spent almost a year and although there's a lot of stuff that i learnt, at the end of the day, every feature i implement takes forever and i feel like electronJS would have gotten me much further and a lot sooner. okay, that sounds a little whiny, but its not all bad. by making Jokolay in rust, we have great performance to show off.

anyway, this issue is to compare alternatives and consider if there's a need to rewrite Jokolay with other languages / platforms instead. although, this also serves as an important pillar to convince myself that i'm not going down the wrong road.

Native Rust

Advantages

1. performance. as in more stuff can be done per frame.
2. resource usage. we do more stuff with less ram / cpu.
3. safety. we almost never need to debug a segmentation fault.
4. Reliability. no surprises. app rarely crashes, and when it does, the logs helps us find the issue immediately.
5. easy to distribute a statically linked executable.

Disadvantages

1. young ecosystem. missing mature or powerful libraries like renderers.
2. very slow to compile. IDE support is very meh.
3. slow iteration. due to compile time and missing libraries and the language itself missing features.
4. cannot run on web. gui crates are VERY limited too. so, egui is our sole salvation.

Native C++

Advantages

1. performance
2. resource usage
3. HUGE gamedev libraries. QT or imgui. bgfx ftw too. vulkanhpp or VMA.
4. IDE support is great with QTcreator
5. stable abi gives us dynamic libraries. we can just pass imgui along to other plugins. or use lua as usual with SoL2 which already has lua bindings for imgui.

Disadvantages

1. unsafe. seg faults will take forever.
2. unreliable. errors are very hard to debug
3. language is horrible. rust atleast gives safety or great typesystem in return for the steep learning curve.

ElectronJS

Advantages

1. reasonable performance
2. much faster to iterate.
3. sandboxed by default.
4. easy to support web + native.
5. HUGE ecosystem. ThreeJS is a god tier library. front end crates like vue / react and their ecosystem.
6. plugins are just JS scripts.

Disadvantages

1. resource usage. obviously.
2. harder to support the passthrough feature
3. performance. much lower throughput compared to cpp/rust.
4. reliability. we have seen Object is Null too many times to understand this part.

This is to ensure that issue numbers in GitLab and GitHub are the same

The idea is to not leave gw2 window when you want to look up an entry in wiki.
https://wiki.guildwars2.com/ is actually a mediawiki server. so, we can use the api exposed by mediawiki.
So, we can use that api in jokolay to render wiki pages in a window within the game itself.

example: https://wiki.guildwars2.com/api.php?action=query&format=json&prop=revisions&titles=Revenant&rvprop=ids%7Ctimestamp%7Cflags%7Ccomment%7Cuser%7Ccontent%7Ccontentmodel%7Ctags&rvslots=*

Fortunately, mediawiki even provides an API sandbox GUI page where we can modify various endpoints and experiment.
example: the above provided api url is generated using https://wiki.guildwars2.com/wiki/Special:ApiSandbox#action=query&format=json&prop=revisions&titles=Revenant&rvprop=ids%7Ctimestamp%7Cflags%7Ccomment%7Cuser%7Ccontent%7Ccontentmodel%7Ctags&rvslots=*

just modify the parameters and get any info you need. we can use the help action to get info about the possible endpoints.
example: getting endpoints/docs of query action -> https://wiki.guildwars2.com/api.php?action=help&modules=query

The returned content of wiki content will use mediawiki formatting
https://wiki.guildwars2.com/wiki/Help:Editing has a helpful guide covering most of the usage includidng templates/images/iwlinks(internal wiki links) etc..
we can probably write a basic renderer for mediawiki using egui.
there's existing api crates too https://github.com/magnusmanske/mediawiki_rust , so we probably don't need to write it.

This is to ensure that issue numbers in GitLab and GitHub are the same

This is to ensure that issue numbers in GitLab and GitHub are the same

references to marker/trail attribute definitions.
https://blishhud.com/docs/markers/attributes/achievement
http://www.gw2taco.com/2016/01/how-to-create-your-own-marker-pack.html

This Issue will serve us in four ways:

1. tracks the implementation status of the Marker and Trail Attributes.
2. This will also serve as a discussion thread for "types" of attributes. eg: instead of having a xpos, ypos, and zpos, we can just say pos = [f32; 3]. we an use such typed improvements to make json format simpler and the structs themselves much eaiser and safer to deal with.
3. display the direct relation between json and xml marker formats, especially what guarantees we can provide when we convert a json to xml pack or viceversa. and any information loss in the process of doing so.
4. document all the attributes and "why" they exist. in what ways they are supposed to be used. and why we support them instead of alternatives.
5. document why we refuse to support certain attributes.

For an attribute to be supported, we must consider its impact in 3 areas:

1. how will it affect the json/xml pack storage formats. how unambiguous will it be compared to other ways of doing the same thing. how easily is it reproducible between xml <-> json conversions.
2. how will it impact the compexity at runtime. how much of a performance impact will it be. this includes cpu/gpu/ram and disk writes usages.
3. for the future overlay developers, will this make it easier for them or harder? because things can only get so complex before devs of overlays (including jokolay), just give up and move on with their life. we are talking about making a stupid mod for a very casual barely alive MMO. for our own sake and future maintainers, lets try to go for simplicity and avoid weird corner cases.

First, we will focus on Markers. these attributes are being worked on at the moment.

• (xpos: f32, ypos: f32, zpos: f32) - position of the marker in world space. required
• mapId: u32 - the map id in which the marker exists. required
• type: string - format "cat1.cat2.cat3" when we want the marker to inherit attributes from cat3 which is a child and inherits from cat2, which is a child of and inherits from cat1. attributes only get inherited if they don't exist already. categories are necessary so that we can only show markers from categories which are enabled in the menu. required.
• iconFile: path - image used as texture for this marker. only png files supported for now. optional.
• iconSize: f32 - default size/scale of the icon. atm, doesn't have units like meters or inches. default 1.0. optional
• alpha: f32 - how opaque you want the object to be by default. between 0.0 (fully transparent) to 1.0 (no transparency. default). optional.
• fadeNear: i32 - in inches (ingame unit). at this distance from player the marker starts to fade out. negative numbers means it never disappears. default is -1. optional.
• fadeFar: u32 - in inches (ingame unit). at this distance from player, the marker completely fades out. negative number means marker won't disappear at any distance. default is -1. optional.
• minSize: u32 - pixels. marker icon, if visible, must be atleast minSize pixels in size. optional.
• maxSize: u32 - pixels. marker icon, if visible can't be bigger than this. optional.
• color: string in format "#rrggbbaa". just use a online color picker and they will give you the hex codes of a color. color tint of marker. optional
• mapDisplaySize: u32 - size on minimap/map. optional
• mapFadeOutScaleLevel: u32 - at what zoom level will marker start fading out. optional
• achievementId: u32 - id of achivement in gw2 api. needs apikey. hides the marker if achivement already done. optional
• achievementBit: u32 - bits representing partial completion of achivement in api. optional
• resetLength: u32 - for beheavior 4, this tells how long marker should not appear after F. for behavior 5, it represents how long the map cycle is. optional
• autoTrigger: bool - tells us if going near marker triggers it. quirk: the triggers need more explanation if its going within a range or pressing F etc..optional
• hasCountDown: bool - whether we should display countdown timer when we trigger it. optional
• triggerRange: f32 - distance at which the marker is triggered by the player. need to check if inches or meters. optional
• toggleCategory: bool - marker should appear in mini map. optional
• infoRange: f32 - distance at which info should be displayed. optional
• info: string - show info when within info range
• copy: string - copy this string to clipboard when triggered
• copy-message: string - show this message when copy is triggered.
• schedule: string - cron expression that tells us times when marker should appear
• schedule-duration: f32 - number of minutes that the appeared marker from cron expression should stay alive before going back to hiding.
• tip-name: string - heading of the tooltip to be displayed when cursor is above this marker.
• tip-description: string - contents of the tooltip above.
• mapVisibility: bool - marker should appear in main map.optional
• miniMapVisibility: bool - marker should appear in mini map. optional
• inGameVisibility: bool - marker should appear in game. optional
• guid: Uuid - used as a unique id for this specific marker. to refer to/store in activation data etc.. optional.
• behavior - integer. it describes the way the marker will behave when a player presses 'F' over it. The following values are valid for this parameter:
  [ ] 0. the default value. Marker is always visible.
  [ ] 1. 'Reappear on map change' - this is not implemented yet, it will be useful for markers that need to reappear if the player changes the map instance.
  [ ] 2. 'Reappear on daily reset' - these markers disappear if the player presses 'F' over them, and reappear at the daily reset. These were used for the orphan markers during wintersday.
  [ ] 3. 'Only visible before activation' - these markers disappear forever once the player pressed 'F' over them. Useful for collection style markers like golden lost badges, etc.
  [ ] 4. 'Reappear after timer' - This behavior makes the marker reappear after a fix amount of time given in 'resetLength'.
  [ ] 5. 'Reappear on map reset' - not implemented yet. This will make the marker reappear when the map cycles. In this case 'resetLength' will define the map cycle length in seconds, and 'resetOffset' will define when the first map cycle of the day begins after the daily reset, in seconds.
  [ ] 6. 'Once per instance' - these markers disappear when triggered but reappear if you go into another instance of the map
  [ ] 7. 'Once daily per character' - these markers disappear when triggered, but reappear with the daily reset, and can be triggered separately for every character

Trail attributes are mostly same.

• type: string - reference to the category just like marker. required
• guid: Uuid - a unique id. required
• trailData: path - file that contains actual trail data which represents the route. without this, trail has no meaning. required
• texture: path - similar to iconFile ofmarker. optional
• animSpeed: f32 - i have no idea what this is. optional
• trailScale: f32- width of the trail. need to find out what units to use for this. optional
• color: u32? - need to check. but similar to marker's color.
• fadeNear: i32 - similar to marker.
• fadeFar: i32 - similar to marker

Attributes that will not be supported.

1. bounce (height, duration, delay).
2. canFade
3. Cull
4. InvertBehavior
5. HeightOffset
6. GUID
7. MapType
8. show/hide
9. rotate (x, y, z)

If we want to support editing markerpacks by multiple people at once, we need a way to to communicate peer2peer to avoid server traffic. And p2p brings the problem of sync-ing a pack when multiple people are editing it.

This issue will serve as the place for discussion, while we first get the jokolay 0.2 out.

for consensus algorithm, we have http://thesecretlivesofdata.com/raft/ which allows a bunch of people to get together (party) and edit a marker pack. Even if any of them goes down, the rest of the players can continue being in sync. https://www.freecodecamp.org/news/in-search-of-an-understandable-consensus-algorithm-a-summary-4bc294c97e0d/ is a good resource for a rough overview.

Hi.
I imported tekkit Markers and was impressed on how nice and easy it works.

When looking at the list of Markers however, the Sorting/Order seems to be a bit mixed up.

This is to ensure that issue numbers in GitLab and GitHub are the same

I thought it would be better for there to be an issue, which clearly tracks the wayland blockers.
We can't support pure Wayland sessions for the following reasons:

1. wine requires X11: wine (and lutris/steam by extension) use X11 windows for the games. So, you cannot run GW2 without X11 (or Xwayland) anyway. Maybe in 2025? https://gitlab.winehq.org/wine/wine/-/merge_requests?scope=all&state=opened&search=winewayland
2. Wayland doesn't support window positions: Wayland has no concept of window "position". we cannot get positions of our window or gw2 window. and we cannot set position of our window. And, without those functions, we cannot move the overlay on to gw2 window. Go express your support on https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/264 , which is just the first step in getting this working.
3. Wayland doesn't provide global cursor position: When our window is "passthrough" (letting mouse clicks hit the window behind us), We are not going to get any mouse events. On X11, we get global cursor position and check if the mouse is on any of our "GUI" areas and decide to disable passthrough feature. But, on Wayland, we cannot get cursor position, and have no idea on when to disable the passthrough feature. Technically, we can use alternate means of doing this, but it would definitely be more like a hack, compared to being a first class feature.
4. less demand: Compared to the vast majority of people who will just use xwayland, very few actually "need" pure wayland support. Should we really be prioritizing this over other features?

I thought this should be documented somewhere.

rust by default uses the guaranteed instruction sets like sse1 and sse2. but most cpus support better features.

The options you give to a compiler are called target_features and we can get the default settings (features) for a configuration by using

rustc -C target-cpu=native --target=x86_64-pc-windows-gnu --print cfg.

ofcourse, the rustc --print cfg is the important part, but the other two options also help a lot in knowing what cpus support which features and what targets support which features by changing them up. target-cpu can be any cpu family name like skylake for intel 6th gen.

to get a list of ALL features available we use rustc --print target-features.

once we are done with the refactoring of the jokolay for alpha 0.2, we can start doing some benchmarks using latest features. if the benchmarks improve performance by a decent margin, we can release jokolay as multiple binaries with and without features enabled during compilation.

it seems that memory usage is at the very least roughly 50MB. after checking other basic triangle windows from glfw, glfw-rs, vulkano-examples and a normal winit window, it seems that this is just normal. I guess we need to check again after we load marker packs, because each pack is like 50 MB :D deserializing that might spike memory usage for a moment

Considering blish already supports Modules, This feature might be requested eventually.

Short Answer: NO. no support for any sort of addons. it is too big in scope and we will consider it after getting the basic overlay stuff working.
Long Answer: Maybe. scripting will help add features easily when they are lightweight. but

1. exposing it for plugins means we need to stabilize an external api and that should not be done hastily, or we will eventually be trapped in backwards compatibility.
2. anyone can make plugins and as we cannot be checking the quality of the plugin, it will be a bad user experience. i know that its not Jokolay's fault for the plugin crash, but we are enabling bad plugins to exist.
3. The most important reason is that we would need to select a language runtime/external api and it is a hard place to be.

Lua obviously makes sense. its popular in gamedev / game modding communitites, has a lot of libraries, fast af. but lifetimes are a pain and rust doesn't have smooth integration with lua yet.
Javascript. nothing needs to be said. every programmer and their mother know this. will be heavily popular with plugin creators. but package management will probably be a little painful to integrate. again, no smooth integration with rust yet.
Python. not as fast as js or lua. but has an okayish integration with python. has lots of libraries and will be easy for newbies that want to make plugins.

Finally, the real candidate would be Web Assembly. one advantage is that we now have speed AND the integration would be smooth considering rust support is first class for wasm. But more importantly, we can write wasm modules in any language and have them compile to wasm, and pack it for rust. But exposing the plugin API in wasm might not be an easy task at all.

No matter which language we select, we will need to provide a sample template repo on github, a guide for setting up the dev environment for plugins, finally a package format for use with Jokolay. thinking in these terms, self contained wasm files should be ez to distribute/package. we could just refer to already established guides about wasm dev setup for different languages. especially kotlin/rust/js/lua which already work with wasm.

too many decisions and we didn't even consider what we want to expose in plugin api, or security, or what happens if a plugin is too slow, do we wait or do we just skip rendering it this frame? or what kinda refactoring would we need in Jokolay itself to keep it clean and finally if we want to cooperate and expose similar api like Blish/ or make our own.

Merges config_manager -> alpha

We have a Core Part of the Overlay which deals with Windowing, Input and Rendering stuff which MUST be handled from the main thread.

We have a Client Part of the Overlay which deals with Texture Management. We maintain a Asset Manager class and we have to make sure to share the pixels and texture atlas layout so that both the core and client can access it.

1. we can just use a simple Arc/Mutex to share the state. but this kinda causes issues with Syncing and stuff.
2. we can maintain the texture manager with egui, and send Texture Commands to core renderer. this will help us abstract away textures and have more control over how and which textures are live.

Merges development -> master

This branch will contain all the development and we will only commit to master a working version to keep master always in a decent state

In GitLab by @coderedart on Oct 29, 2021, 23:34

previous discussion https://gitlab.com/coderedart/jokolay/-/issues/27

Bevy uses ECS, which requires us to split most of the data and its related behavior cleanly. This will allow us to design things and document the requirements explicitly.

This issue serves as list of things to implement, as well as track the progress of the overall rewrite process.

1. Glfw
2. Device Query
3. egui
4. Lua

This is to ensure that issue numbers in GitLab and GitHub are the same

This is to ensure that issue numbers in GitLab and GitHub are the same

Blish uses Mono Engine (Xna Framework) as its base. (can't deal with linux and uses C#. has scripting support with C# and maybe Lua in nearby future )
Burrito uses Godot Engine as its base. (can't deal with windows and okay-ish rust support. has gdscript too)
Taco uses raw dx11. (can't deal with linux and uses c++)
Electaco uses threejs (i think) with electron. (could be cross platform, but just windows-only for now. can use rust via wasm probably. has js for scripting)

Jokolay uses raw wgpu (webgpu with vulkan backend). can do both linux and windows. uses rust. will probably use Lua as scripting lang.

But there is some merit on being based on a game engine. especially for a lot of things like gltf support or sound etc.. which are readily provided in an engine rather than having to figure it all out ourselves. Although, it is a lot of fun to know how all of this works, its not exactly productive if we want to add things like video markers.

So, this issue will track the status of other game engines and their viability to work for us.

All things considered, we are doing reasonably fine imo. its just that we chose a language like Rust with its young undeveloped ecosystem instead of C++ which would have been a breeze with either godot or a hundred other convenient libraries like Sol2 for lua / imgui.

part of #31

Bevy Glfw is the crate to be used as the windowing backend for Jokolay.

The main reason to use Glfw over Winit is the PASSTHROUGH feature. As the feature is still not released in Glfw Stable releases, we will use the patched glfw-rs and glfw-sys repos. we must shift back to piston glfw crates when PASSTHROUGH is in stable release.

Functionality of Bevy Glfw and its implementation status:

• Create a Glfw Window
• Transparent Window
• Decorations toggle
• Work with the Event loop
• Translate Events from Glfw to Bevy
• Passthrough
• Event Handling for Passthrough windows as they cannot get cursor positon etc..

when two gw2 instances are started, by default, both of them write to the same MumbleLink shared memory.

when we see a uitick being less than previous tick, at the moment, we assume that previous instance has crashed and we are looking at the data of a new gw2 instance. this will lead to jokolay switching between both windows and acting "crazy" for newbie multi-boxers.

Case 1: only supporting one active mumblelink per active running gw2 instance

we must explicitly check that the previous gw2 instance is still alive or dead, and if its alive, show a notification or messagebox to inform user that we are following a new gw2 instance now and that they should use different mumblelink names to support jokolay's multiboxing requirements.
this will be a good way to annoy the newbie multi-boxers to actually use different mumble link names.and it will only be annoying to them and the rest of users would never see this message.

Case 2: multiple gw2 instances can write to the same mumblelink memory

we will differentiate between them based on the pid/xid from the data.this is bad. now our mumble data per instance is not updated as often and we are going to "skip" frames. the alternative is to update mumble more often which is pointless when there's only one gw2 instance.

We will go with Case 1 for now.

In most maps the paths are cut somewhere and meet underneath it.

Primary issues with existing marker packs:

1. undefined filesystem structure.
   1. categories are spread across files and we need to parse all of them to get the final category menu tree.
   2. markers from a specific map are spread across files. need to parse all of them for a single map's markers.
   3. images and trl binaries can be anywhere.
   4. case insensitive paths (okay on windows, but won't work on linux).
2. inconsistent attribute names. eg: MapID, xpos, type, GUID, trailData all have different case
3. 2D specific design:
   1. marker / trail size based on texture...? 2D billboard size based on texture width / height. what about 3D billboards tho..?
   2. minSize / maxSize constraints only for 2D. do we just ignore for 3D?
4. units are inconsistent. xpos is in meters. fadeNear is in game units (inches). color uses u8 for components like alpha in range of 0..255. alpha attribute uses float in range of 0.0..1.0
5. Inheritance of attributes from categories. makes it less explicit as to where a certain attribute value for a marker / trail comes from.
6. Lack of documentation. need to go through blish / taco codebase to figure out the implementation details. If both decide to disagree on a certain attribute's semantics, who do we follow?
7. XML syntax errors. we had to make a rapidxml bindings for rust.. ugh. prevents wasm support as c/cpp bindings aren't supported in wasm32-unknown-unknown.
8. POI and Trail tags are mixed in with each other under the same POIs container.
9. not exactly marker pack's issue, but rust support for xml is very poor.
10. using UUID which could contain duplicates. and really not necessary as a marker already has a unique identity with its category + mapID + pack + its position inside POIs.
11. hard to edit a marker pack.
12. trl binaries are not in model space. the positions (vec3) are not aligned at beginning properly due version + mapID fields.

If we don't care about editing though, pretty much all of these issues can be just a one time workaround.
sure, it won't be the most efficient way of doing things. but readonly packs can still be significantly performant as we will

• inherit all attributes when we load marker packs
• intern all strings (attributes like tip-description)
• load the whole zip files or all its images / trl binaries into ram in advance
• prepare texture atlas when we load into map for fewer texture bindings. even use texture arrays.
• cache draw calls (especially with indirect rendering) as we only need to show / hide markers based on activation data(storage buffer of bools / bitflags). and no changes in meshes / textures etc...
• convert the relevant attributes like alpha into u8 etc.. in advance while loading pack.

and in terms of advantages:

1. compatibility with other overlays. can just reuse existing marker packs hosted online
2. don't need to write an editor for this at all
3. simpler implementation and quicker release. no custom json pack <-> xml pack conversions
4. jokolay's scene format design is not restricted by trying to support conversion from / into xml packs.

some disadvantages like lack of documentation still exist. but whatever.

previous attempt #19 to use Json packs which are a super set of marker packs is way too hard. it is better to just let Json Packs be their own scene format with lua support.