istador / pine-interactive-map Goto Github PK

Add a readme to the repository that explains how to extract data from the game.

I've already written a short readme, but the wording is terrible (for non developers), the tools are awful and it takes several hours or too much manual involvement.

For the moment, this might be helpful for you:

• They are located in .\Pine_Data\StreamingAssets\bundles\
• The .manifest files can be greped to find out what is in which file.
• The files are compressed .asset files - they may need to be decompressed depending on your tool.
  • UABE can only open one compressed file at a time, but offers batch decompression (but no batch export :/ ).
  • UABE needed ~5-10 minutes to decompress all but ~9-12 hours to open all .asset files at once (=> open only the ones you need, based on the .manifest).
• Game Objects and Transforms are separate from each others and reference to each other via Path-ID and File-ID.
• Be careful when dealing with the Path-IDs. These IDs are 64-bit signed integers. Not all tools handle them well.
  • jq gives you back wrong values without an error or warning.
  • UABE has the Path-ID signed in the extracted JSON and unsigned in the filename.
  • unsigned2signed: echo $unsigned | awk -M '$1>=2^63{$1-=2^64}1'

Im fairly new to coding and JS in general. Im trying to use this project to help understand leaflet and i like the game. When i download the files and open the html file it just says. "requires javascript......." Im also not seeing any map images in the folder either. That could be useful to me if someone could help me out. Maybe write a quick guide for this or provide me with a download that would meet these problems. Thanks.

Follow up to #6.

Extract coordinates from the game assets using uTinyRipper instead of UABE, to have less manual steps and the item amounts (and item IDs) in the output.

What to extract/export:

• Item IDs
• Item amounts
• Things being inside of instanced areas
• Caves
• Villages
• Village Merchants
• Village Donation Boxes
• Scavenger Fragments

extract.sh

• rewrite in python to improve calculate speed
• only export active game objects
• exceptions for game objects that are activated dynamically
• respect localRotation when calculating coordinates
• fix Item IDs for chests and ideas

pretify.py

• script to detect changes between game versions, that auto-confirms depending on if and what has changed
• add columns for title, description, source and screenshot that are manually changed but persist between versions

wikify.sh

• script to transform csv data into wikitext
• create a reverse function that transforms from wikitext to csv (as there being a lot of manual changes by now only in the wiki)

The language selection takes up too much space. When the other four languages are added even more.

By default, only the selected language should be visible. When it is clicked (or hovered?), the other languages should appear to allow selecting them.

Allow all layers of one type to be linked by only using the type.

Examples:

• #layers=idea instead of #layers=idea.chest,idea.pickup,idea.quest
• #layers=area instead of #layers=area.1,area.2,[...],area.21

• duplicate data source
• mark all entries as needs check
• check if there is a new version of the in-game map texture and generate new tiles if needed
• extract all coordinates from the new game version
  • add new entries
  • remove what was there before and is now missing
  • auto-confirm what hasn't changed ans was confirmed before

Bugfixes:

• Item-IDs for Chests and Ideas
• Missing Chest 17
• Scavenger's Fragments
• Add title, description and source to the prettified csv files

When there is a new version of this website, the user needs to manually update his cache to get the newest version, e.g. by reloading with CTRL+SHIFT+R.

To automatically update it, all generated build artifacts (*.min.js and *.min.css) should contain the content hash in the filename, which needs to be put into the index.html (via html-webpack-plugin).

Build artifacts should be kept in a sub directory and not in the root folder, so that old versions can easily be kept for a few days and being deleted later.

Define for each file how long the Browser can cache it.

E.g. the build artifacts can be cached for a long time (several days), because if they change they generate a new file with a new path. Whereas the index.html needs to be fetched frequently to get updates and fixes to the software.

It should be possible to link to specific markers by using their ID, e.g. with #id=123. The popup of the marker should open and the marker should be centered and zoomed in on. The popup should offer a permanent link button as well as changing the site's history when opening and closing.

Highlighting a specific coordinate pair (drawing a circle around it, center it and zoom in) should also be possible, e.g. #x=200&z=-345&zoom=4.

Linking to a specific layer selection might also be useful, e.g. #layers=entrance.vault,unique.idea (but should not be saved unless the user changes the selection manually).

The site might need to be able to react to manual changes to the hash portion of the URI by the user, by opening up popups, (de-)highlight coordinates and pan/zoom/center around.

This might be very useful for other websites (e.g. the Pine Wiki) or users that want to point to specific POIs or coordinates within the game.

Follow up to issue #18:

Allow several IDs to be linked at once.
E.g.: #id=123,456

Then it should circle all markers, move the camera to the bounding box of all IDs and not open a popup.

Maybe even allow several coordinates at once also (e.g. #x=-10.2,-30.3&z=-45.5,40.5) and add a bounding box parameter (e.g. #bb=-10,-10,10,10).

When the data source contains the same ID twice (human error), the interactive map should not show the ID and not allow the user to mark one of the markers as completed, as this might mark the wrong one as completed.

Similarly, if the user already marked something as completed, but later the ID is detected as an duplicate, then the marker should no longer be styled and handled as completed.

(Optional / open for feedback:)
When an already completed marker is later detected as a duplicate, the application should remove the completed flag from the localStorage, to prevent that the mark moves to another marker when the issue is resolved in the spreadsheet.

[Edit:]
On the other hand, when someone changes something temporarily in the spreadsheet, the user might loose all of his completion data, when he access the page at the wrong moment.
=> Don't implement this.

When the spreadsheet was created by thymaro#5985, there were columns for beta1, beta2, release1 and release1.X with the values: confirmed and needs check.

I removed the columns beta2, release1 and release1.X, because they are currently not in use, take up visual space in the spreadsheet, and can easily be added later, when the need for them arises.

Currently the beta1 column is misused to mark which columns need to be verified by other users, and not if it is contained within the beta (they all are currently included by definition, except for mistakes).

While the idea is good to track which entries exist in which version of the game, it's not going far enough and isn't great to maintain.

• When items are removed or added from the game, a simple confirmed or missing is enough.
• When items are moved to another location, they might get marked as missing and is marked as new in the new location.
• There is no way to figure out what has changed between versions.
• There is only usable data for the most recent game version.

What I have in mind is having one sheet per game version that contains all data that exists in that version. Data that is missing is removed in the new sheet, new data is only added to the new sheet and changes can be made to all fields in the new sheet (except for the ID).

When a new game version is released, you just copy the most recent sheet and use that as the master.

Contra: ID generation need to be unique over all sheets. The same ID in different versions refers to the same game object.

This would also allow the interactive map to offer a view to older game version, where it just uses an older sheet as a data source, reinitialize all markers and changes the baseLayer to load different map tiles (images).

The layer controller (or a custom controller directly below it) should contain the option to switch between the following modes:

• Show all markers.
• Hide all markers that are completed.
• Show only the completed markers.

The option that the user selects should be saved in the localStorage for future website visits.

Because Pine will be localized into several languages, the Interactive Map should also support displaying different languages.

The selected language should be saved in the localStorage.
A default language should be selected based on the client's browser language (if able).

Other people should be able to contribute other languages via Pull Requests.

• Support UI strings by language parameter.
• Support Names by language parameter.
• Exchange components / strings on language change.
• Implement a language selector.
• Auto-select a language based on the language of the browser.

See stefanocudini/leaflet-panel-layers#51

Options:

• Waiting for the issue to be resolved
• Fork the project and implement the change by myself (and create a pull request)

hi
you would probably be interested in building a new map with a much higher level of user interaction!
This I could do by combining the work you have done on this map by connecting the data and using a new platform designed specifically for geosocial networks and/or web games like these.

KeplerJs.io

it's open source and you can use it as much as you want;)

in addition to the official demo(https://demo.keplerjs.io/) you can find one of the first applications created with kepler online here, a climbers social network
climbing.social

Currently, IE11 just shows a blank white page, because it doesn't understand newer ECMAScript syntax.

TODO: transpile to ES5 with babel and polyfill missing functionality.

Any malicious person (or bot) that knows the URL to the spreadsheet can modify it or delete entries in it.

Currently I do backups of the spreadsheet from time to time to not lose data already inserted.

To not publish the URL of the spreadsheet, the URL of the interactive map is currently not public and this repository doesn't contain the real .env file.

In the future either the spreadsheet needs to be protected in some way, or this project needs another data source.

Ways to protect the sheet could be:

• Allow only specific persons to edit it.
  • Contra: discourages collaboration.
• Allow only specific persons to edit the main sheet and offer additional sheets to contribute new data or corrections.
  • Contra: corrections are separated from the actual items, harder to insert and need to be merged to the main sheet.

Other data sources could be:

• Website with a database and forms to edit and add new data, administration area for verification.
  • Contra: too much effort to build.
  • Easy to use CMS?
• Hard-coded JSON file(s) on Github with Pull Requests.
  • Pro: easy to integrate into the JavaScript application.
  • Pro: easy to integrate into this repository (extra branch).
  • Pro: transparent change history and authorship.
  • Pro: Github CDN
  • Contra: requires Gitlhub account to contribute.
  • Contra: too complicated for the average contributor to manipulate JSON and learn Git nomenclature?
• Wiki with page protection.
  • Pro: URLs that can be linked to within popups.
  • Contra: Where to host?
    • self hosted? => slow.
    • Wikia aka. fandom.com? Needs to be created anyway by somebody. So, why not now by me?
    • [Edit:] https://pine.gamepedia.com/

Investigate:

• How do other game maps host their mapping data?

All icons should be combined into one image file and being selected via background-position.

Currently, each icon is very small and has its own designated file, which is bad to load for the browser.

I already worked with spritesmith in the past and would use it again.

Also there is a little visible delay after adding the markers and the layer controller to the map, where it fetches the images, because the icon places in the layer controller are briefly empty. When all images are in one file, one (transparent) pixel of that file can be added as an <img> to the DOM to pre-load it.

Currently the game shows the coordinates all the time, and the coordinates are more than enough to find the items from the map in game.

But the game will probably not show the coordinates all the time in the release version. It might not even show them at all anywhere (very likely, because this isn't a technical/mathematical world and it would break immersion).

From game play experience with Breath of the Wild I know that it isn't that easy to find the right things in the game based on an interactive map. Screenshots do help - a lot - which is why the popups should have the option to include an image.

The data source should contain two fields for that:

1. A URL to an thumbnail of the screenshot that is displayed inside the popup
2. A URL to the high resolution version of the same image, that can be linked to with an <a>nchor on the thumbnail.

Collecting and hosting the images and generating the thumbnails is out of scope of this project, but I'd suggest a standardized thumbnail size of 270p or 360p. Images with other aspect ratios should be cropped to it.

The UI of the map should use shades of green, like the game itself and it's website.

Merge the existing settings for version, language, and styling for completed/unconfirmed markers into a popup menu, that only has one visible button on the map (instead of currently 9 buttons + 3 radio buttons).

Inside of the new settings menu, there can also be a few more options:

• button: reset settings
• button: reset completed markers (with confirmation)
• text: about the map
• links: to imprint, github and wiki (remove from attribution?)

This should reduce clutter and benefits issue #11

Add meta tags, so that search engines can show flavor text.

Follow up to issue #15.

The current beta build only offers the English language, but the final game is going to be translated into several languages.

To offer localized item names in the map, the strings should be extracted from the game.

• English
• German
• French
• Italian
• Spanish
• Dutch
• Portuguese
• Chinese (too hard to transcribe for me)
• Russian (too hard to transcribe for me)
• Korean (too hard to transcribe for me)
• Japanese (too hard to transcribe for me)

Chinese, Russian, Korean and Japanese need to be copied from the game assets (somewhere) or being transcribed by someone else than me. It's too hard for me to transcribe them character by character without proper keyboards for it.

I already fixed this with 6700377 for Chrome and Firefox, but the fix had no effect for IE11.

It's caused by https://pine.gamepedia.com rejecting embedding images with referrers.

By JDM12983 on Discord:

One option that would be nice addition for the item selection on the right (choosing which things to show or hide); would be "Select"/"Show All" check box - to select everything -- or, at least, and "Select All" for each section [or at least the few sections that have a bunch of boxes.

What should be possible is to add very small buttons to each layer group (Entrance, Food, Item, ...). Or at least indeterminate checkboxes, if buttons would take up too much space inside the UI element.

This feature should be added to leaflet-panel-layers, but likely I'll have to do it myself (my other pull request haven't been merged yet).

By thymaro#5985 on 2019-07-23 at 09:31 in Discord:

when I open the page in FF, the map is at the top of the page and I see a lot of water. The map is only half visible instead of centered in the browser window.

I also noticed this a few times during development (Chrome and Firefox). The map is bugged and limited to an decreased area of the viewport at the top.

It isn't a constant behavior for me, but it only happens rarely. It is usually fixed by reloading the page or by resizing the window. I couldn't reproduce it reliably yet to investigate further.

I think I might initialize the map to early, call fitBounds too early, or defer isn't working correctly and I need to move the <script>s to the end of the <body> or even manually wait for DOMContentLoaded.

Shouldn't be used by anyone anymore, except out of curiosity. It might confuse people.

• Remove version Beta 1.
• Remove version Beta 2.
• On an unknown value in pine-selected-version use the newest version (seems to work that way already).
• Hide the version selection, if there's only one version (keep the functionality in there, e.g. for the local development version).

The area images that are hardcoded are working.

But the screenshots from the wiki data don't work, because they have URLs like https://pine.fandom.com/wiki/Collector's_Dream?file=[[:File:Screenshot-Amphiscus_Orb-48.jpg]] instead of https://pine.fandom.com/wiki/Collector's_Dream?file=Screenshot-Amphiscus_Orb-48.jpg.

This is likely because of the wtf_wikipedia update from 098e317, but I thought I tested that when I made that change.

I found the following two files in the game assets.

RegionMap.png:

AreaMap.png:

It should be easy to combine and transform these two PNG images to SVG files, label/identify them, and use them on the map to highlight and select specific areas on the interactive map.

Example on how this might look from the interactive Subnautica map:

• https://subnauticamap.io/

They should get popups with mark as complete buttons (e.g. to mark that you got their Glossary entry) and permalinks, e.g. #area=Tall_Ridge or via IDs.

This might also be done for caves and vaults. For those I found terrain images (that aren't used within the game) like the ones I use to render the game map itself.

I need to render them and see if they might be useful for the map, or if I just use their alpha channel and display them monochrome like the areas. They need to be positioned correctly within the game coordinates, because the images only contain an extract for their own terrain.

The layers might then be coupled with the entrances and the other POIs that are inside of the instances. Like on the Subnautica map. (I already export the information which POIs are inside of an instance with the area column.)

These SVG files should also be useful for the wiki to create an overview image that shows all areas. Or even individual images for each area (though linking to the areas on the map should be good enough).

Additional things that seem to be important enough to have designated icons:

• entrance.mohlenhill (I think they'll be fast travel points)
• entrance.vault
• item.equipment
• material.beagalite (Beagalite sword)
• material.gravelmoss
• material.lunarados
• unique.journal

The types entrance, food, material, mechanical and nest should each have their own default icon, that might automatically get a different color tint for each item (e.g. food.roseberrys in red, food.obergine [sic] in purple).

• entrance
• food
• material
• mechanical
• nest
• Tint icons without specific images.

I could make a new Chrome user profile or something, but surely there's an obvious way to reset the check boxes on this map, and I'm just missing it?

Thanks for all your effort on this, it's very helpful!

Follow up issue of #25

This should not change anything for the normal desktop mode (and tablet mode?).

Remove:

• Zoom buttons
• Fullscreen button
• Panel that displays the mouse coordinates

Change:

• Collapse the layerController.
• Adjust the size of the layerController to be readable.
• Adjust the size of the popups to be readable.
  • ... or even decouple them from the map as an own fixed <div> overlay or modal.
• Offer @2x tiles (512x512 instead of 256x256)
  • See commenthol/gdal2tiles-leaflet#6
  • I already tried this, by changing 256 to 512 in the source code of gdal2tiles-leaflet.
  • Alternative to rendering twice: set L.tileLayer's tileSize to 128

Follow up issue of #25