umautobots / gtavisionexport

Code to export full segmentations from GTA

GTAVisionExport

Code to export full segmentations from GTA

gtavisionexport's Issues

Coordinate systems and transformation matrices

In the dumpData method you calculate viewMatrix and projectionMatrix which is then stored to database along with other snapshot data.
Both are 4x4 matrices, but when I see the values, projection matrix has all elements non-zero, but view matrix has last row [0, 0, 0, 1] in all cases.
And here starts my confusion. View matrix seems like representing rotation and translation in homogenous coordinates, and I do not understand between which coordinates systems it performs the transformation.
Also, I thought projection matrix reduces the dimansion, and projects object on some hyperplane (to space with lower dimension), but since the projection matrix is 4x4 with all elements non-zero, it does not look like performing projection.

Also, I do not understand how exactly does depth in depth matrix exactly relate to coordinate system of the world and how does it relate to viewer coordinate system (from camera's perspective).

I tried to track where do these matrices come from, but I ended here where it seems to me like matrix values are somehow magically being filled without calling any GTA V related function.

I would be glad for any clarification.

No need for "experimental" here.

GTAVisionExport/native/src/main.cpp

Line 23 in 717be0d

using namespace std::experimental::filesystem;

https://en.cppreference.com/w/cpp/filesystem

The filesystem library was originally developed as boost.filesystem, was published as the technical specification ISO/IEC TS 18822:2015, and finally merged to ISO C++ as of C++17. The boost implementation is currently available on more compilers and platforms than the C++17 library.

on my compilation it said unable to find filesystem but worked after removing "experimental".

Disclaimer: I am still kind of a noob in c++

module machine type 'x64' conflicts with target machine type 'x86'

I use followed the instruction and get this error

Severity Code Description Project File Line Suppression State
Error LNK1112 module machine type 'x64' conflicts with target machine type 'x86' GTAVisionNative C:\Users\yipengm\Desktop\GTAVisionExport-master\GTAVisionExport\master\native\build\src\ScriptHookV.lib(ScriptHookV.dll)

I used nuget to install MinHook

where can we get the intrinsic parameter of the camera?

Only NativeUI managed to load

I have built everything and setup a DB and created the tables.
When looking into ScriptHookVDotNet2.log I see:

[20:33:54] [DEBUG] Starting 1 script(s) ...

and that one script is NativeUI:

[20:33:53] [DEBUG] Found 1 script(s) in 'NativeUI.dll'.

The managed part of the tool does not seem to work. Nothing in the database.
Have been struggling with this for days @barcharcraz @krosaen @IanKirwan please help
I am going to post logs below.

Problem in running the whole data collection pipeline

Have anyone tried the complete process of data collection so far. I am using following pipeline:

1. After game starts, I press Page Up in order to start the session.
2. Then I press the key H to enter in to the vehicle and start navigation. (Sometimes after getting in to the car I have to pause and resume the game in order to make auto driving start)

Once auto drive starts it never stops, even if I press Page Down. I can see entries in tables instances, runs, sessions and systems only.

By looking in to the code when I try to stop the session, I think it waits to finish the run task but I am not sure how long it will take to finish the task is there any way to control it somehow?

Any tips will be really appreciated.

Version problem about managed branch

Is there anybody can give the concrete version problem about ScriptHookVDotNet2 and VAutodrive? Also if the dll files can be shared would be great.

Post processing code for depth and stencil

Hi @barcharcraz , thanks for this amazing work!
I'm having trouble processing the captured tiff image. Would you like to post the post-processing code for depth and stencil? It's still a mount of work figuring out the exact logarithmic encoding the game developer used in depth buffer, and for stencil the object ID and "some flags(according to the paper)" are still unclear.
I started with this article decoding the depth buffer, but I'm not sure if z = log(C*w + 1) / log(C*Far + 1) * w //DirectX with depth range 0..1 is the exact way they encode the depth. Having your post-processing code will really save a lot of effort for me and the community.
Thanks for your time!

how to change the camera into the Orthophoto projection ？

Thanks for your Great work!

Your code helps me a lot!
When I create a new camera, I find that the resulting camera works with the Central projection.
Now, I want to create a new camera with the Orthophoto projection.
How can I change the code to do that?

• I guess there must be a function in the CAM namespace that can solve this problem, but I can't find it.

• Meanwhile, I am not sure of the mean of the parameters of the function:

        static Cam CREATE_CAM(char* camName, BOOL p1) { return invoke<Cam>(0xC3981DCE61D9E13F, camName, p1); } // 0xC3981DCE61D9E13F 0xE9BF2A7D
	static Cam CREATE_CAM_WITH_PARAMS(char* camName, float posX, float posY, float posZ, float rotX, float rotY, float rotZ, float fov, BOOL p8, int p9) { return invoke<Cam>(0xB51194800B257161, camName, posX, posY, posZ, rotX, rotY, rotZ, fov, p8, p9); } // 0xB51194800B257161 0x23B02F15
	static Cam CREATE_CAMERA(Hash camHash, BOOL p1) { return invoke<Cam>(0x5E3CF89C6BCCA67D, camHash, p1); } // 0x5E3CF89C6BCCA67D 0x5D6739AE
	static Cam CREATE_CAMERA_WITH_PARAMS(Hash camHash, float p1, float p2, float p3, float p4, float p5, float p6, float p7, BOOL p8, Any p9) { return invoke<Cam>(0x6ABFA3E16460F22D, camHash, p1, p2, p3, p4, p5, p6, p7, p8, p9); } // 0x6ABFA3E16460F22D 0x0688BE9A

1. what is the mean of ''BOOL p1" of CREATE_CAM?
2. what is the mean of '' BOOL p8, int p9" of CREATE_CAM_WITH_PARAMS?

Please help.

Thanks!

Possible to get road segmentation?

Now it seems like buildings and roads are of the same color.

how to get intance segmentation label in GTAV?

R.T

Managed plugin not interacting with native after compile.

I sucesfully got native plugin working and compiled all 3 projects in the managed plugin solution, and copied the compiled files into my GTA V dir/scripts.
But when running the game and pressing L, the native plugin takes the screenshot and creates *.raw files, but then nothing happens, no data is persisted into the database. Logs do not show any error report.
Is there some way how to find out why it does not work or what is wrong?

This is output of ScriptHookV.log

// GTA V SCRIPT HOOK (build Sep  3 2017, v1.0.1180.2)
//     (C) Alexander Blade 2015-2017
[19:55:52] INIT: Started
[19:55:52] INIT: Success, game version is VER_1_0_1180_2_STEAM
[19:55:52] INIT: Registering script 'ScriptHookVDotNet.asi' (0x00007FFD55CC1380)
[19:56:10] INIT: Pool 1 extended
[19:56:10] INIT: Pool 2 extended
[19:56:10] INIT: Pool 3 extended
[19:56:10] INIT: Pool 4 extended
[19:56:16] INIT: GtaThread collection size 189
[19:56:16] INIT: wnd proc 0x00000000FFFF02AB
[19:56:16] INIT: IDXGISwapChain 0x00007FFD58D0C018 (0x00007FFD58CB2140)
[19:56:16] INIT: IDXGISwapChain set
[19:56:16] INIT: DX init started
[19:56:16] INIT: DX feature level B000
[19:56:16] INIT: DX init succeeded
[19:57:56] CORE: Requesting thread creation
[19:57:56] CORE: Creating threads
[19:57:57] CORE: Started control thread, id 28 active 1
[19:57:57] CORE: Started thread 'ScriptHookVDotNet.asi' (0x00007FFD55CC1380), id 29 active 1
[19:57:57] CORE: Launching main() for 'ScriptHookVDotNet.asi' (0x00007FFD55CC1380), id 29
[20:13:38] UNINIT: Unregistering script 'ScriptHookVDotNet.asi'

And this is output of ScriptHookVDotNet2.log

[19:57:57] [DEBUG] Created script domain 'ScriptDomain_B3A330DE' with v2.10.3.
[19:57:57] [DEBUG] Loading scripts from 'D:\Program Files (x86)\SteamLibrary\steamapps\common\Grand Theft Auto V\scripts' into script domain 'ScriptDomain_B3A330DE' ...
[19:57:57] [INFO] Loading assembly 'AWSSDK.dll' ...
[19:57:59] [DEBUG] Found 0 script(s) in 'AWSSDK.dll'.
[19:57:59] [INFO] Loading assembly 'BitMiracle.LibTiff.NET.dll' ...
[19:57:59] [DEBUG] Found 0 script(s) in 'BitMiracle.LibTiff.NET.dll'.
[19:57:59] [INFO] Loading assembly 'FreeImageNET.dll' ...
[19:57:59] [DEBUG] Found 0 script(s) in 'FreeImageNET.dll'.
[19:57:59] [INFO] Loading assembly 'gdalconst_csharp.dll' ...
[19:58:00] [DEBUG] Found 0 script(s) in 'gdalconst_csharp.dll'.
[19:58:00] [INFO] Loading assembly 'gdal_csharp.dll' ...
[19:58:00] [DEBUG] Found 0 script(s) in 'gdal_csharp.dll'.
[19:58:00] [INFO] Loading assembly 'GTAVisionExport.dll' ...
[19:58:00] [DEBUG] Found 0 script(s) in 'GTAVisionExport.dll'.
[19:58:00] [INFO] Loading assembly 'GTAVisionUtils.dll' ...
[19:58:00] [DEBUG] Found 0 script(s) in 'GTAVisionUtils.dll'.
[19:58:00] [INFO] Loading assembly 'INIFileParser.dll' ...
[19:58:00] [DEBUG] Found 0 script(s) in 'INIFileParser.dll'.
[19:58:00] [INFO] Loading assembly 'MathNet.Numerics.dll' ...
[19:58:00] [DEBUG] Found 0 script(s) in 'MathNet.Numerics.dll'.
[19:58:00] [INFO] Loading assembly 'Microsoft.Extensions.DependencyInjection.Abstractions.dll' ...
[19:58:01] [DEBUG] Found 0 script(s) in 'Microsoft.Extensions.DependencyInjection.Abstractions.dll'.
[19:58:01] [INFO] Loading assembly 'Microsoft.Extensions.Logging.Abstractions.dll' ...
[19:58:01] [DEBUG] Found 0 script(s) in 'Microsoft.Extensions.Logging.Abstractions.dll'.
[19:58:01] [INFO] Loading assembly 'Microsoft.Extensions.Logging.dll' ...
[19:58:01] [DEBUG] Found 0 script(s) in 'Microsoft.Extensions.Logging.dll'.
[19:58:01] [INFO] Loading assembly 'NativeUI.dll' ...
[19:58:01] [DEBUG] Found 1 script(s) in 'NativeUI.dll'.
[19:58:01] [INFO] Loading assembly 'Npgsql.dll' ...
[19:58:01] [DEBUG] Found 0 script(s) in 'Npgsql.dll'.
[19:58:01] [INFO] Loading assembly 'ogr_csharp.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'ogr_csharp.dll'.
[19:58:02] [INFO] Loading assembly 'osr_csharp.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'osr_csharp.dll'.
[19:58:02] [WARNING] Skipped assembly 'ScriptHookVDotNet2.dll'. Please remove it from the directory.
[19:58:02] [INFO] Loading assembly 'SharpDX.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'SharpDX.dll'.
[19:58:02] [INFO] Loading assembly 'SharpDX.Mathematics.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'SharpDX.Mathematics.dll'.
[19:58:02] [INFO] Loading assembly 'System.Runtime.InteropServices.RuntimeInformation.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'System.Runtime.InteropServices.RuntimeInformation.dll'.
[19:58:02] [INFO] Loading assembly 'System.Threading.Tasks.Extensions.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'System.Threading.Tasks.Extensions.dll'.
[19:58:02] [INFO] Loading assembly 'VAutodrive.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'VAutodrive.dll'.
[19:58:02] [INFO] Loading assembly 'VCommonFunctions.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'VCommonFunctions.dll'.
[19:58:02] [INFO] Loading assembly 'YamlDotNet.dll' ...
[19:58:02] [DEBUG] Found 0 script(s) in 'YamlDotNet.dll'.
[19:58:02] [DEBUG] Starting 1 script(s) ...
[19:58:02] [DEBUG] Instantiating script 'NativeUI.BigMessageThread' in script domain 'ScriptDomain_B3A330DE' ...
[19:58:02] [DEBUG] Started script 'NativeUI.BigMessageThread'.

Dataset link does not work

Links to the Sim10k dataset on the FCAV website does not work.

Images: https://deepblue.lib.umich.edu/data/downloads/ks65hc58r
Annotations: https://deepblue.lib.umich.edu/data/downloads/ww72bb84h
Segmentation images: https://deepblue.lib.umich.edu/data/downloads/x920fx14x

Frame captured with no depth and stencil

Hi there, I managed to compile the project and get it working(well partially) in the game. Frame capture is working fine but there's no depth and stencil in the result tiff image. The depth and stencil layer in tiff are just zeros and 255, while color frames are perfectly fine.
GTAVisionNative.asi plugin is working normally with proper size raw file captured with 'L' key.
I put the native asi file in GTA root folder and other compile output from 'managed' project in 'scripts' folder. I suspect the problem is caused by some linking problem with 'native' and 'managed', but the build process didn't generate any error.
Does GTAVisionUtils.dll include GTAVisionNative.asi already? or these files should be put in some other order?

Everything seems to be well in log files, this is content in ScriptHookVDotNet2.log:

[17:23:24] [DEBUG] Created script domain 'ScriptDomain_3BFA16BC' with v2.10.3.
[17:23:24] [DEBUG] Loading scripts from 'C:\Program Files (x86)\Steam\steamapps\common\Grand Theft Auto V\scripts' into script domain 'ScriptDomain_3BFA16BC' ...
[17:23:24] [INFO] Loading assembly 'AWSSDK.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'AWSSDK.dll'.
[17:23:25] [INFO] Loading assembly 'BitMiracle.LibTiff.NET.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'BitMiracle.LibTiff.NET.dll'.
[17:23:25] [INFO] Loading assembly 'gdal_csharp.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'gdal_csharp.dll'.
[17:23:25] [INFO] Loading assembly 'GTAVisionExport.dll' ...
[17:23:25] [DEBUG] Found 1 script(s) in 'GTAVisionExport.dll'.
[17:23:25] [INFO] Loading assembly 'GTAVisionUtils.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'GTAVisionUtils.dll'.
[17:23:25] [INFO] Loading assembly 'INIFileParser.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'INIFileParser.dll'.
[17:23:25] [INFO] Loading assembly 'MathNet.Numerics.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'MathNet.Numerics.dll'.
[17:23:25] [INFO] Loading assembly 'Microsoft.Extensions.DependencyInjection.Abstractions.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'Microsoft.Extensions.DependencyInjection.Abstractions.dll'.
[17:23:25] [INFO] Loading assembly 'Microsoft.Extensions.Logging.Abstractions.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'Microsoft.Extensions.Logging.Abstractions.dll'.
[17:23:25] [INFO] Loading assembly 'Microsoft.Extensions.Logging.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'Microsoft.Extensions.Logging.dll'.
[17:23:25] [INFO] Loading assembly 'NativeUI.dll' ...
[17:23:25] [DEBUG] Found 1 script(s) in 'NativeUI.dll'.
[17:23:25] [INFO] Loading assembly 'Npgsql.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'Npgsql.dll'.
[17:23:25] [INFO] Loading assembly 'ogr_csharp.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'ogr_csharp.dll'.
[17:23:25] [INFO] Loading assembly 'osr_csharp.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'osr_csharp.dll'.
[17:23:25] [INFO] Loading assembly 'SharpDX.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'SharpDX.dll'.
[17:23:25] [INFO] Loading assembly 'SharpDX.Mathematics.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'SharpDX.Mathematics.dll'.
[17:23:25] [INFO] Loading assembly 'System.Runtime.InteropServices.RuntimeInformation.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'System.Runtime.InteropServices.RuntimeInformation.dll'.
[17:23:25] [INFO] Loading assembly 'System.Threading.Tasks.Extensions.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'System.Threading.Tasks.Extensions.dll'.
[17:23:25] [INFO] Loading assembly 'VAutodrive.dll' ...
[17:23:25] [DEBUG] Found 6 script(s) in 'VAutodrive.dll'.
[17:23:25] [INFO] Loading assembly 'VCommonFunctions.dll' ...
[17:23:25] [DEBUG] Found 2 script(s) in 'VCommonFunctions.dll'.
[17:23:25] [INFO] Loading assembly 'YamlDotNet.dll' ...
[17:23:25] [DEBUG] Found 0 script(s) in 'YamlDotNet.dll'.
[17:23:25] [DEBUG] Starting 10 script(s) ...
[17:23:25] [DEBUG] Instantiating script 'GTAVisionExport.VisionExport' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'GTAVisionExport.VisionExport'.
[17:23:26] [DEBUG] Instantiating script 'NativeUI.BigMessageThread' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'NativeUI.BigMessageThread'.
[17:23:26] [DEBUG] Instantiating script 'VAutodrive.VehicleMoversHandling' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'VAutodrive.VehicleMoversHandling'.
[17:23:26] [DEBUG] Instantiating script 'VAutodrive.MenuHandling' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'VAutodrive.MenuHandling'.
[17:23:26] [DEBUG] Instantiating script 'VAutodrive.KeyHandling' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'VAutodrive.KeyHandling'.
[17:23:26] [DEBUG] Instantiating script 'VAutodrive.Autopilot' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'VAutodrive.Autopilot'.
[17:23:26] [DEBUG] Instantiating script 'VAutodrive.SeatHandling' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'VAutodrive.SeatHandling'.
[17:23:26] [DEBUG] Instantiating script 'VAutodrive.WalkHandling' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'VAutodrive.WalkHandling'.
[17:23:26] [DEBUG] Instantiating script 'Cyron43.GtaV.Common.NotificationTask' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'Cyron43.GtaV.Common.NotificationTask'.
[17:23:26] [DEBUG] Instantiating script 'Cyron43.GtaV.Common.PerFrameFunctions' in script domain 'ScriptDomain_3BFA16BC' ...
[17:23:26] [DEBUG] Started script 'Cyron43.GtaV.Common.PerFrameFunctions'.

I would like to provide other log files if needed.
Thanks!

There is a problem loading saved images.

Thank you for providing a good library.
I obtained color, depth stencil through the procedure.
However, it is difficult to read because the files are in raw format. Is there a good way to read it? Or am I able to edit and save as jpg, png?
Also, when I read the file through Photoshop, I get the following strange results. Please help me.

Empty buffers, "detected target change, someone else is screwing with our functions"

Hi,

for some reason I'm not able to get the depth and stencil buffers. I'm not sure if this is connected, but the log often shows the message "detected target change, someone else is screwing with our functions".
The returned buffer size by the exported functions is often -1 and if it isn't, the buffer is just filled with 0s (except for the color buffer, that usually works fine).

Could you explain what the error message means and what the reason of this may be?

Thanks!

Can not compile managed plugin.

Hi,
I sucessfully managed to get the native plugin work, but now I am stuggling with the managed plugin.
I downloaded all the nuget references
copied ScriptHookVDotNet dll files to [my GTA V directory]
copied NativeUI and VAutodrive: dll files to [my GTA V directory]/scripts
and then I referenced them in References in Visual Studio.
And there I have few issues.

1. There was the ScriptHookVDotNet dependency, but in the latest release of ScriptHookVDotNet, there is only ScriptHookVDotNet2.dll, no ScriptHookVDotNet.dll
2. In the WMIInformation.cs, many classes are unknown (ManagementBaseObject, ObjectQuery...). But I resolved all non-nuget references for the GTAVisionUtils and it still does not compile.
   I am newbie in C#, so maybe I am missing something.
   Is there any dependency I need to add instead of these that were already there?

Thanks.

ground-truth of Road Line is accesible ?

thanks for your work!
i have a question: ground-truth of Road Line is accesible ?

Stencil and depth data

Hey, I tried reading the RGB images from the .tiffs generated by the managed code. They look fine but the stencil looks weird. I'm reading it using the code provided in the generate_bboxes.py and get the following result if I display the image. (I looked at ImageUtils.WriteToTiff() and it just writes byte[], so should be straight forward to read. I logged the sizes of these byte arrays and got the following

Color[0].length: 8294400, depth.length: 3686400, stencil.length: 2073600, screen: 1920x1080

The stencil is an exact 1080 by 1920, but when I read it in python using the code below I get the following image. It seems to be shifted/read in sideways? somehow but the color images read perfectly fine.

What did I mess up?
Can you help me?
Code used:

    imgpath = data_dir + filename + ".tiff"
    tiffimg = TIFF.open(imgpath)
    img = Image.open(imgpath)
    w = img.width
    h = img.height
    del img
    image = np.empty((h, w, 4), dtype=np.uint8)
    depth = np.empty((h, w), dtype=np.float32)
    stencil = np.empty((h, w), dtype=np.uint8)
    TIFF.setdirectory(tiffimg, 0)
    TIFF.readencodedstrip(tiffimg, 0, image.ctypes.data, -1)
    lastdir = num_directories(tiffimg) - 1
    TIFF.setdirectory(tiffimg, lastdir - 1)
    TIFF.readencodedstrip(tiffimg, 0, depth.ctypes.data, -1)
    TIFF.setdirectory(tiffimg, lastdir)
    TIFF.readencodedstrip(tiffimg, 0, stencil.ctypes.data, -1)
    cv2.imshow("mask", stencil)
    cv2.waitKey(0)

edit:
Also my depth data is messed up, I get the following warnings when trying to load them in python:

TIFFReadDirectory: Warning, Sum of Photometric type-related color channels and ExtraSamples doesn't match SamplesPerPixel. Defining non-color channels as ExtraSamples..
TIFFReadDirectory: Warning, Sum of Photometric type-related color channels and ExtraSamples doesn't match SamplesPerPixel. Defining non-color channels as ExtraSamples..
LZWDecode: Not enough data at scanline 0 (short 4608000 bytes).
depth.min(): nan depth.max(): nan

I split the color from the depth and stencil image to see if that made a difference but no luck so far...

        var colorTiffs = Tiff.Open(Path.Combine(dataPath, "gtav" + i.ToString() + ".tiff"), "w");
        var depthTiff = Tiff.Open(Path.Combine(dataPath, "gtav" + i.ToString() + "-depth.tiff"), "w");
        var stencilTiff = Tiff.Open(Path.Combine(dataPath, "gtav" + i.ToString() + "-stencil.tiff"), "w");
        var pages = colors.Count;
        var page = 0;
        foreach (var color in colors)
        {
            colorTiffs.CreateDirectory();
            colorTiffs.SetField(TiffTag.IMAGEWIDTH, width);
            colorTiffs.SetField(TiffTag.IMAGELENGTH, height);
            colorTiffs.SetField(TiffTag.PLANARCONFIG, PlanarConfig.CONTIG);
            colorTiffs.SetField(TiffTag.SAMPLESPERPIXEL, 4);
            colorTiffs.SetField(TiffTag.ROWSPERSTRIP, height);
            colorTiffs.SetField(TiffTag.BITSPERSAMPLE, 8);
            colorTiffs.SetField(TiffTag.SUBFILETYPE, FileType.PAGE);
            colorTiffs.SetField(TiffTag.PHOTOMETRIC, Photometric.RGB);
            colorTiffs.SetField(TiffTag.COMPRESSION, Compression.JPEG);
            colorTiffs.SetField(TiffTag.JPEGQUALITY, 95);
            colorTiffs.SetField(TiffTag.PREDICTOR, Predictor.HORIZONTAL);
            colorTiffs.SetField(TiffTag.SAMPLEFORMAT, SampleFormat.UINT);
            colorTiffs.SetField(TiffTag.PAGENUMBER, page, pages);
            colorTiffs.WriteEncodedStrip(0, color, color.Length);
            colorTiffs.WriteDirectory();
            page++;
        }
        colorTiffs.Flush();
        colorTiffs.Close();
        depthTiff.CreateDirectory();
        depthTiff.SetField(TiffTag.IMAGEWIDTH, width);
        depthTiff.SetField(TiffTag.IMAGELENGTH, height);
        depthTiff.SetField(TiffTag.ROWSPERSTRIP, height);
        depthTiff.SetField(TiffTag.PLANARCONFIG, PlanarConfig.CONTIG);
        depthTiff.SetField(TiffTag.SAMPLESPERPIXEL, 1);
        depthTiff.SetField(TiffTag.BITSPERSAMPLE, 32);
        depthTiff.SetField(TiffTag.SUBFILETYPE, FileType.PAGE);
        depthTiff.SetField(TiffTag.PHOTOMETRIC, Photometric.MINISBLACK);
        depthTiff.SetField(TiffTag.COMPRESSION, Compression.LZW);
        depthTiff.SetField(TiffTag.PREDICTOR, Predictor.FLOATINGPOINT);
        depthTiff.SetField(TiffTag.SAMPLEFORMAT, SampleFormat.IEEEFP);
        depthTiff.SetField(TiffTag.PAGENUMBER, 0, 1);
        depthTiff.WriteEncodedStrip(0, depth, depth.Length);
        depthTiff.WriteDirectory();
        depthTiff.Flush();
        depthTiff.Close();
        stencilTiff.CreateDirectory();
        stencilTiff.SetField(TiffTag.IMAGEWIDTH, width);
        stencilTiff.SetField(TiffTag.IMAGELENGTH, height);
        stencilTiff.SetField(TiffTag.ROWSPERSTRIP, height);
        stencilTiff.SetField(TiffTag.PLANARCONFIG, PlanarConfig.CONTIG);
        stencilTiff.SetField(TiffTag.SAMPLESPERPIXEL, 1);
        stencilTiff.SetField(TiffTag.BITSPERSAMPLE, 8);
        stencilTiff.SetField(TiffTag.SUBFILETYPE, FileType.PAGE);
        stencilTiff.SetField(TiffTag.PHOTOMETRIC, Photometric.MINISBLACK);
        stencilTiff.SetField(TiffTag.COMPRESSION, Compression.LZW);
        stencilTiff.SetField(TiffTag.PREDICTOR, Predictor.HORIZONTAL);
        stencilTiff.SetField(TiffTag.SAMPLEFORMAT, SampleFormat.UINT);
        stencilTiff.SetField(TiffTag.PAGENUMBER, 0, 1);
        stencilTiff.WriteEncodedStrip(0, stencil, stencil.Length);
        stencilTiff.WriteDirectory();
        stencilTiff.Flush();
        stencilTiff.Close();

SOLVED:
Enabled MSAA + VSYNC, enabled mod and disable MSAA and vsync again. Now everything looks fine! Thanks to Cuky88

Native part generates color.raw and depth.raw with only pixels and stencil.raw totaly black

color.raw is like:

depth.raw is like:

stencil.raw is pure black:

After looked up all 50 issues in discussion section, I didn't find a similar problem.
Did anyone face this issue? Could you give me a hint?

logfiles are:
asiloader.log
ScriptHookV.log
GTANativePlugin.log
Nothing seems wrong to me.

I didn't downgrade GTA or Launcher since I found it's not so easy to prevent auto-update.
I tried some suggestion on shut down some graph settings but it didn't work out.
Could anyone provide a solid graph setting for windows10 GeForce RTX 2070?
Thanks!

Wrong projection of points behind camera

I'm using GTAVision without any changes and your post-processing from https://github.com/umautobots/gta-postprocessing. There, I only changed the numbers 1920 to 1280 and 1080 to 720.
I collected some data for tests in postgreSQL and I have a problem with the coordinate transformation. You can find sample images attached in zip.
1.bmp: Box around the car is good, but the ones on the left shouldn't be there, since there is nothing within 150 meters.
2.bmp: Box around car is still good, but the other boxes are also there
3.bmp: Now, this problem exists in many coordinate transformation methods within GTA. With yours, I tried 3 different methods and always got this problem. As soon as the camera passes a point (in this case the back corners of the car) the corners will be projected wrong and appear in front of the car. As soon as I pass the last visible corner of the car (car is completely out of image), the projection will stop.
4.bmp: Shows an extreme case, since there are much more cars around.

Did you also encounter this problem? I'm not sure, but I think, that clipping is needed, so that points behind the car and within the frustum won't be projected. I tried this with scripted cams and also with gameplay camera with same error.

images.zip

Can you also tell a bit about the npz files and the results.pkl file, which are created by the post-processing? How can I view them, are these images with the pixel colors and the 2D bounding boxes?

Thanks.

When I press the N key,The following error may occur in the WriteToTiff method of ImageUtils. Try and fail several times. Is there a better solution

on native: d3d11_4.h is missing

Hi,

I ran cmake without any problems. I have also made the settings according to your Readme file in VS17. But when I want to compile it, I receive the error, that d3d11_4.h is missing:
Error C1083 Datei (Include) kann nicht geöffnet werden: "d3d11_4.h": No such file or directory GTAVisionNative D:\Devel\GTAVisionExport\native\src\main.cpp 4

I looked into to the SDK folder on Windows 10, there is only d3d11.h. Microsoft says, that on Windows 10 only d3d11_3.h exists and not d3d11_4.h, but I cannot even find d3d11_3.h. Can you please help me out?

Thanks.

The version problem of ScriptHookVDotNet2 in this repo

I used SHVDN binaries built from the source codes(release version: 2.10.4). After I got all the build and copy works done, I run the GTAV.exe. Then the ScriptHookVDotNet2.log showed:
A script references v2.10.1 which may not be compatible with the current v0.0.0 and was therefore ignored;
[18:07:25] [ERROR] Failed to instantiate script 'GTAVisionExport.VisionExport' because constructor threw an exception: System.IO.FileNotFoundException: Failed to load a file or assembly “ScriptHookVDotNet2, Version=2.10.1.0, Culture=neutral, PublicKeyToken=null” or one of its dependencies. The system can not find the specified file. File name:“ScriptHookVDotNet2, Version=2.10.1.0, Culture=neutral, PublicKeyToken=null” in GTAVisionExport.VisionExport..ctor();

0. Background:
   If you choose to build SHVDN from the source codes.
1. Cause:
   The SHVDN binaries built from the source codes don't contain any version info. Thus a default version: 0.0.0 is assigned to the binaries. Then it caused the failure.
2. Solution:
   a. remove the referance(ScriptHookVDotNet2) of the repo and add again manually;
   b. add version definition in file "AssemblyInfo.cpp" of project "ScriptHookVDotNet". Such as: #define SHVDN_VERSION "2.10.4", or higher.

Managed code script not loading

@barcharcraz Any clues?

[17:46:34] [DEBUG] Created script domain 'ScriptDomain_B5AB06DC' with v2.10.5.
[17:46:34] [DEBUG] Loading scripts from 'C:\Program Files\Rockstar Games\Grand Theft Auto V\scripts' into script domain 'ScriptDomain_B5AB06DC' ...
[17:46:34] [INFO] Loading assembly 'AWSSDK.dll' ...
[17:46:34] [DEBUG] Found 0 script(s) in 'AWSSDK.dll'.
.
.
.
[17:46:34] [INFO] Loading assembly 'NativeUI.dll' ...
[17:46:34] [DEBUG] Found 1 script(s) in 'NativeUI.dll'.
.
.
.
[17:46:34] [INFO] Loading assembly 'VAutodrive.dll' ...
[17:46:34] [DEBUG] Found 6 script(s) in 'VAutodrive.dll'.
[17:46:34] [INFO] Loading assembly 'VCommonFunctions.dll' ...
[17:46:34] [DEBUG] Found 2 script(s) in 'VCommonFunctions.dll'.
[17:46:34] [INFO] Loading assembly 'YamlDotNet.dll' ...
[17:46:34] [DEBUG] Found 0 script(s) in 'YamlDotNet.dll'.
[17:46:34] [DEBUG] Starting 10 script(s) ...
[17:46:34] [DEBUG] Instantiating script 'GTAVisionExport.VisionExport' in script domain 'ScriptDomain_B5AB06DC' ...
[17:46:35] [ERROR] Failed to instantiate script 'GTAVisionExport.VisionExport' because constructor threw an exception:
System.ArgumentNullException: Value cannot be null.
Parameter name: path
   at System.IO.File.WriteAllText(String path, String contents)
   at GTAVisionExport.VisionExport..ctor() in D:\Workspace\vs\GTAVisionExport\managed\GTAVisionExport\VisionExport.cs:line 82
[17:46:35] [DEBUG] Instantiating script 'NativeUI.BigMessageThread' in script domain 'ScriptDomain_B5AB06DC' ...
[17:46:35] [DEBUG] Started script 'NativeUI.BigMessageThread'.
.
.
.
[17:46:35] [DEBUG] Instantiating script 'Cyron43.GtaV.Common.PerFrameFunctions' in script domain 'ScriptDomain_B5AB06DC' ...
[17:46:35] [DEBUG] Started script 'Cyron43.GtaV.Common.PerFrameFunctions'.

GTAVisionExport.cs contains the constructor code:

            // loading ini file
            var parser = new FileIniDataParser();
            var location = AppDomain.CurrentDomain.BaseDirectory;
            var data = parser.ReadFile(Path.Combine(location, "GTAVision.ini"));

            //UINotify(ConfigurationManager.AppSettings["database_connection"]);
            dataPath = data["Snapshots"]["OutputDir"];
            logFilePath = data["Snapshots"]["LogFile"];

            System.IO.File.WriteAllText(logFilePath, "VisionExport constructor called.\n");
            if (!Directory.Exists(dataPath)) Directory.CreateDirectory(dataPath);

I'm no c# programmer so I'm assuming this is trying to get logFilePath and dataPath from GTAVision.ini but because that info is not in GTAVision.ini null values are passed to WriteAllText etc. I don't know what this info should look look but tried modifying GTAVision.ini to look like this:

[Database]
ConnectionString=Server=127.0.0.1;Port=5432;Database=postgres;User Id=postgres;Password=postgres;
[Snapshots]
LogFile=D:\Snapshots
OutputDir=D:\Snapshots\Output

ScriptHookVDotNet2.log now reports:

[18:39:11] [ERROR] Failed to instantiate script 'GTAVisionExport.VisionExport' because constructor threw an exception:
System.UnauthorizedAccessException: Access to the path 'D:\Snapshots' is denied.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.StreamWriter.CreateFile(String path, Boolean append, Boolean checkHost)
   at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost)
   at System.IO.File.InternalWriteAllText(String path, String contents, Encoding encoding, Boolean checkHost)
   at GTAVisionExport.VisionExport..ctor() in D:\Workspace\vs\GTAVisionExport\managed\GTAVisionExport\VisionExport.cs:line 83

Please help.

Offset between color and depth image

Hi,
I run the managed mod to save all files directly to the drive, since I couldn't manage to run the mod with database. Taking a closer look to the images, I saw that here is a huge offset between the color and depth image. Also the very first image in the tiff file is not aligned with the other images. Did someone else notice this? What could be causing this?

Cheers,
Selcuk

Managed plugin could not be loaded

I sucesfully got native plugin working and compiled all 3 projects in the managed plugin solution, and copied the compiled files into my GTA V dir/scripts.
But when running the game and pressing L, the native plugin takes the screenshot and creates *.raw files, but managed plugin can't creates a *.tiff file. In addition, I can't turn the plugin on by pressing "Page Up"
This is output of ScriptHookVDotNet2.log like this :
[19:23:23] [DEBUG] Created script domain 'ScriptDomain_50846D2E' with v2.10.9.
[19:23:23] [DEBUG] Loading scripts from'D ':\GTA5\Grand Theft Auto V\scripts' into script domain 'ScriptDomain_50846D2E'...
[19:23:23] [ERROR] Failed to compile 'GdalConfiguration. Vb 'with 14 ERROR (s):
At the line 35: The namespace or type specified in Imports "OSGeo.GDAL. GDAL" does not contain any common members, or cannot be found. Make sure that the namespace or type is defined and that it contains at least one common member; Also make sure that the imported element names are not aliased.
At the line of 36: The namespace or type specified in the Imports "OSGeo.OGR. OGR" does not contain any common members, or cannot be found. Make sure that the namespace or type is defined and that it contains at least one common member; Also make sure that the imported element names are not aliased.
At line 90: "Gdal" is not declared. It may be inaccessible due to its level of protection.
At line 116: "Ogr" is not declared. It may be inaccessible due to its level of protection.
[19:23:24] [DEBUG] Successfully generated 'gameutils. cs'.
[19:23:24] [DEBUG] Found 0 script(s) in 'gameutils. cs'.
[19:23:24] [DEBUG] Successfully produced 'stereocamera. cs'.
[19:23:24] [DEBUG] Found 0 script(s) in 'stereocamera. cs'.
[19:23:24] [ERROR] Failed to compile 'VisionExport. Cs' with 23 ERROR (s):
At line 13: Could not find the type or namespace name "YamlDotNet" (is a using instruction or assembly reference missing?).
At the line 15: Failed to find the type or namespace name "BitMiracle" (is there a lack of using instructions or assembly references?).
At the line 17: The using directive for "system.drawing.Imaging" has been shown before in this namespace
At the line 18: Could not find the type or namespace name "Amazon" (is a using instruction or assembly reference missing?).

GTA V does not start (launch) with GTAVisionNative.asi

I have compiled everything successfully. Unfortunately, when I put GTAVisionNative.asi in my main GTA V installation folder, the game does not launch at all without giving any specific error or warning [I have not forgot to put GTAVisionNative.lib in installation folder but that file is not causing any problem with launching the game].

I've also downgraded GTA V, its launcher, social club, and ScripthookV to recommended version so that couldn't be the problem.

Is there any workaround for this? Thank you very much!

color and depth raw data have incorrect dimensions

After getting the raw data for multiple frames, I checked the dimensions of the depth, color and stencil, and this is what happened to me (my screen resolution is 1920 * 1200):

1. Stencil always have the correct dimension, which is 2304000 = 1920 * 1200
2. For color buffer, the dimension is not correct, a little larger than 1920 * 1200 * 4 = 9216000, e.g. 9216007. (But I do get 1 out of 8 that has the correct dimension)
3. For depth buffer, the dimensions are much larger than 1920 * 1200 * 4 = 9216000, e.g. 9240629.

I'm wondering whether anyone has any idea on why this could happen. Is it because of some hardware issue?

Offset and length were out of bounds: ImageUtils.cs:line 41

This occured approximately 4 hours into a capture run.

System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection.
   at System.Buffer.BlockCopy(Array src, Int32 srcOffset, Array dst, Int32 dstOffset, Int32 count)
   at BitMiracle.LibTiff.Classic.Internal.CodecWithPredictor.fpDiff(Byte[] buffer, Int32 offset, Int32 count)
   at BitMiracle.LibTiff.Classic.Internal.CodecWithPredictor.PredictorEncodeTile(Byte[] buffer, Int32 offset, Int32 count, Int16 plane)
   at BitMiracle.LibTiff.Classic.Tiff.WriteEncodedStrip(Int32 strip, Byte[] buffer, Int32 offset, Int32 count)
   at BitMiracle.LibTiff.Classic.Tiff.WriteEncodedStrip(Int32 strip, Byte[] buffer, Int32 count)
   at GTAVisionUtils.ImageUtils.WriteToTiff(Tiff t, Int32 width, Int32 height, List`1 colors, Byte[] depth, Byte[] stencil) in D:\Git\GTAVisionExport\managed\GTAVisionUtils\ImageUtils.cs:line 93
   at GTAVisionUtils.ImageUtils.UploadToArchive(ZipArchive archive, String name, Int32 w, Int32 h, List`1 colors, Byte[] depth, Byte[] stencil) in D:\Git\GTAVisionExport\managed\GTAVisionUtils\ImageUtils.cs:line 41
   at GTAVisionUtils.ImageUtils.<>c__DisplayClass4_0.<StartUploadTask>b__0() in D:\Git\GTAVisionExport\managed\GTAVisionUtils\ImageUtils.cs:line 33
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at GTAVisionUtils.ImageUtils.WaitForProcessing() in D:\Git\GTAVisionExport\managed\GTAVisionUtils\ImageUtils.cs:line 26
   at GTAVisionExport.VisionExport.OnTick(Object o, EventArgs e) in D:\Git\GTAVisionExport\managed\GTAVisionExport\VisionExport.cs:line 313
   at GTA.Script.raise_Tick(Object value0, EventArgs value1)
   at GTA.Script.MainLoop()
---> (Inner Exception #0) System.ArgumentException: Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection.
   at System.Buffer.BlockCopy(Array src, Int32 srcOffset, Array dst, Int32 dstOffset, Int32 count)
   at BitMiracle.LibTiff.Classic.Internal.CodecWithPredictor.fpDiff(Byte[] buffer, Int32 offset, Int32 count)
   at BitMiracle.LibTiff.Classic.Internal.CodecWithPredictor.PredictorEncodeTile(Byte[] buffer, Int32 offset, Int32 count, Int16 plane)
   at BitMiracle.LibTiff.Classic.Tiff.WriteEncodedStrip(Int32 strip, Byte[] buffer, Int32 offset, Int32 count)
   at BitMiracle.LibTiff.Classic.Tiff.WriteEncodedStrip(Int32 strip, Byte[] buffer, Int32 count)
   at GTAVisionUtils.ImageUtils.WriteToTiff(Tiff t, Int32 width, Int32 height, List`1 colors, Byte[] depth, Byte[] stencil) in D:\Git\GTAVisionExport\managed\GTAVisionUtils\ImageUtils.cs:line 93
   at GTAVisionUtils.ImageUtils.UploadToArchive(ZipArchive archive, String name, Int32 w, Int32 h, List`1 colors, Byte[] depth, Byte[] stencil) in D:\Git\GTAVisionExport\managed\GTAVisionUtils\ImageUtils.cs:line 41
   at GTAVisionUtils.ImageUtils.<>c__DisplayClass4_0.<StartUploadTask>b__0() in D:\Git\GTAVisionExport\managed\GTAVisionUtils\ImageUtils.cs:line 33
   at System.Threading.Tasks.Task.Execute()<---

color and depth not synchronized

Hi,

after successfully building and binding GTAVisionExport with my steam version GTA 5, I gathered some color and depth data. I checked those data, and find that sometimes depth and color data are not synchronized well... Here is a sample.

It happens when I shake view(camera) or other characters (or car) moves.
Is there any way to solve this problem? To get synchronized data?

Thanks!

what is inside constant buffer?

HI, what can I do with the constant buffer rage_matrices (exported by export_get_constant_buffer(rage_matrices* buf))

Socket connection for autodriving

In the VisionExport.cs, there is socket connection to some server here and apparently being used to instruct this mod here
but it is nowhere mentioned in the documentation, neither I found source code for this.
Do you plan to opensource it?
Did anyone manage to get this working with their own implementation?

HRESULT 0x80004002

In the presentCallback function,

ComPtr<ID3D11DeviceContext> ctx;
ComPtr<ID3D11Multithread> multithread;
hr = ctx.As(&multithread);
if (hr != S_OK) throw std::system_error(hr, std::system_category());
multithread->SetMultithreadProtected(true);

the returned HRESULT is 0x80004002, which means E_NOINTERFACE error. Any way to solve this problem?

https://github.com/umautobots/GTAVisionExport/blob/master/native/src/main.cpp#L276

How to export specular map

Should I follow the same pipeline of exporting depth buffer and color buffer?
Any tips will be appreciated

Depth distorted

Hi all, my depth.raw is heavily distorted, just wondering if anybody else is seeing issues like this? Did go through most of the other threads before posting this. Thanks!

3D Bounding boxes

could someone please explain how the 3D bounding boxes are determined here? Thanks

GTAV version problem

Hi,

Thank you for sharing your work. It is very useful for my research.

I can't get the old version of GTAV instead I have the latest version from steam. I test this script and every times I press 'L' the game quit immediately. I then debugging the script found the problem is on the ucrtbase.dll. This file is not missing because I can find it in the system32 folder by installed the Windows Software Development Kit but may be the version is wrong. I wonder whether if you have some advice.

thank you very much.

Missing Documentation

Hi, first of all thanks for sharing your work, I really appreciate it!

Would it be possible to add some documentation on how to compile the project and what the dependencies are?

Can you release a compailed asi file?

thx

GTA internal camera intrinsic matrix

Hi, how are you? Great work. So i am trying to do a fundamental check on the poses provided by gta and i was wondering if you you could guide me as to how i can get the camera intrinsic matrix. From what i understand about your work, you map an image to 3d world for mapping, correct me if I'm wrong please. What I'm trying to do is kinda opposite. i want to map the 3d pose of game-play camera of gta to an image i got from gta so that i can confirm if the pose provided is correct. If you have extracted the camera intrinsic matrix then could you share it with me or guide me as to how I can extract it correctly myself. Thank you very much in advance.

Linker error prevents creation of GTAVisionNative.asi

I created the SLN using Cmake3.9 for VS 2015 64 bit because I could not get Cmake to recognize VS 2017. I used VS 2017 to open the solution and modified the include and linker locations as described. Compiling I get this error:

1>------ Rebuild All started: Project: ZERO_CHECK, Configuration: Release x64 ------
1>Checking Build System
1>CMake does not need to re-run because D:/programs/GTAVisionExport-master/native/build/CMakeFiles/generate.stamp is up-to-date.
1>CMake does not need to re-run because D:/programs/GTAVisionExport-master/native/build/src/CMakeFiles/generate.stamp is up-to-date.
2>------ Rebuild All started: Project: GTAVisionNative, Configuration: Release x64 ------
2>Building Custom Rule D:/programs/GTAVisionExport-master/native/src/CMakeLists.txt
2>CMake does not need to re-run because D:\programs\GTAVisionExport-master\native\build\src\CMakeFiles\generate.stamp is up-to-date.
2>main.cpp
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(71): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\stdio.h(207): note: see declaration of 'fopen'
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(101): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\stdio.h(207): note: see declaration of 'fopen'
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(106): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\stdio.h(207): note: see declaration of 'fopen'
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(110): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\stdio.h(207): note: see declaration of 'fopen'
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(175): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\stdio.h(207): note: see declaration of 'fopen'
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(181): warning C4101: 'hr': unreferenced local variable
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(251): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\stdio.h(207): note: see declaration of 'fopen'
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(127): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\ucrt\stdio.h(207): note: see declaration of 'fopen'
2>D:\programs\GTAVisionExport-master\native\src\main.cpp(279): note: see reference to function template instantiation 'void hook_function<12,T>(T *,void *,bool)' being compiled
2> with
2> [
2> T=ID3D11DeviceContext
2> ]
2>export.cpp
2>D:\programs\GTAVisionExport-master\native\src\export.cpp(60): warning C4018: '<': signed/unsigned mismatch
2>D:\programs\GTAVisionExport-master\native\src\export.cpp(62): warning C4018: '<': signed/unsigned mismatch
2>D:\programs\GTAVisionExport-master\native\src\export.cpp(152): warning C4018: '<': signed/unsigned mismatch
2>D:\programs\GTAVisionExport-master\native\src\export.cpp(154): warning C4018: '<': signed/unsigned mismatch
2>D:\programs\GTAVisionExport-master\native\src\export.cpp(220): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data
2>D:\programs\GTAVisionExport-master\native\src\export.cpp(227): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data
2>D:\programs\GTAVisionExport-master\native\src\export.cpp(234): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data
2>Generating Code...
2> Creating library D:/programs/GTAVisionExport-master/native/build/src/Release/GTAVisionNative.lib and object D:/programs/GTAVisionExport-master/native/build/src/Release/GTAVisionNative.exp
2>main.obj : error LNK2019: unresolved external symbol MH_Initialize referenced in function DllMain
2>main.obj : error LNK2019: unresolved external symbol MH_Uninitialize referenced in function DllMain
2>main.obj : error LNK2019: unresolved external symbol MH_CreateHook referenced in function "void __cdecl hook_function<53,struct ID3D11DeviceContext>(struct ID3D11DeviceContext *,void *,bool)" (??$hook_function@$0DF@UID3D11DeviceContext@@@@YAXPEAUID3D11DeviceContext@@PEAX_N@Z)
2>main.obj : error LNK2019: unresolved external symbol MH_RemoveHook referenced in function "void __cdecl hook_function<53,struct ID3D11DeviceContext>(struct ID3D11DeviceContext *,void *,bool)" (??$hook_function@$0DF@UID3D11DeviceContext@@@@YAXPEAUID3D11DeviceContext@@PEAX_N@Z)
2>main.obj : error LNK2019: unresolved external symbol MH_EnableHook referenced in function "void __cdecl hook_function<53,struct ID3D11DeviceContext>(struct ID3D11DeviceContext *,void *,bool)" (??$hook_function@$0DF@UID3D11DeviceContext@@@@YAXPEAUID3D11DeviceContext@@PEAX_N@Z)
2>main.obj : error LNK2019: unresolved external symbol MH_DisableHook referenced in function "void __cdecl hook_function<53,struct ID3D11DeviceContext>(struct ID3D11DeviceContext *,void *,bool)" (??$hook_function@$0DF@UID3D11DeviceContext@@@@YAXPEAUID3D11DeviceContext@@PEAX_N@Z)
2>D:\programs\GTAVisionExport-master\native\build\src\Release\GTAVisionNative.asi : fatal error LNK1120: 6 unresolved externals
2>Done building project "GTAVisionNative.vcxproj" -- FAILED.
========== Rebuild All: 1 succeeded, 1 failed, 0 skipped ==========

Postgresql dump containing not valid types.

In managed readme there is database schema, which is useful, but has some downsides, few types are missing:
namely detection_type and weather. I gues these are some enums. It would be nice if they were included.

Also I am not sure where exactly should be the ini file stored. In my GTAV directory or in my GTAV directory/scripts?
I see name of the ini, bit not the location where it is expected.

camera vibrated with the vehicle

When I set the camera with "First Person Vehicle Hood" to extract tiff images, the camera will be vibrated with the vehicle terribly, and sometimes the camera points to the sky in 1st frame and then points to the ground in next frame.
How to solve this problems?
Does anyone has an idea?
just rename the attachment *.mp4 to watch the video of this problem
camera_vibration.mp4.zip

Can not configure native project

Hi, I tried to install this vision export. but I could not configure a generate the project.
I sucesfully followed steps 1 - 7.
But when I hit "configure" button, there is no EIGEN3_INCLUDE_DIR entry.

Although I added this entry, it did not configure properly, so I yould not generate the project.

Versions:
Cmake: 3.10.0-rc1
Eigen3: f562a193118d

Garbled depth images

Hiya, I'm getting garbled depth images out from the managed tool (example below). I have MSAA turned off, and for reflections too. Window resolution 1280x720, all of the graphics settings turned down to 'normal'. Any clues as to how I might resolve this? PS stencil looks like its garbled too.

Used the following code to display the depth in Python:

from PIL import Image, ImageFile, TiffImagePlugin
import numpy as np
import cv2
ImageFile.LOAD_TRUNCATED_IMAGES = True
TiffImagePlugin.READ_LIBTIFF = True

im = Image.open('D:/Snapshots/Output/45871.tiff') # with your own image path.
im.seek(1) # use seek() to switch between pages
image = np.array(im.getdata()).reshape(im.size[1], im.size[0], 1)
maximg = np.max(image)
imageint = (image/maximg)*255

print(im.size[0])
print(im.size[1])
cv2.imshow('img',imageint)
cv2.waitKey(0)