Using own data? about deep-fluids

[byungsook]

data.py, which includes a data loader, assumes a dataset with the following hierarchy:

data/dataset
data/dataset/v
data/dataset/args.txt // parameter list for dataset generation
data/dataset/v_range.txt // the value range of whole dataset

data/dataset/args.txt should contain the following info mostly for loading and normalization of each parameter (for instance):

path_format: %d_%d_%d.npz
num_param: 3
p0: src_x_pos
p1: src_radius
p2: frames
num_src_x_pos: 21
min_src_x_pos: 0.2
max_src_x_pos: 0.8
num_src_radius: 5
min_src_radius: 0.04
max_src_radius: 0.12
num_frames: 200
min_frames: 0
max_frames: 199

An example of data/dataset/v_range.txt is like this:

-10.862
10.808

Each velocity profile should follow the npz path format written in args.txt.
As you mentioned, x is a velocity fields of a shape [H,W,2] ([D,H,W,3] in 3D), and y is a corresponding parameter for x as described in args.txt.

You can test whether the data loader can work for your own dataset by running python data.py.
Note that you need to set proper attributes like in line 415-417.
If it works, it will create a sample image in log directory like this

It might be helpful to generate a sample dataset using mantaflow and then compare with your dataset.

Let me know if there is any missing information.

[alok]

When you say y is the corresponding parameter, I don't follow. Looking at args.txt, y is src_x_pos and src_radius. For my data, I have wind speed and temperature (3+1 = 4 channels) for each point in a 3D box. What should y be in that case (or can I not use that sort of data with this project)?

To simplify, given a tensor of shape [Time (120), Channels (4, 3 for velocity and 1 for temperature), H (64), W (64), D (128)], I'm trying to generate that tensor given only a starting slice of it (say the first 10 frames). Would deep fluids work for this?

[byungsook]

In the example, there are three parameters which is the dimension of y for 21x5 scenes with different src_x_pos and src_radius and 200 frames for each scene. In your case, I don't see other scene parameters but time, cause wind speed and temperature are not parameters but fields. (i.e., 1-to-DHW4 mapping)

In addition, the data loader assumes to handle only scalar fields or 2D/3D vector fields (see line 53 in data.py). Thus, you are required to change the code to train on 4D vector fields. Also need to turn off use_curl option in config.py.