GSD Reader with additional shape frame is not working about garnett

[lyrivera]

Just to be clear, the consensus is to remove the read_gsd_shape_data flag?

[bdice]

Original comment by Joshua Anderson (Bitbucket: joaander, GitHub: joaander).

---

On the vislab, glotz-formats is the latest tagged version: 0.4.1

[klywang]

@vyasr could you provide more information about this issue?

[vyasr]

Oh boy. Let me see if I can reproduce, it's possible that the problem has since been fixed given the many changes to shape behavior in glotzformats.

[vyasr]

I no longer remember the problem that made me raise this issue, unfortunately. I'm a little confused by how the shapedef field is currently being used for non-pos files, because I'm unable to see actual shape definition information even though the shape information is clearly present since plato is able to visualize these systems correctly.

For example, given a gsd file with no shape information and a pos file with the corresponding shape information, is this the expected behavior?

>>> from glotzformats import reader
>>> gsd_reader = reader.GSDHOOMDFileReader()
>>> pos_reader = reader.PosFileReader()
>>> with open('testfile.pos') as pf:
...         with open('testfile.gsd', 'rb') as gf:
...             shape_frame = pos_reader.read(pf)[0]
...             traj = gsd_reader.read(gf)
...             print(traj[0].shapedef)
...             print(shape_frame.shapedef)
OrderedDict()
OrderedDict([('Octahedra', poly3d 6 0.0 0.0 0.90856027603 0.0 0.90856027603 -0.0 0.90856027603 -0.0 -0.0 -0.90856027603 0.0 -0.0 -0.0 -0.90856027603 -0.0 -0.0 0.0 -0.90856027603 005984FF)])

[bdice]

@vyasr Does this behavior surprise you? The GSD file (with no shape information) has an empty shapedef, while the POS file (with shape information) has a non-empty shapedef.

If you're using this with platoviz, you're effectively combining the trajectory data from the GSD file with the shape information from the POS file, see the frame argument here: https://github.com/glotzerlab/glotzformats/blob/master/glotzformats/gsdhoomdfilereader.py#L194

[vyasr]

@bdice you're right. My original confusion was about something different, but in the process of trying to understand the behavior I wrote that sample above, and I clearly did so hastily. Let's try this again:

from glotzformats import reader
gsd_reader = reader.GSDHOOMDFileReader()
pos_reader = reader.PosFileReader()
with open('posfile_with_shape.pos') as pf:
    with open('gsdfile_without_shape.gsd', 'rb') as gf:
        shape_frame = pos_reader.read(pf)[0]
        traj_noshape = gsd_reader.read(gf)
        traj_shape = gsd_reader.read(gf, shape_frame)
        print(traj_shape[0].shapedef)
        print(traj_noshape[0].shapedef)
        print(shape_frame.shapedef)

with open('gsdfile_with_shape.gsd', 'rb') as f:
    traj_gsd_shape = gsd_reader.read(f)
    print(traj_gsd_shape[0].shapedef)

The output from that script is:

OrderedDict([('Octahedra', poly3d 6 0.0 0.0 0.90856027603 0.0 0.90856027603 -0.0 0.90856027603 -0.0 -0.0 -0.90856027603 0.0 -0.0 -0.0 -0.90856027603 -0.0 -0.0 0.0 -0.90856027603 005984FF)])
OrderedDict()
OrderedDict([('Octahedra', poly3d 6 0.0 0.0 0.90856027603 0.0 0.90856027603 -0.0 0.90856027603 -0.0 -0.0 -0.90856027603 0.0 -0.0 -0.0 -0.90856027603 -0.0 -0.0 0.0 -0.90856027603 005984FF)])
OrderedDict()

The first three are what I would expect, I just had the wrong code in my previous example since I failed to provide the pos frame to the gsd reader. The thing that confused me is the last output. According to the gsd reader code, the shapes from the HOOMD GSD schema are parsed into a shapedef dictionary that is then saved to the raw frame in order to mirror how other shape information is stored. So I'm confused as to why the shapedef property is empty for the gsd file with shape (I've confirmed that in platoviz the shapes are correctly rendered).

[lyrivera]

@vyasr which branch were you testing on? I just merged PR #42 that may have a fix for this, although Im not 100% sure. Will perform some tests on my own and see if I can still get the same behavior.

[lyrivera]

I can confirm the issue is resolved on the latest master update. The out of the following script

from glotzformats import reader
gsd_reader = reader.GSDHOOMDFileReader()
pos_reader = reader.PosFileReader()

with open('gsdfile_with_shape.gsd', 'rb') as f:
    traj_gsd_shape = gsd_reader.read(f)
    print(traj_gsd_shape[0].shapedef)

is

OrderedDict([('A', poly3d 4 0.5 0.5 0.5 0.5 -0.5 -0.5 -0.5 0.5 -0.5 -0.5 -0.5 0.5 005984FF)])

[vyasr]

I can confirm that the gsd problem is now fixed on master. However, it looks like the ability to read a shape definition from a pos file into a gsd frame has been lost. Running the exact same script as I posted previously, I now get the following output:

OrderedDict()
OrderedDict()
OrderedDict([('Octahedra', poly3d 6 0.0 0.0 0.90856027603 0.0 0.90856027603 -0.0 0.90856027603 -0.0 -0.0 -0.90856027603 0.0 -0.0 -0.0 -0.90856027603 -0.0 -0.0 0.0 -0.90856027603 005984FF)])
OrderedDict([('Polyhedra', poly3d 12 0.20926048 0.7750388 0.20926048 0.7750388 0.20926048 0.20926048 0.20926048 0.20926048 0.7750388 -0.7750388 -0.20926048 0.20926048 -0.20926048 -0.7750388 0.20926048 -0.20926048 -0.20926048 0.7750388 -0.20926048 0.20926048 -0.7750388 -
0.20926048 0.7750388 -0.20926048 -0.7750388 0.20926048 -0.20926048 0.20926048 -0.20926048 -0.7750388 0.7750388 -0.20926048 -0.20926048 0.20926048 -0.7750388 -0.20926048 005984FF)])

The two are different trajectories, so the shapes shouldn't match, but the first output shouldn't be 0. I can share the files if you'd like to see them.

[lyrivera]

I see. So thats what the issue was initially about. I'll check on this.

[lyrivera]

@vyasr I can reproduce that behavior. The reason is that the flag read_gsd_shape_data is set toTrue by default on the GSDHOOMDFileReader constructor. When read_gsd_shape_data==True, the reader ignores the shape definition on the provided frame and tries to retrieve it from the gsd file, but in this case the gsd file contains no shape information, thus the reader returns an empty dict.

The following script illustrates this behavior:

from glotzformats import reader
gsd_reader = reader.GSDHOOMDFileReader(read_gsd_shape_data=False)
pos_reader = reader.PosFileReader()

with open('posfile_with_shape.pos') as pf:
    with open('gsdfile_without_shape.gsd', 'rb') as gf:
        shape_frame = pos_reader.read(pf)[0]
        traj_noshape = gsd_reader.read(gf)
        traj_shape = gsd_reader.read(gf, shape_frame)
        print(traj_shape[0].shapedef)
        print(traj_noshape[0].shapedef)
        print(shape_frame.shapedef)

gsd_reader = reader.GSDHOOMDFileReader(read_gsd_shape_data=True)
with open('gsdfile_with_shape.gsd', 'rb') as f:
    traj_gsd_shape = gsd_reader.read(f)
    print(traj_gsd_shape[0].shapedef)

which produces the following output

OrderedDict([('A', poly3d 4 0.5 0.5 0.5 0.5 -0.5 -0.5 -0.5 0.5 -0.5 -0.5 -0.5 0.5 005984FF)])
OrderedDict()
OrderedDict([('A', poly3d 4 0.5 0.5 0.5 0.5 -0.5 -0.5 -0.5 0.5 -0.5 -0.5 -0.5 0.5 005984FF)])
OrderedDict([('A', poly3d 4 0.5 0.5 0.5 0.5 -0.5 -0.5 -0.5 0.5 -0.5 -0.5 -0.5 0.5 005984FF)])

[bdice]

@lyrivera Good find! I think the most intuitive thing would be to use the provided frame's shapedef first, then try to read GSD shape data as a fallback. I'm not sure if we actually need the read_gsd_shape_data flag if that's the case. Adding tests for this behavior would be good too.

It doesn't look like we discussed this behavior in the GSD shape pull request. https://bitbucket.org/glotzer/glotz-formats/pull-requests/40/gsd-shape-reading-support/diff

[lyrivera]

@bdice I agree the order should be inverted, and if that's the case I can't think of a reason to keep read_gsd_shape_data when a frame is provided.

[vyasr]

I agree with what you've said in terms of the order in which those two data sources for shape should be resolved. That would address this issue and retain backwards compatibility.

[bdice]

@lyrivera Yup, that's my vote!

[lyrivera]

@bdice I agree. @csadorf Thoughts on this?

[csadorf]

I agree, the read_gsd_shape_data flag should be removed. Make sure that the documentation accurately reflects the then implemented behavior.

[lyrivera]

This issue has been addressed on #55 and #58 . Closing now.