nmwsharp / happly

A C++ header-only parser for the PLY file format. Parse .ply happily!

A header-only C++ reader/writer for the PLY file format. Parse .ply happily!

Features:

• Header only-- drop in and use!
• Read and write to plaintext and binary variants of format with same API!
• Supports general data in .ply files, along with common-case helpers for reading/writing mesh data!
• Automatic type promotion-- eg, if a file contains a float field, you can seamlessly read it as a double!
• Tested, documented, and MIT-licensed!

The .ply format and hapPLY

The .ply format is a general-purpose flat file format useful for recording numerical data on unstructured domains, which includes both plaintext and binary representations. The format has been kicking around since the 90s: Paul Bourke's webpage serves as both an introduction and the most official specification. hapPLY grew out of my own personal code for .ply files-- the format is extremely useful for working with 3D meshes and other geometric data, but no easily accessible C++ implementation was available.

Although the .ply format is commonly used to store 3D mesh and point cloud data, the format itself technically has nothing to do with meshes or point clouds; it simply specifies a collection elements, and data (called properties) associated with those elements. For instance in a mesh, the elements are vertices and faces; vertices then have properties like "position" and "color", while faces have a property which is a list of vertex indices. hapPLY exposes a general API for reading and writing elements and properties, as well as special-purpose helpers for the common conventions surrounding mesh data.

Examples

Read basic data

#include "happly.h"

// Construct a data object by reading from file
happly::PLYData plyIn("my_file.ply");

// Get data from the object
std::vector<float> elementA_prop1 = plyIn.getElement("elementA").getProperty<float>("prop1");
std::vector<double> elementA_prop2 = plyIn.getElement("elementA").getProperty<double>("prop1");
std::vector<std::vector<double>> elementB_listProp = 
    plyIn.getElement("elementB").getListProperty<double>("listprop1");

// Type promotion is automatic for numeric types: even if this property was stored as a float, 
// we can access it as a double
std::vector<double> elementA_prop1_as_double = 
    plyIn.getElement("elementA").getProperty<double>("prop1"); 

Write basic data

#include "happly.h"

// Suppose these hold your data
std::vector<float> elementA_prop1;
std::vector<int> elementA_prop2;
std::vector<std::vector<double>> elementB_listProp;

// Create an empty object
happly::PLYData plyOut;

// Add elements
plyOut.addElement("elementA", 20);
plyOut.addElement("elementB", 42);

// Add properties to those elements
plyOut.getElement("elementA").addProperty<float>("prop1", elementA_prop1);
plyOut.getElement("elementA").addProperty<int>("prop2", elementA_prop2);
plyOut.getElement("elementB").addListProperty<double>("listprop1", elementB_listProp);

// Write the object to file
plyOut.write("my_output_file.ply", happly::DataFormat::Binary);

Read mesh-like data

#include "happly.h"

// Construct the data object by reading from file
happly::PLYData plyIn("my_mesh_file.ply");

// Get mesh-style data from the object
std::vector<std::array<double, 3>> vPos = plyIn.getVertexPositions();
std::vector<std::vector<size_t>> fInd = plyIn.getFaceIndices<size_t>();

Write mesh-like data

#include "happly.h"

// Suppose these hold your data
std::vector<std::array<double, 3>> meshVertexPositions;
std::vector<std::array<double, 3>> meshVertexColors;
std::vector<std::vector<size_t>> meshFaceIndices;

// Create an empty object
happly::PLYData plyOut;

// Add mesh data (elements are created automatically)
plyOut.addVertexPositions(meshVertexPositions);
plyOut.addVertexColors(meshVertexColors);
plyOut.addFaceIndices(meshFaceIndices);


// Write the object to file
plyOut.write("my_output_mesh_file.ply", happly::DataFormat::ASCII);

API

This assumes a basic familiarity with the file format; I suggest reading Paul Bourke's webpage if you are new to .ply.

All of the outward-facing functionality of hapPLY is grouped under a single (namespaced) class called happly::PLYData, which represents a collection of elements and their properties. PLYData objects can be constructed from an existing file PLYData::PLYData("my_input.ply"), or you can fill with your own data and then write to file PLYData::write("my_output.ply", DataFormat::ASCII).

Generally speaking, hapPLY uses C++ exceptions to communicate errors-- most of these methods will throw if something is wrong. hapPLY attempts to provide basic sanity checks and informative errors, but does not guarantee robustness to malformed input.

Reading and writing objects:

• PLYData() Construct an empty PLYData containing no elements or properties.

• PLYData(std::string filename, bool verbose = false) Construct a new PLYData object from a file, automatically detecting whether the file is plaintext or binary. If verbose=true, useful information about the file will be printed to stdout.

• PLYData(std::istream& inStream, bool verbose = false) Like the previous constructor, but reads from anistream.

• PLYData::validate() Perform some basic sanity checks on the object, throwing if any fail. Called internally before writing.

• PLYData::write(std::string filename, DataFormat format = DataFormat::ASCII) Write the object to file. Specifying DataFormat::ASCII, DataFormat::Binary, or DataFormat::BinaryBigEndian controls the kind of output file.

• PLYData::write(std::ostream& outStream, DataFormat format = DataFormat::ASCII) Like the previous method, but writes to anostream.

Accessing and adding data to an object:

• void addElement(std::string name, size_t count) Add a new element type to the object, with the given name and number of elements.

• Element& getElement(std::string target) Get a reference to an element type contained in the object.

• bool hasElement(std::string target) Check if an element type is contained in the object.

• std::vector<std::string> getElementNames() List of all element names.

• std::vector<T> Element::getProperty(std::string propertyName) Get a vector of property data for an element. Will automatically promote types if possible, eg getProperty<int>("my_prop") will succeed even if the object contains "my_prop" with type short.

• std::vector<std::vector<T>> Element::getListProperty(std::string propertyName) Get a vector of list property data for an element. Supports type promotion just like getProperty().

• void Element::addProperty(std::string propertyName, std::vector<T>& data) Add a new property to an element type. data must be the same length as the number of elements of that type.

• void addListProperty(std::string propertyName, std::vector<std::vector<T>>& data) Add a new list property to an element type. data must be the same length as the number of elements of that type.

Misc object options:

• std::vector<std::string> PLYData::comments Comments included in the .ply file, one string per line. These are populated after reading and written when writing.

• std::vector<std::string> PLYData::objInfoComments Lines prefaced with obj_info included in the .ply file, which are effectively a different kind of comment, one string per line. These seem to be an ad-hoc extension to .ply, but they are pretty common, so we support them.

Common-case helpers for mesh data:

• std::vector<std::array<double, 3>> getVertexPositions(std::string vertexElementName = "vertex") Returns x,y,z vertex positions from an object. vertexElementName specifies the name of the element type holding vertices, which is conventionally "vertex".

• void addVertexPositions(std::vector<std::array<double, 3>>& vertexPositions) Adds x,y,z vertex positions to an object, under the element name "vertex".

• std::vector<std::array<unsigned char, 3>> getVertexColors(std::string vertexElementName = "vertex") Returns r,g,b vertex colors from an object. vertexElementName specifies the name of the element type holding vertices, which is conventionally "vertex".

• void addVertexColors(std::vector<std::array<unsigned char, 3>>& vertexColors) Adds r,g,b vertex colors positions to an object, under the element name "vertex".

• void addVertexColors(std::vector<std::array<double, 3>>& vertexColors) Adds r,g,b vertex colors positions to an object, under the element name "vertex". Assumes input is in [0.0,1.0], and internally converts to 0-255 char values

• std::vector<std::vector<T>> getFaceIndices() Returns indices in to a vertex list for each face. Usually 0-indexed, but there are no formal rules in the format. Supports type promotion as in getProperty(), and furthermore converts signed to unsigned and vice-versa, though the conversion is performed naively.

• void addFaceIndices(std::vector<std::vector<T>>& indices) Adds vertex indices for faces to an object, under the element name "face" with the property name "vertex_indices". Automatically converts to a 32-bit integer type with the same signedness as the input type, and throws if the data cannot be converted to that type.

Known issues:

• Writing floating-point values of inf or nan in ASCII mode is not supported, because the .ply format does not specify how they should be written (C++'s ofstream and ifstream don't even treat them consistently). These values work just fine in binary mode.
• Currently hapPLY does not allow the user to specify a type for the variable which indicates how many elements are in a list; it always uses uchar (and throws and error if the data does not fit in a uchar). Note that at least for mesh-like data, popular software only accepts uchar.
• Almost all modern computers are little-endian. If you happen to have a big-endian platform, be aware that the codebase has not been tested in a big-endian environment, and might have bugs related to binary reading/writing there. Note that the platform endianness is distinct from the file endianness---reading/writing either big- or little-endian files certainly works just fine as long as you're running the code on a little-endian computer (as you problably are).

Current TODOs:

• Add more common-case helpers for meshes (texture coordinates, etc)
• Add common-case helpers for point clouds
• Bindings for Python, Matlab?

---

By Nicholas Sharp. Credit to Keenan Crane for early feedback and the logo!

Development of this software was funded in part by NSF Award 1717320, an NSF graduate research fellowship, and gifts from Adobe Research and Autodesk, Inc.

happly's Issues

Error after the last commit.

I am getting the following error:
error: the operand of a pointer dynamic_cast must be a pointer to a complete class type
lines: 954, 997

However, when I roll back to commit 88f7972 it works fine.

Make lack of implict copy very clear. Add .copy() function.

Right now the main PlyData class cannot be implicitly copied. This is probably the right behavior; doing so is likely a mistake.

However, we can improve the situation in two ways

• Explicitly delete the copy constructor, to make it clear that the behavior is intentional
• Provide a .copy() function for the few situations where one really does want to explicitly copy

Improve exception message when trying to read list property as scalar

I have a custom element with a list property:

element track 572
property list uint16 uint32 vertex_index

When trying to read it via:

const std::vector<uint32_t> track_ids = tracks.getElement("track").getProperty<uint32_t>("vertex_index");

I am getting the exception:

PLY parser: property vertex_index cannot be coerced to requested type uint. Has type unit

which is a bit confusing, since it detects the correct type (Has type unit) but fails to convert the type to the same type (cannot be coerced to requested type uint).

Is this a bug in the conversion, or is the uint32 type just not supported?

Write that big endian is supported.

README.md says: "the code assumes the platform is little-endian." under "Known issues", but happly.h says that big endian is supported. If happly is now supporting big endian platforms, README.md should be changed, I guess.

Thanks for this nice library in just 1 file :)

Pointcload read/write

Hi,

This is not a bug.
Could you please give an example how to read and write a pointclošd with colors?

Parallelizing ply parser

Hi, I'm recently working with analysis of large PLY-formatted mesh model. Since, 3D model I/O is really big bottleneck of my analysis pipeline, I'm trying to parallelizing ply parser based on this repository using the idea of [1].
If you think that this feature is worth and suitable to this repository, I'd like to contribute my implementation here.

Thanks for reading this issue.

Reference
[1] Jo, Sunghun, Yuna Jeong, and Sungkil Lee. "GPU-driven scalable parser for OBJ models." Journal of Computer Science and Technology 33.2 (2018): 417-428.

Small bug in read basic data example

It seems this line
std::vector<int> elementA_prop2 = plyIn.getElement("elementA").getProperty<double>("prop1");
should be
std::vector<double> elementA_prop2 = plyIn.getElement("elementA").getProperty<double>("prop1");

Store type enum for error messages and string names

Very useful for writing as well.

Clean up CanonicalName things

I noticed some of the template use in Happly is not ideal. Go back and look closer:

• CanonicalName<size_t> looks like it has a bug, perhaps not caught because no one uses 32bit machines?
• Try to avoid copy when converting to canonical
• Several places where we through runtime_errors could be compile time checks (though the compile checks might have much more confusing error message...)

Can't read obj_info header written by CloudCompare

Thanks for the great parser! We were so happy to find it. ❤️😄🌈🎉

One issue we ran into: CloudCompare writes a obj_info header, e.g.

ply
format ascii 1.0
comment Created by CloudCompare v2.10.2 (Zephyrus)
comment Created 8/1/19 9:04 AM
obj_info Generated by CloudCompare!
element vertex 4308
property float x
property float y
property float z
element face 8612
property list uchar int vertex_indices
end_header
74.5034 161.291 168.961 
77.047 149.808 164.128 
73.8367 161.097 169.151 
73.1975 160.869 169.417 
...

I can ping someone to track down the exact line of code that's failing, but it seems pretty cut and dry that you simply have to remove the obj_info line to avoid getting an error about an invalid header. Seems like just a simple obscure PLY header field that's not handled.

We'll probably have to fix this, but at the very least wanted to get it reported. Thanks again!

See also: PointCloudLibrary/pcl#2763 (comment)

MeshLab fails to read face lists with number of vertices specified by uint

Perhaps not a hapPLY-specific issue, but still seemingly important in terms of common use cases:

hapPLY uses a face specification

property list uint uint vertex_indices

but MeshLab expects

property list uchar uint vertex_indices

(and chokes on the former).

Significant slowdown on large files

The current version of happly is quite slow for large files compared to an home-brewed solution I cooked up. The profiler suggest that the problem is allocating many small vectors in list properties. On the Lucy model from the Stanford repo, happly takes about 16 seconds of which 7 are just vector allocs. My home-brewed solution takes half that time.

I propose the following changes:

1. change the storage of list props from vector<vector<T>> data to three vectors for start, count and data std::vector<size_t> start; std::vector<uint8_t> count; vector<T> data;, where data has the concatenated list of elements, start has the starting index for each list and count contains the lists sizes
2. in a backward compatible manner, add getListProperty(vector<array<T, N>>& data, vector<uint8_t>& count) to read the data in preallocated lists; maintain previous versions for backward compatibility

If this sounds good, I may even take a crack at it, but only if this feels right.

Auto-convert floats to double?

We already auto-convert signed to unsigned ints in some cases, to support the common situation where indices have (strangely) been stored as a signed value.

A similar issue is downconverting doubles to floats. Personally, I tend towards using double everywhere, but I think others follow the opposite policy, and they would probably be fine with the narrowing conversion.

This is probably best implemented with a special-case check, as was done with size_t.

Documentation

// TODO document

could we add a CMakeLists.txt file to make it cowork with cmake or vcpkg?

Hi,
could we add a CMakeLists.txt file so that this library can be included using cmake or even vcpkg?
I've written an example of CMakeLists.txt, but I am no export of cmake, so I am not making a pull request.
This file can be found here for your reference.
single header makes this library easy to use. But I think cmake or vcpkg will make it easier for people to include it into their projects.

Google Test branch is now main

CMake fails in the test directory due to a configuration error in Google Test. They have changed the "master" branch to "main". This issue can be resolved by changing line 8 in the test/CMakeLists.txt.in file to reflect the change in branch tag.

Small bug in example code

The "Write mesh-like data" example in the Readme references fInd, which should be meshFaceIndices