magicdgs / hdf5j Goto Github PK

The FileAddress and FileAddressManager interfaces should be refactored after #24, to be in the utils package. Maybe they can live in the package for the implementation of the file format, but that requires to start the implementation after the layaout is included.

To make the interface independent of our FileAddress object, we should return a number value instead of the object itself to allow other implementations to get rid of our abstraction. There are two possibilities:

• Return a long. This will allow to seek directly a file by using the returned value, and it will enforce a maximum value for the implementations. We are not planning to support pointers larger than a long yet (I have to think how that can be done).
• Return a Number. Like that, some implementations can use BigIntegeror their own number implementation: e.g., our FileAddress can extend Number to be used directly. In this case, we should point in our javadoc that the number will be converted to long for being seek, so arithmetic exceptions should be considered.

I prefer the first point, because like that all the enforcing is implementation-dependent. Something similar will be done with the object sizes.

After #18, we should create an annotation to indicate in block interfaces/implementations which methods are allowed for each version. Some ideas:

• Interface methods specific for some version should have an annotation to provide the allowed versions for it. Programmatically, an API user can get the methods allowed for the block-version: the getVersion method from the block can be compared to the annotation versions; if the annotation is not present, all the versions can call the method.
• Implementations for concrete blocks may contain an annotation for methods that are forbidden, because they may fail if they are called.
• The implementation for the latest version of a concrete block should have another annotation, to be able to find the latest implementation. New files should be written always using the latest version of every block. The rest of the implementation is for backwards-compatibility with previous formats (reading and maintaining the format of the block if copied).

More ideas will come, but this is once the project is a bit more mature...

This will allow to keep a clean documentation for the whole library.

Because java is always signed, the interfaces force to return int for unsigned bytes/shorts and long for unsigned integers. In addition, the signatures are also encoded as unsigned byte arrays. Thus, the API user should be cautious on how we treat this values for: encoding/decoding, operating with returned values, enforce the values to fit into the range, etc. For instance, we can change the signature for the returned values to be the correct one (byte or short), and then the user should be aware that they should operate as unsigned values (e.g., using Guava).

One idea is to implement some annotations to mark how HDF5 specifies this fields. Something like @HDF5UnsignedByte will be enough. Maybe this will be also useful to perform encoding/decoding by getting the annotations for each field, but that's a different topic.

Because each block has it's own version, we require a block interface to be able to add some common checking of versioning independently of the block. This block requires:

• A method to return the version type.
• A method to retrieve the block size. This may be useful to allocate memory for writing the block (or for allocate the memory/disk-space for adding the block afterwards)
• A method to write the block into the disk.

http://www.burtleburtle.net/bob/c/lookup3.c

Maybe it is a good idea to implement a multi-module structure for the HDF5j project: one with the API/layout (to allow custom implementations) and another one with the pure java implementation.

The idea behind this is to being able to have a plugin to use the native code with our own layout, but that is a future plan for compare performance and implementation differences.

We should implement reading/writing of superblock version 0, 1, 2 and 3.