robinka / tfga

Python package for Geometric / Clifford Algebra with TensorFlow

TFGA - TensorFlow Geometric Algebra

GitHub | Docs | Benchmarks | Slides

Python package for Geometric / Clifford Algebra with TensorFlow 2.

This project is a work in progress. Its API may change and the examples aren't polished yet.

Pull requests and suggestions either by opening an issue or by sending me an email are welcome.

Installation

Install using pip: pip install tfga

Requirements:

• Python 3
• tensorflow 2
• numpy

Basic usage

There are two ways to use this library. In both ways we first create a GeometricAlgebra instance given a metric. Then we can either work on tf.Tensor instances directly where the last axis is assumed to correspond to the algebra's blades.

import tensorflow as tf
from tfga import GeometricAlgebra

# Create an algebra with 3 basis vectors given their metric.
# Contains geometric algebra operations.
ga = GeometricAlgebra(metric=[1, 1, 1])

# Create geometric algebra tf.Tensor for vector blades (ie. e_0 + e_1 + e_2).
# Represented as tf.Tensor with shape [8] (one value for each blade of the algebra).
# tf.Tensor: [0, 1, 1, 1, 0, 0, 0, 0]
ordinary_vector = ga.from_tensor_with_kind(tf.ones(3), kind="vector")

# 5 + 5 e_01 + 5 e_02 + 5 e_12
quaternion = ga.from_tensor_with_kind(tf.fill(dims=4, value=5), kind="even")

# 5 + 1 e_0 + 1 e_1 + 1 e_2 + 5 e_01 + 5 e_02 + 5 e_12
multivector = ordinary_vector + quaternion

# Inner product e_0 | (e_0 + e_1 + e_2) = 1
# ga.print is like print, but has extra formatting for geometric algebra tf.Tensor instances.
ga.print(ga.inner_prod(ga.e0, ordinary_vector))

# Exterior product e_0 ^ e_1 = e_01.
ga.print(ga.ext_prod(ga.e0, ga.e1))

# Grade reversal ~(5 + 5 e_01 + 5 e_02 + 5 e_12)
# = 5 + 5 e_10 + 5 e_20 + 5 e_21
# = 5 - 5 e_01 - 5 e_02 - 5 e_12
ga.print(ga.reversion(quaternion))

# tf.Tensor 5
ga.print(quaternion[0])

# tf.Tensor of shape [1]: -5 (ie. reversed sign of e_01 component)
ga.print(ga.select_blades_with_name(quaternion, "10"))

# tf.Tensor of shape [8] with only e_01 component equal to 5
ga.print(ga.keep_blades_with_name(quaternion, "10"))

Alternatively we can convert the geometric algebra tf.Tensor instance to MultiVector instances which wrap the operations and provide operator overrides for convenience. This can be done by using the __call__ operator of the GeometricAlgebra instance.

# Create geometric algebra tf.Tensor instances
a = ga.e123
b = ga.e1

# Wrap them as `MultiVector` instances
mv_a = ga(a)
mv_b = ga(b)

# Reversion ((~mv_a).tensor equivalent to ga.reversion(a))
print(~mv_a)

# Geometric / inner / exterior product
print(mv_a * mv_b)
print(mv_a | mv_b)
print(mv_a ^ mv_b)

Keras layers

TFGA also provides Keras layers which provide layers similar to the existing ones but using multivectors instead. For example the GeometricProductDense layer is exactly the same as the Dense layer but uses multivector-valued weights and biases instead of scalar ones. The exact kind of multivector-type can be passed too. Example:

import tensorflow as tf
from tfga import GeometricAlgebra
from tfga.layers import TensorToGeometric, GeometricToTensor, GeometricProductDense

# 4 basis vectors (e0^2=+1, e1^2=-1, e2^2=-1, e3^2=-1)
sta = GeometricAlgebra([1, -1, -1, -1])

# We want our dense layer to perform a matrix multiply
# with a matrix that has vector-valued entries.
vector_blade_indices = sta.get_kind_blade_indices(BladeKind.VECTOR),

# Create our input of shape [Batch, Units, BladeValues]
tensor = tf.ones([20, 6, 4])

# The matrix-multiply will perform vector * vector
# so our result will be scalar + bivector.
# Use the resulting blade type for the bias too which is
# added to the result.
result_indices = tf.concat([
    sta.get_kind_blade_indices(BladeKind.SCALAR), # 1 index
    sta.get_kind_blade_indices(BladeKind.BIVECTOR) # 6 indices
], axis=0)

sequence = tf.keras.Sequential([
    # Converts the last axis to a dense multivector
    # (so, 4 -> 16 (total number of blades in the algebra))
    TensorToGeometric(sta, blade_indices=vector_blade_indices),
    # Perform matrix multiply with vector-valued matrix
    GeometricProductDense(
        algebra=sta, units=8, # units is analagous to Keras' Dense layer
        blade_indices_kernel=vector_blade_indices,
        blade_indices_bias=result_indices
    ),
    # Extract our wanted blade indices (last axis 16 -> 7 (1+6))
    GeometricToTensor(sta, blade_indices=result_indices)
])

# Result will have shape [20, 8, 7]
result = sequence(tensor)

Available layers

Class	Description
GeometricProductDense	Analagous to Keras' Dense with multivector-valued weights and biases. Each term in the matrix multiplication does the geometric product x * w.
GeometricSandwichProductDense	Analagous to Keras' Dense with multivector-valued weights and biases. Each term in the matrix multiplication does the geometric product w *x * ~w.
GeometricProductElementwise	Performs multivector-valued elementwise geometric product of the input units with a different weight for each unit.
GeometricSandwichProductElementwise	Performs multivector-valued elementwise geometric sandwich product of the input units with a different weight for each unit.
GeometricProductConv1D	Analagous to Keras' Conv1D with multivector-valued kernels and biases. Each term in the kernel multiplication does the geometric product x * k.
TensorToGeometric	Converts from a tf.Tensor to the geometric algebra tf.Tensor with as many blades on the last axis as basis blades in the algebra where blade indices determine which basis blades the input's values belong to.
GeometricToTensor	Converts from a geometric algebra tf.Tensor with as many blades on the last axis as basis blades in the algebra to a tf.Tensor where blade indices determine which basis blades we extract for the output.
TensorWithKindToGeometric	Same as TensorToGeometric but using BladeKind (eg. "bivector", "even") instead of blade indices.
GeometricToTensorWithKind	Same as GeometricToTensor but using BladeKind (eg. "bivector", "even") instead of blade indices.
GeometricAlgebraExp	Calculates the exponential function of the input. Input must square to a scalar.

Notebooks

Generic examples

Using Keras layers to estimate triangle area

Classical Electromagnetism using Geometric Algebra

Quantum Electrodynamics using Geometric Algebra

Projective Geometric Algebra

1D Multivector-valued Convolution Example

Tests

Tests using Python's built-in unittest module are available in the tests directory. All tests can be run by executing python -m unittest discover tests from the root directory of the repository.

Citing

See our Zenodo page. For citing all versions the following BibTeX can be used

@software{python_tfga,
  author       = {Kahlow, Robin},
  title        = {TensorFlow Geometric Algebra},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.3902404},
  url          = {https://doi.org/10.5281/zenodo.3902404}
}

Disclaimer

TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc.

tfga's Issues

Support Convolution with MV-valued kernels

Can naively be implemented using matrix multiply and toeplitz matrix but is way too wasteful so it really should be implemented using a custom op (see https://github.com/tensorflow/custom-op).

Should support standard convolution and graph-convolution (useful eg. for operating on vertices / meshes).

// Edit
Tried implementing this in two different ways using only built-in ops:

• Using N^2 normal convolutions (N=#blades), build x_ij... = conv(a_...i, k_j), then contract with cayley tensor to get result x_ij..., c_ijk -> y_...k. This approach uses way too much memory (as we store something the size of the output times N^2).
• Same as first, but instead of doing N^2 normal convs in parallel, we do them sequentially and accumulate the results. The memory usage of this is good but is way too slow (as the convs are done sequentially)

The only good solution seems to be writing a custom op with cuda kernel.

Make all Keras layers serializable

get_config() needs to return a serializable dict, right now it contains some tensors and other objects which can't be serialized

Add exp and log

Here's how galgebra solved exp: https://github.com/pygae/galgebra/blob/b5eb070340434d030dd737a5656fbf709538b0b1/galgebra/mv.py#L1093-L1145

https://www.euclideanspace.com/maths/algebra/clifford/algebra/functions/exponent/index.htm

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4361175/

• Exp: #20
• Log

Create module for interfacing with the clifford python package

Create a module for interfacing with Clifford.

• Convert from clifford's MVArray to TFGA tf.Tensor
• Convert from TFGA tf.Tensor to MVArray (or maybe that should be part of Clifford instead)

Make blade parameters consistent

Right now when passing blades to functions, we either pass blade indices, blade kind or blade names. This should be made more consistent so all functions take the same type of input.

Two possible options:

• Make everything take blade indices, provide functions to easily convert from kind or name to index
• Create a new data structure that can hold one of the three and make all functions accept this data structure

Add Tensorflow / Keras layers

Could add layers such as GeometricDense(kind=...) analagous to keras.Dense, where instead of matrix-multiplying with scalars, we matrix-multiply using the geometric product.

Add elementwise multiplication layer

Multivector Coefficients

Right now only tf's dtypes as coefficients are supported. Multivector coefficients could be convenient for example to use dual numbers for automatic differentiation (although this is already possible without them since we are using tensorflow of course).

Possible implementation

• N blades in the algebra
• For each blade, we have a coefficient that is a multivector itself, so an [N, N] tensor.
• Take elementwise geometric product of N multivector-coefficients with the N blades to get a single multivector

ga = tfga.GeometricAlgebra(metric=[0])

# Coefficient 5 + e_0 for all blades
# coefficients.shape: [2, 2]
# [5 + e_0, 5 + e_0]
coefficients = tf.tile(tf.expand_dims(ga.e0 + 5.0 * ga.e(""), axis=0), [ga.num_blades, 1])

# blades.shape: [2, 2]
# [1, e_0]
blades = ga.blade_mvs

# mv = [5 + e_0, 5 + e_0] elementwise geom. prod. [1, e_0] = [5 + e_0, 5 e_0]
# mv.shape: [2, 2]
mv = ga.geom_prod(coefficients, blades)

Survey on the Quaternion Usage in Neural Network

I think it is interested to think of the projects using TFGA for quaternion transformation with nn.

Improve geometric product performance

Right now the geometric product is basically done by using a 3-Tensor C_ijk which is very sparse:

a_i b_j C_ijk -> y_k

Initially in this project we attempted to exploit this sparsity by slicing out all the zeroes of the 3-tensor. However this turned out to be ~50 times slower on GPU for full mv mv products in STA. Find a better, GPU-friendly way to exploit this sparsity.

Availability of Exp(.) and Log(.)

Does this library have the capability mentioned in #14? I should be working in PGA (G+ 3,0,1) and I'm trying to grok the text mentioned below as to whether I need CGA (4,1) or Lie Alg/Group transformations.

Utilizing chained applications of screws/motors on MediaPipe landmarks without actual kinematics should be sufficient (though I'm hand-waving here without fully understanding much...)

Depending on how complicated things become, I may abandon using GA anyways, as the code I'm writing for Kaggle cloud instances requires Python 3.7 and code that diverges on my local machine is already an existential risk. The TF Lite submission can be developed wherever and only needs to be submitted online.

However, I have an AMD GPU and I'm running into some compatibility issues anyways. I got a docker container to run with ROCm and I'm checking the functionality now. Your library seems to extend TF without adding C++ code, so AFAIK it wouldn't need a rebuild. The errors I'm getting now involve SEE3/etc and AVX/2 instructions in oneDNN which indicates I need to rebuild anyways. I think I can handle that.

I don't really need help with the model and that may be against the rules. I'm not sure. To give you some background on what I'm doing:

The project is for a Kaggle competition using MediaPipe data on ASL and I think using GA would give me an advantage. I have this GA Applications text for Robotics to help me, but the math is a bit over my head. I understand some aspects from a high-level.

What I'm hoping to gain via GA is using some multivector values for data analysis & training, potentially using this method for live classification, but the rules require the TFLite build to be less than 40MB. Other submissions are reportedly less than 5MB which seems low.