sunset1995 / py360convert

Python implementation of convertion between equirectangular, cubemap and perspective. (equirect2cube, cube2equirect, equirect2perspec)

py360convert

A new library including more fuctionality for 360 is under contruction by my colleague. This repo will be depreciated then.

Features of this project:

• Convertion between cubemap and equirectangular
  
• Equirectangular to planar
  
• Pure python implementation and depend only on numpy and scipy
• Vectorization implementation (in most of the place)
  • c2e takes 300ms and e2c takes 160ms on 1.6 GHz Intel Core i5 CPU

Requirements

• numpy
• scipy
• pillow (for example code to load/save image)

Install

pip install py360convert

Now at everywhere, you can import py360convert or use the command line tool convert360.

Command line examples

You can run command line tool to use the functionality. Please See convert360 -h for detailed. The python script is also an example code to see how to use this as a package in your code.

convert360 --convert e2c --i assert/example_input.png --o assert/example_e2c.png --w 200

Input Equirectangular	Output Cubemap

---

convert360 --convert c2e --i assert/example_e2c.png --o assert/example_c2e.png --w 800 --h 400

Input Cubemap	Output Equirectangular

You can see the blurring artifacts in the polar region because the equirectangular in above figure are resampled twice (e2c then c2e).

---

convert360 --convert e2p --i assert/example_input.png --o assert/example_e2p.png --w 300 --h 300 --u_deg 120 --v_deg 23

Input Equirectangular	Output Perspective

Doc

e2c(e_img, face_w=256, mode='bilinear', cube_format='dice')

Convert the given equirectangular to cubemap.
Parameters:

• e_img: Numpy array with shape [H, W, C].
• face_w: The width of each cube face.
• mode: bilinear or nearest.
• cube_format: See c2e explaination.

e2p(e_img, fov_deg, u_deg, v_deg, out_hw, in_rot_deg=0, mode='bilinear')

Take perspective image from given equirectangular. Parameters:

• e_img: Numpy array with shape [H, W, C].
• fov_deg: Field of view given in int or tuple (h_fov_deg, v_fov_deg).
• u_deg: Horizontal viewing angle in range [-pi, pi]. (- Left / + Right).
• v_deg: Vertical viewing angle in range [-pi/2, pi/2]. (- Down/ + Up).
• out_hw: Output image (height, width) in tuple.
• in_rot_deg: Inplane rotation.
• mode: bilinear or nearest.

c2e(cubemap, h, w, cube_format='dice')

Convert the given cubemap to equirectangular.
Parameters:

• cubemap: Numpy array or list/dict of numpy array (depend on cube_format).
• h: Output equirectangular height.
• w: Output equirectangular width.
• cube_format: 'dice' (default) or 'horizon' or 'dict' or 'list'. Telling the format of the given cubemap.
  • Say that each face of the cube is in shape of 256 (width) x 256 (height)
  • 'dice': a numpy array in shape of 1024 x 768 like below example
  • 'horizon': a numpy array in shape of 1536 x 256 like below example
  • 'list': a list with 6 elements each of which is a numpy array in shape of 256 x 256. It's just converted from 'horizon' format with one line of code: np.split(cube_h, 6, axis=1).
  • 'dict': a dict with 6 elements with keys 'F', 'R', 'B', 'L', 'U', 'D' each of which is a numpy array in shape of 256 x 256.
  • Please refer to the source code if you still have question about the conversion between formats.

Example:

import numpy as np
from PIL import Image
import py360convert

cube_dice = np.array(Image.open('assert/demo_cube.png'))

# You can make convertion between supported cubemap format
cube_h = py360convert.cube_dice2h(cube_dice)  # the inverse is cube_h2dice
cube_dict = py360convert.cube_h2dict(cube_h)  # the inverse is cube_dict2h
cube_list = py360convert.cube_h2list(cube_h)  # the inverse is cube_list2h
print('cube_dice.shape:', cube_dice.shape)
print('cube_h.shape:', cube_h.shape)
print('cube_dict.keys():', cube_dict.keys())
print('cube_dict["F"].shape:', cube_dict["F"].shape)
print('len(cube_list):', len(cube_list))
print('cube_list[0].shape:', cube_list[0].shape)

Output:

cube_dice.shape: (768, 1024, 3)
cube_h.shape: (256, 1536, 3)
cube_dict.keys(): dict_keys(['F', 'R', 'B', 'L', 'U', 'D'])
cube_dict["F"].shape: (256, 256, 3)
len(cube_list): 6
cube_list[0].shape: (256, 256, 3)

py360convert's Issues

Problem with Py360Convert and Pillow

When I play the code on Python it gives me this error and I think it is a problem with PIL

/bin/python3 "/home/jubei/Documentos/Python/Images 360/equiretangular.py"
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2813, in fromarray
    mode, rawmode = _fromarray_typemap[typekey]
KeyError: ((1, 1, 3), '<f8')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jubei/Documentos/Python/Images 360/equiretangular.py", line 9, in <module>
    image = Image.fromarray(equiretangular)
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2815, in fromarray
    raise TypeError("Cannot handle this data type: %s, %s" % typekey) from e
TypeError: Cannot handle this data type: (1, 1, 3), <f8

New PyPi Release

Thanks for your library @sunset1995 ! I was wondering if you could tag a new release for pypi; seems like some bugs have been fixed since the initial release. Thanks!

Can we get a fisheye to equirectangular implementation?

Just wondering how difficult it would be.

error in py360convert/utils.py line no: 217

Hey,
Using your example code :

import numpy as np
from PIL import Image
import py360convert

cube_dice = np.array(Image.open('MyImage.jpg'))

cube_h = py360convert.cube_dice2h(cube_dice)  # the inverse is cube_h2dice
cube_dict = py360convert.cube_h2dict(cube_h)  # the inverse is cube_dict2h
cube_list = py360convert.cube_h2list(cube_h)  # the inverse is cube_list2h
print('cube_dice.shape:', cube_dice.shape)
print('cube_h.shape:', cube_h.shape)
print('cube_dict.keys():', cube_dict.keys())
print('cube_dict["F"].shape:', cube_dict["F"].shape)
print('len(cube_list):', len(cube_list))
print('cube_list[0].shape:', cube_list[0].shape)

I've got this error message :

Traceback (most recent call last):
  File "PhotoToCubeMap.py", line 8, in <module>
    cube_h = py360convert.cube_dice2h(cube_dice)  # the inverse is cube_h2dice
  File "C:\Program Files\Python37\lib\site-packages\py360convert\utils.py", line 217, in cube_dice2h
    assert cube_dice.shape[0] == w * 3 and cube_dice.shape[1] == w * 4
AssertionError

python.exe: No module named convert360

Thanks for this great tool :)
Unfortunately I cannot execute it, any idea what I'm doing wrong?
I tried both python 2.7 and python3, both fail:

> python -m convert360 --convert e2c --i equirectangular-0.jpg -o e2c.png --w 512
D:\Program Files\Python27\python.exe: No module named convert360

> python -m py360convert --convert e2c --i equirectangular-0.jpg -o e2c.png --w 512
Traceback (most recent call last):
  File "D:\Program Files\Python27\lib\runpy.py", line 151, in _run_module_as_mai
n
    mod_name, loader, code, fname = _get_module_details(mod_name)
  File "D:\Program Files\Python27\lib\runpy.py", line 109, in _get_module_detail
s
    return _get_module_details(pkg_main_name)
  File "D:\Program Files\Python27\lib\runpy.py", line 101, in _get_module_detail
s
    loader = get_loader(mod_name)
  File "D:\Program Files\Python27\lib\pkgutil.py", line 464, in get_loader
    return find_loader(fullname)
  File "D:\Program Files\Python27\lib\pkgutil.py", line 474, in find_loader
    for importer in iter_importers(fullname):
  File "D:\Program Files\Python27\lib\pkgutil.py", line 430, in iter_importers
    __import__(pkg)
  File "D:\Program Files\Python27\lib\site-packages\py360convert\__init__.py", l
ine 1, in <module>
    from .e2c import e2c
  File "D:\Program Files\Python27\lib\site-packages\py360convert\e2c.py", line 3
, in <module>
    from . import utils
  File "D:\Program Files\Python27\lib\site-packages\py360convert\utils.py", line
 68
    out = np.ones((*out_hw, 3), np.float32)
                   ^
SyntaxError: invalid syntax

> python3 -m py360convert --convert e2c --i equirectangular-0.jpg -o e2c.png --w 512
C:\Users\me\scoop\apps\python\current\python.exe: No module named py360convert

> python3 -m py360convert --convert e2c --i equirectangular-0.jpg -o e2c.png --w 512
C:\Users\me\scoop\apps\python\current\python.exe: No module named py360convert._
_main__; 'py360convert' is a package and cannot be directly executed

> python3 -m convert360 --convert e2c --i equirectangular-0.jpg -o e2c.png --w 512
C:\Users\me\scoop\apps\python\current\python.exe: No module named convert360

Btw, with this tool, how can I generate 6 different image files for the cubemap faces?

Support for p2e conversion

I am trying to create a photosphere using a combination of a few normal camera images.
I know the fov, u, v and also the height and width of the input image.

My initial image (equirectangular)

My input image (planer or perspective view [fov=75, u=0, v=0, img_w=640, img_h=480])

I was able to get the (x,y) position using your e2p function and fit those images in the sample equirectangular image near the equator(mid of the image). But as I go near the edges, the images start to overlap since I am missing the pixel manipulation to form curves.

Can you please provide some reading material, since your code has no documentation or, if you got time, can you please help me out.

Add support for cube projecting with overlap

For photogrammetry it is helpful to have projected images with overlap on each face with its neighbouring faces.

Numba Support

With some rewriting to support Numba, this library could be even faster. It would require a bit of refactoring, but it might make the code even faster and unlocks the ability to use fastmath, autoparallelilzation and other features.

The jit_module function from numba gets you part of the way there, but there were several parts of the functions like c2e that needs to be rewritten into smaller functions or slightly modified to use the supported functions in numpy.

mapping between equirectangular and cubemap

Is there a way to get a mapping between coordinates of equirectangular and cubemap image

recommended reading materials

Thank you for this great contribution!
Yet, I am wondering where we could find are some reading materials that might help us understand the theory foundation behind the implementations.
Any recommendation would be appreciated!

error in py360convert/utils.py line no: 193

In py360convert/utils.py line no: 193, you have an unknown variable reference "k" commenting that line works though.

def cube_dict2h(cube_dict, face_k=['F', 'R', 'B', 'L', 'U', 'D']):
    # assert len(k) == 6
    return cube_list2h([cube_dict[k] for k in face_k])

Bug in c2e

I am using p360convert with python 3.7.8. I tried e2c and it worked as expected but I am having issue in c2e. I used the example, but same issue appeared. Am I doing it wrong or is this a bug?

cube_dice = np.array(Image.open('assert/example_e2c.png'))
Image.fromarray(py360convert.c2e(cube_dice,400,800), 'RGB').show()

a big bug in c2e

The list of equirectangulars generated by c2e is not right, look the picture：

the right, back and up picture is reversed, use an another tool, we got a right list pictures.

What does each cubemap format means

In the README.md, c2e categorize the input cubemap's format, 'dice' (default), 'horizon', 'dict' and 'list'. But, there's no further explain about that format because i guess there's no formal sentence for cubemap format.

I searched about cubemap formap and there's some shape of cubemap.

So, if anyone have time to add the more precise explain about format. Please add it :)
Thank you for reading.

Document is missing

TODO:

• introduction to the coordinate system this repo use
• functionality documentation & example with image