deepsphere / deepsphere-cosmo-tf2 Goto Github PK
View Code? Open in Web Editor NEWA spherical convolutional neural network for cosmology (TFv2).
Home Page: https://arxiv.org/abs/1810.12186
License: MIT License
A spherical convolutional neural network for cosmology (TFv2).
Home Page: https://arxiv.org/abs/1810.12186
License: MIT License
I created a conda environment, activated it and installed pip in it; I then followed steps 1 and 2 in https://github.com/deepsphere/deepsphere-cosmo-tf2, but it returns
"
note: This error originates from a subprocess, and is likely not a
problem with pip.
ERROR: Failed building wheel for scipy
Building wheel for PyGSP (setup.py) ... done
Created wheel for PyGSP: filename=PyGSP-0.5.1-py3-none-any.whl
size=1886744
sha256=622eda0ca58242241bf95de9e2aac471594f6db650332a7def884a07cd1d967c
Stored in directory:
/private/var/folders/5l/8sxy3dbs4_sbyqdhz5g0tv2w0000gn/T/pip-ephem-wheel-cac
he-1xrlcgi6/wheels/02/45/86/e3968e70e73a8a08f3f802ca85992fede12b1a01485f22f3
29
Successfully built PyGSP
Failed to build scipy
ERROR: Could not build wheels for scipy, which is required to install
pyproject.toml-based projects "
Installing scipy by hand doesn't solve the problem.
What can I do?
The python version is 3.11.5
TypeError Traceback (most recent call last)
in
1 tf.keras.backend.clear_session()
----> 2 model = HealpyGCNN(nside=nside, indices=indices, layers=layers, n_neighbors=20)
3 batch_size = 16
4 model.build(input_shape=(None, len(indices), 1))
5 #model.summary(110)
~/deepsphere-cosmo-tf2/deepsphere/healpy_networks.py in init(self, nside, indices, layers, n_neighbors)
85 hp_nn.Healpy_Transformer,hp_nn.HealpyBernstein)):
86 # we need to calculate the current L and get the actual layer
---> 87 sphere = SphereHealpix(subdivisions=current_nside, indexes=current_indices, nest=True,
88 k=self.n_neighbors, lap_type='normalized')
89 current_L = sphere.L
~/.local/lib/python3.9/site-packages/pygsp/graphs/nngraphs/spherehealpix.py in init(self, subdivisions, indexes, nest, **kwargs)
114 ' {} subdivisions.'.format(k, subdivisions))
115
--> 116 super(SphereHealpix, self).init(coords, k=k,
117 kernel_width=kernel_width,
118 **kwargs)
~/.local/lib/python3.9/site-packages/pygsp/graphs/nngraphs/nngraph.py in init(self, features, standardize, metric, order, kind, k, radius, kernel, kernel_width, backend, **kwargs)
417
418
--> 419 neighbors, distances = function(features, metric, order,
420 kind, k, radius, kwargs)
421 # ------ MARTINO's MODIFICATION ------
~/.local/lib/python3.9/site-packages/pygsp/graphs/nngraphs/nngraph.py in _scipy_ckdtree(features, _, order, kind, k, radius, params)
64 params['k'] = features.shape[0] # number of vertices
65 params['distance_upper_bound'] = radius
---> 66 distances, neighbors = tree.query(features, **params)
67 if kind == 'knn':
68 return neighbors, distances
_ckdtree.pyx in scipy.spatial._ckdtree.cKDTree.query()
_ckdtree.pyx in scipy.spatial._ckdtree.get_num_workers()
TypeError: Unexpected keyword argument {'n_jobs': -1}
and also I tested installation via pytest test and got same error in
test_gnn_transformers.py
tests/test_healpy_networks.py .
Now I'm using python 3.9 and I'm program noob. If you have some advice to fix this, it's great to get that.
Hello,
I am getting an error when I am running either 'pytest tests' or the quick_start notebook. This error is: "ValueError: All layers added to a Sequential model should have unique names. Name "" is already the name of a layer in this model. Update the name
argument to pass a unique name.";
which appears during the creation of the model:
model = HealpyGCNN(nside=nside, indices=indices, layers=layers, n_neighbors=20).
I am running deepsphere-cosmotf2 in tf.2.7 and python3.8.10. Anyone has ideas to solve the problem?
Thank you very much.
Hi,
I am trying to install deepsphere tf2. Although, it shows that the installation is completed, however, when trying to import dependencies from deepsphere, the following error occurs:
cannot import name 'SphereHealpix' from 'pygsp.graphs'
How should I solve this?
Many Thanks
Hi,
I've been having issues running the quick_start notebook (and also the 'pytest test')
The environment I'm using (ujsing you requirements.txt): Python 3.10.12, TF 2.16.1, scipy 1.8.0, no GPU
When I run the quick_start notebook, I got an error when
model.build(input_shape=(None, len(indices), 1))
is executed.
The error says:
`
File ~/****/deepsphere-cosmo-tf2/deepsphere/gnn_layers.py:81, in Chebyshev.build(self, input_shape)
79 stddev = 1 / np.sqrt(Fin * (self.K + 0.5) / 2)
80 initializer = tf.initializers.TruncatedNormal(stddev=stddev)
---> 81 self.kernel = self.add_weight("kernel", shape=[self.K * Fin, Fout],
82 initializer=initializer, **self.kwargs)
83 else:
84 self.kernel = self.add_weight("kernel", shape=[self.K * Fin, Fout],
85 initializer=self.initializer, **self.kwargs)
TypeError: Layer.add_weight() got multiple values for argument 'shape'
`
When I try the 'pytest tests' I got similar errors:
========================================================== short test summary info ========================================================== FAILED tests/test_gnn_layers.py::test_Chebyshev - TypeError: Layer.add_weight() got multiple values for argument 'shape' FAILED tests/test_gnn_layers.py::test_Monimials - TypeError: Layer.add_weight() got multiple values for argument 'shape' FAILED tests/test_gnn_layers.py::test_Bernstein - TypeError: Layer.add_weight() got multiple values for argument 'shape' FAILED tests/test_gnn_layers.py::test_GCNN_ResidualLayer - TypeError: Exception encountered when calling GCNN_ResidualLayer.call(). FAILED tests/test_gnn_transformers.py::test_Graph_ViT - TypeError: Layer.add_weight() got multiple values for argument 'initializer' FAILED tests/test_gnn_transformers.py::test_Graph_Transformer - TypeError: Layer.add_weight() got multiple values for argument 'initializer' FAILED tests/test_healpy_layers.py::test_HealpyChebyshev - TypeError: Layer.add_weight() got multiple values for argument 'shape' FAILED tests/test_healpy_layers.py::test_HealpyMonomial - TypeError: Layer.add_weight() got multiple values for argument 'shape' FAILED tests/test_healpy_layers.py::test_Healpy_ResidualLayer - TypeError: Exception encountered when calling GCNN_ResidualLayer.call(). FAILED tests/test_healpy_networks.py::test_HealpyGCNN_plotting - TypeError: Layer.add_weight() got multiple values for argument 'shape' FAILED tests/test_healpy_networks.py::test_HealpyGCNN - TypeError: Layer.add_weight() got multiple values for argument 'shape'
Is a problem of the environment? Have you encountered this before?
Thank you!
Hi,
This might not be the right place, but didn't find other contact options so posting it here.
It is regarding the autoencoder model in deepshphere tf2 tutorial. Using this tutorial, with a good amount of sample data, I am able to reconstruct a input healpy map with reasonably recovered features. However, instead of passing the training data in both x and y, if I try to pass a label dataset against the training dataset in y, in an attempt to denoising the map, I find that the model doesn't recover the desired features. I am wondering if there is any similar tutorial for denoising or if you have come across similar situation.
Many Thanks
Wasim
Hello, I was wondering if it's possible to use this library with a TPU, or if there are some limitations that prevent one to do it.
Thank you very much for your help!
Hello guys,
After training a custom deep-sphere model (based on ResidualLayer+HealpyChebyshev), I decided to save it in keras as usual, via callbacks:
callback_model= tf.keras.callbacks.ModelCheckpoint(
filepath=filepath+"cp-{epoch:04d}.ckpt",
monitor="val_loss",
verbose=1,
save_best_only=True,
save_weights_only=False,
mode="min")
However, I've got this error:
~/anaconda3/lib/python3.8/site-packages/keras/saving/saved_model/save_impl.py in call_and_return_conditional_lossess(*args, **kwargs)
631 def call_and_return_conditional_losses(*args, **kwargs):
632 """Returns layer (call_output, conditional losses) tuple."""
--> 633 call_output = layer_call(*args, **kwargs)
634 if version_utils.is_v1_layer_or_model(layer):
635 conditional_losses = layer.get_losses_for(
TypeError: call() got multiple values for argument 'training'
Then, I tried to save the model located in the examples folder: "quick_start.ipynb", and also I obtained the same error!! Models cannot be saved with model.save_weights(checkpoint_path) either. I wonder if this is a problem from my environment, or this is due to the custom layers defined in the repository.
I really appreciate your comments.
Best,
Javier
The polynomial values are computed in a loop:
deepsphere-cosmo-tf2/deepsphere/gnn_layers.py
Line 127 in 3facedf
For commonly used degree of 5, that requires 5 consecutive steps. It may be faster to execute this using a single sparse-dense-matmul.
That would require building a sparse laplacian for each polynomial component, which can be pre-computed.
The single sparse-dense-matmul can be launched once with input of
sparse-dense-matmul( block sparse-L, tile input )
followed by a reshape and a sum.
For N=polynomial degree and M=map length and ignoring channels and batch dimensions, the matrix multiplication could look like this
[ N*M x N*M ] is the block-sparse L
[ N*M x 1 ] is the input tiled N times
[ N*M x 1 ] = sparse-dense-matmul( [ N*M x N*M ], [ N*M x 1 ] )
[ N x M x 1 ] = reshape( [ N*M x 1 ], dims=[ N, M, 1 ] )
which gives N feature maps.
Any chance that would make sense?
Also tagging @Arne-Thomsen
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.