Comments (21)
YES! We definitely need a small benchmark project. Of course, we can also include Montage in the benchmark. Do you want to create a repository for this?
from reproject.
Yeah, sure, will do. Any thoughts on good input images to test? I was just using a single gaussian...
from reproject.
Also, do you know how to use kapteyn well enough to include it in such a suite?
from reproject.
@keflavich - how about picking a cutout from a Spitzer survey? I don't use kapteyn, so not really.
from reproject.
@astrofrog - OK, will try a cutout from Spitzer at some point. For now, starting with the ultra-trivial examples...
https://github.com/keflavich/reprojection_benchmarks
from reproject.
Awesome! This will be a great way to benchmark accuracy, speed, and approximations!
from reproject.
yep. Right now, it looks like everything is wrong... hooray!
from reproject.
It's going to be interesting ;) If the cutout is small enough, can you include it in the repository?
from reproject.
So it looks like starlink crops, reproject leaves nans as nans, and hcongrid turns nans to zeros (but there's a setting for that).
I think this indicates a problem in reproject, actually.
from reproject.
@astrofrog - yes, will include any cutouts. For now, I'm using generated data, as I think it's a more reliable starting point. There might even be a True, Correct, Analytical "Right Answer" when using generated data.
from reproject.
@keflavich - just so I understand, is reproject the one in this package, or the wrapper around Montage?
from reproject.
@astrofrog - this package. I'll rename it 'pyreproject' as I'm adding the montage-reproject now
from reproject.
I wouldn't be surprised that pyreproject
is wrong :) But hopefully it will guide us towards the 'right'.
from reproject.
Yep! Here it is with montage:
from reproject.
Interesting - the packages clearly treat edge conditions differently. I think that's tunable in montage, using exact_size
. Not sure about starlink. @dsberry, whenever you have a chance, could you comment on when/whether AST chooses to crop out NaN pixels along the edges?
from reproject.
@astrofrog - if you have any tips or ideas on how to make a better grid without spending half an hour tuning the plot, I'd be interested. I've started writing wrappers of this kind of stuff for myself recently in https://github.com/keflavich/mpl_plot_templates; perhaps an aplpy-grid wrapper is called for
from reproject.
Timing benchmarks now:
reproject_cmd 8 16 32 64 128
pyreproject 0.208455085754 0.575993061066 2.24823093414 7.49773812294 29.2678470612
montage_hdu 0.541980981827 0.587301015854 0.491894006729 0.515506029129 0.596334934235
hcongrid_hdu 0.070965051651 0.076486825943 0.0782270431519 0.0865521430969 0.129561901093
wcsalign 0.286320924759 0.232869148254 0.229321956635 0.230861902237 0.239408016205
from reproject.
cool - as I said, pyreproject
has not at all been optimized yet :)
from reproject.
Indeed, but now you can! Those nested for-loops look ripe for
cython/numba/pypy-ification.
from reproject.
@keflavich - indeed! Actually I first want to make sure the results are correct, which I think they are not. Then, I'll add a unit test that verifies the benchmark output. Then I'll optimize.
from reproject.
The Ast.Resample method does not crop the output - it is created with the pixel bounds you request. If this area is larger than the area containing good input values, then the borders are filled with bad values. Likewise, any bad values in the input are propagated to the output if they fall within the requested output bounds. But starlink does not use NaNs to represent bad values, it uses a special value which you specify using the "badval" argument when calling Resample. See the section "Propagation of Missing Data" at http://starlink.jach.hawaii.edu/devdocs/sun211.htx/node384.html
from reproject.
Related Issues (20)
- Building with Python 3.11 fails HOT 1
- devdeps CI is failing HOT 2
- Implement support for dask input/output
- reproject_interp with parallel=True ignores order='nearest-neighbor'? HOT 2
- Avoid writing out a temporary memmap if the input data can already be recognized as a memmap HOT 1
- Use multi-threading instead of multi-processing HOT 40
- Upload wheels for Python 3.12 HOT 3
- reproject_interp docstring says it only takes FITS-WCS
- test_reproject_healpix_to_image_footprint failures on Debian with ARM (and other) CPUs HOT 6
- Cython performance warnings in deforest.pyx
- Reproject to Non-standard Coordinate Systems HOT 1
- BLD: build is currently broken on Python 3.8 HOT 4
- Blank pixels should be set to ... HOT 2
- In find_optimal_celestial_wcs, don't assume cdelt[0] is always negative
- Chess-like pattern when using rountrip_coords with GWCS, when using dask HOT 3
- map_coordinates in adaptive reprojection does not accept read-only arrays
- Document logging
- Artefacts appearing during reprojecting back to healpix HOT 5
- test_reproject_order[block_size1] test fails on the installed package
- Reproject using biquadratic and bicubic interpolation returns Nans HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from reproject.