Comments (12)
the problem I think comes from this commit
7a4d78d
from tangos.
I think the best way forward with this is to change the output from the halo finder specific input handler such that finder_id maps to the catalog position rather than the ID number. This allows for, I think, easier fixes/changes to be made to other catalogs
from tangos.
The trouble is then we have three layers of IDs - the halo_number, the finder_id, and this new “sometimes the finder_id and sometimes something else” thing... is there a cleaner solution?
from tangos.
sorry I wasn't clear... my proposal is to keep just those two, but have the AHF input_handler output the "correct" finder_id rather than the one directly taken from the stat file. Because this is how the AHF catalog works in pynbody. finder_id must map to the halo such that h.load_copy(finder_id)
returns the desired halo. right now this would not be the case for MPI AHF.
from tangos.
the problem that I fore see is that the "correct" finder_id might depend on the handler you use (pynbody vs yt, etc)
from tangos.
I guess I am not following what is the ‘correct’ finder_id if not the one actually listed in the AHF stat file? It seems more like a bug in AHF than in tangos...
from tangos.
Note that this dependence I think already exists given the "id_offset" defined for AHF catalogs... that is clearly (I think?) specific to pynbody (i.e. ID 0 -> 1 and so on). presumably there could exist an input handler for particle data that used the original IDs rather than ID+1.
from tangos.
For halo finders run with MPI the ID of the halo is disconnected from the position within the file, because the latter is not known a priori. Each separate process writes out their halo information to the file separately so the final ordering really is just which process writes out the information first. In pynbody catalogs at least, for AHF, the order within the catalog is what matters. h.load_copy(N)
grabs the Nth entry in the list of halos, not halo with ID number N+1, though this is what happens in the "normal" (non-MPI) AHF output.
from tangos.
I suppose the question is whether we should change the AHF reader to handle the raw ID numbers or change tangos to just always use the raw order within the catalog.
from tangos.
Argh! It’s a horrible thing.
Basically my preferred solution would be to fix AHF so it post-processes and renumbers its halos to something a bit more sensible. But let’s assume that’s not going to happen in which case yes I agree the finder_id
should just be defined to be the thing you pass to your handler... in which case for pynbody handlers it would be whatever pynbody expects.
What a mess!
Are you able to make a PR that implements this change?
from tangos.
give me some time but yes, I can come up with a fix. My goal will be to constrain the fix to the AHF handler.... I suppose this is the challenge of trying to be useful to everyone!
from tangos.
The way to think about it I suppose is that we are defining the AHF catalog ID as the order within the AHF_halos file. Other people will need to implement their input handlers to compensate for that I suppose. I don't think that's a very big ask IMO (especially given that this is how it normally is!)
from tangos.
Related Issues (20)
- config_local.py not being read in properly
- TypeError in numpy called from BH.BHAccHistogram
- Can't access raw simulation data via Tangos tutorial page: ftp://ftp.star.ucl.ac.uk/app/tangos/tutorial_gadget.tar.gz HOT 3
- Tangos serve AND nosetests issues HOT 2
- Importing non-numeric properties from halo finder HOT 3
- tangos serve isn't working with certain databases HOT 4
- Tangos Importing of properties does not check if properties already exist? HOT 2
- SQLAlchemy warning HOT 1
- Draw the merger tree using the canvas rather than an SVG
- progenitor tracking is failing for BHs HOT 1
- Systematic database lock when importing first properties with MPI HOT 2
- Importing properties from old pynbody handlers might be incomplete
- Is it possible to calculate quantities for trackers that depend on their host halo properties? HOT 1
- Bug when linking timesteps in sqlite HOT 2
- Mixed type data returns from property calculations
- Tangos seems broken with SQLAlchemy > 2.0 HOT 1
- Installation instructions mention nose instead of pytests HOT 1
- consistent_trees_ids missing for phantoms HOT 2
- `pip install`ed version do not seem to include the .svg files HOT 3
- file_ignore_pattern ignored if extension is None in finding.py HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tangos.