• Now: F = 0.5
• Required: F such that formation time is when M200(zf) = M-2(z0)

This issue should be removed by ffcc17a and 359a66c. However, both of these are
now obsolete due to 75cdf6c (due to inefficiency of both algorithm and
implementation).

New test should be devised to remove unwanted haloes from CMH (much like
dc1b226, but for flattened, optimised non-tree calculation).

Plan

• bring plotting code from old projects into the repository (27c228f)

Plan

• spawn hbtp run on snapshots z=2, z=1, z=0
• bring dhalo, hbtp and hbt analysis code together
  • keep code in separate submodules in one repository
  • refactor common elements
• PR to Kambrian/HBTplus with an updated HBTReader.py

Please explain the issue. The first line will become the title. Trailing

lines starting with '#' (like these) will be ignored, and empty messages will

not be submitted. Issues are formatted with GitHub Flavored Markdown (GFM):

http://github.github.com/github-flavored-markdown

On oleskiewicz/merger_tree

#11: 2017-08-29

@oleskiewicz opened this issue 3 days ago. [open]

@oleskiewicz is assigned. [jrnl]

## Plan

- [ ] bring dhalo , hbtp and hbt analysis code together

- [x] move DHalo into a separate submodule (17167b3)

- [x] refactor HBTReader into a library and command-line (4b6f91a, 4ee975f)

- [ ] consider using caching (ala Galform plots) for efficiency (which is

now pretty bad, and all optimisation is left for under-the-hood Python)

- [x] create make pipeline for density profiles, CMHs and properties

(4ee975f)

- still some problems with Python filenames -> module names (Make does not

like that)

- [ ] PR to Kambrian/HBTplus with an updated HBTReader.py

On oleskiewicz/merger_tree

Plan

• bring dhalo, hbtp and hbt analysis code together
  • move DHalo into a separate submodule (17167b3)
  • refactor HBTReader into a library and command-line (4b6f91a, 4ee975f)
    • consider using caching (ala Galform plots) for efficiency (which is
      now pretty bad, and all optimisation is left for under-the-hood Python)
  • create make pipeline for density profiles, CMHs and properties
    (4ee975f)
    • still some problems with Python filenames -> module names (Make does not
      like that)
• PR to Kambrian/HBTplus with an updated HBTReader.py

Plan

• remove last two bins (x -> x[:-2], d -> d[:,:-2]) and check the profiles
  after change
  • they are better

• finer sieve on haloes
  • dynamic time (t_50 vs 2r200/v200)
  • centre of mass offset
• fit ax+b in rhof_rhos plane
• wiggle bins around
• fit Einasto profile instead of NFW
  • bilinear
  • test F parameter behaviour
• demonstrate predictive density profile (calculate from CMH)
  • invert Einasto & NFW formula to c(rho_s)
• PCA on m, z0, rho_crit(z), rho_s

• master plot for GR
• master plot for $f(R)$
  • nest ./output/{gr,fr}
  • CMH runs
  • other runs
• FoF evolution movie
  • particles
  • density profile
• CMH explanation
  • merger tree diagram
    • wrong ways to construct a tree
  • CMH vs MAH
• density profile explanation
  • density fits with and without cuts
  • Einasto fit
• PDF with slides

Optional:

• particle evolution differences

Exclude subhaloes with HostHaloId = -1 from merger trees

Plan

• include missing CMHs
  • re-run missing CMHs for all snapshots
  • forge
• schedule call with Aaron

Plan

• resolve issue with 2 most outside bins
  • run halo_virial code using Steve's configuration file
  • re-run halo_virial for snap > 81 (corrupt files)
• devise a correct binning
  • check subhalo mass function with hmfcalc
• re-run missing CMHs for all snapshots
• implement interpolation for formation calculation
• schedule call with Aaron

Plan

• find out M200Crit units
  • ask: 1e10 M_sun/h
• resolve issue with 2 most outside bins
  • ask:
    • R200mean should be closer to the FoF boundary
  • fit both full and limited
  • change particle query from particles > subhaloes > FoF groups to
    particles > FoF group
• devise a correct binning
  • check halo mass function
  • test a mass cut
  • check subhalo mass function with hmfcalc
• compare mass from FoF particle query and M200Crit
• schedule call with Aaron

Plan:

• render FoF movie in absolute frame (1f6b86e)
• implement HaloSize...hdf5 files (f152855)
• render FoF movie in normalised frame (a8a327f)
• implement full FoF density profile (based on subhalo profile) (a47fdfc)
• fix #1
  • filter non-centrals (dc1b226)
• create density profile & 3D particle plots (4692dc5)

Plan:

• extract CMH from fixed tree
  • fix #1
    • test if dc1b226 fixed the problem
  • fix #5
    • filter out all tracks with HostHaloId == -1
    • should be automatically fixed by only following FoF…?
  • flatten tree to progenitors
• extract density profile data (5a95636)
  • remove "empty" FoF haloes (i.e. 5507@23) from profile data
  • what is currently used as a plot, should be exported as a data file
  • preferably in a format digested by the code existing in halo_data

Plan

• devise a correct binning
  • check subhalo mass function with hmfcalc
• include missing CMHs
  • re-run missing CMHs for all snapshots
  • forge
• implement interpolation for formation calculation
• test if removing bins / imposing cut on data improves concentration fit
• remove np.std weights
• schedule call with Aaron

Haloes with HostHaloId == -1 should not be included in the merger tree.

Plan

• improve efficiency
  • move away from full MT extraction to a specialised CMH calculation
    • vectorised functions inside while loop (essentially reimplementing
      original halo_formation code) (e7cbbb6)
    • leverage "globalness" of IDs in HBT and simply query numpy named
      arrays for all HostHaloIds of all TrackIds in a root host halo
  • keep HDF5 files open for re-reading
    • unnecessary so long as point above is implemented
  • cache already available results
    • like new Galform plotting scripts
• spawn hbtp run on snapshots z=2, z=1, z=0
• bring dhalo, hbtp and hbt analysis code together
  • keep code in separate submodules in one repository
  • refactor common elements
• PR to Kambrian/HBTplus with an updated HBTReader.py

Plan:

• extract CMH from fixed tree
  • fix #1
    • test if dc1b226 fixed the problem
  • fix #5
    • filter out all tracks with HostHaloId == -1
    • should be automatically fixed by only following FoF…?
  • flatten tree to progenitors
• extract density profile data
  • what is currently used as a plot, should be exported as a data file
  • preferably in a format digested by the code existing in halo_data

Meeting:

• 3D particle videos
• fixed? - no non-central HBT trees
• discuss "main halo" definition from Ludlow+2016
• next steps: extract CMH & profile data for processing

Plan

• improve efficiency (75cdf6c)
  • move away from full MT extraction to a specialised CMH calculation
    • vectorised functions inside while loop (essentially reimplementing
      original halo_formation code) (e7cbbb6)
    • leverage "globalness" of IDs in HBT and simply query numpy named
      arrays for all HostHaloIds of all TrackIds in a root host halo

Plan:

• extract CMH from fixed tree (359a66c)
  • fix #1
    • dc1b226 fixes the problem
  • fix #5
    • filter out all tracks with HostHaloId == -1
    • should be automatically fixed by only following FoF…?
  • flatten tree to progenitors (d8b38a5, 359a66c)
    • non-trivial because of different ID conventions - favour on-the-fly CMH
      calculation
    • implemented in OOP style instead
• extract halo parameters: mass & radius