davidgardenier / chromos Goto Github PK

Aborts extraction as "Filtering resulted in having no data!". Could perhaps have to do with the bin size (usually 1/128s) that you give as input?

Check in create_power_spectra and figure out a way to correct for number of segments is 0.

Currently the output of pyxspec is displayed in the terminal, but is not piped to a the log file of the calculate_hi scripts (presumably as it's not using a print command).

One file (20092-01-05-030) has a obsid which is one character longer than all other obsids. Unsure whether any part of the code may trip up due to this.

Runtime calculation in logging currently shows full runtime rather than runtime per subscript.

Currently only saving the ratio of these bands, not the separate power colours.

It seems that sometimes X-ray flares are detected in say event mode files, but not in the binned mode files. This could be because the binned mode files have a different time range (and are usually only in the order of seconds), but could also be because the count rate never gets above 7sigma. Ideally I should check which case is happening, but at the moment none of the binned files pass the power spectra step anyway, so not too concerned.

You might want to consider using astropy.io.fits instead of fitsio. There's a nice breakdown of the performance differences here: http://docs.astropy.org/en/stable/io/fits/appendix/faq.html#what-performance-differences-are-there-between-astropy-io-fits-and-fitsio

Response files are occasionally being read in to XSPEC with an ampersand at the end of the file name. I thought I had fixed this earlier in the year, but still occasionally crops up. Perhaps the version of astropy/xspec have changed in the mean time, leading to different behaviour. The quick fix is to (manually) copy the response name (without including the ampersand) into the queried input on which XSPEC stops, and press enter. Means some additional oversight is needed for some objects (unsure when this issue does or doesn't pop up).

At the moment the log script will start logging as soon as the function is called in the main pipeline, but won't stop logging when a new log starts.

Adapt the create_power_colours script so that errors are calculated

At the moment I don't download obsids folders with a letter behind them, but in theory folders with a 'G' behind it should work (means _G_appy, so has longer time gaps in the observations).

Don't have this implemented at the moment, using no bitfile (as allowed when using all pcus) at the moment.

Currently the scripts only use one event mode file per obsid. File input & output options will have to be changed to allow for multiple event mode files

If wanting to group power colour ratios in sets other than the arbitrary time ranges of an obsid, you'll need power colour ratios per time units (in this case per 512s would be a good start - you'll not want to go to a lower time-resolution).

FYI: indents should be spaces, not tabs. Four spaces is standard for one indent level.

Most (fancier) text editors will let you specify that hitting the 'tab' key actually uses 4 spaces, so that you can indent with spaces and still be lazy (by only hitting one key).

Sort out the errors on power spectra after binning

At the moment if an observation falls exactly on the boundary between two epochs, pcarsp won't create a response file, but asks you to rerun it with the '-d' option to shift the observation by a day. I skip these files at the moment, but these should ideally be caught and rerun with the '-d' option.

Currently the filter files do not cut on basis of PCU changes. Might not even have to implement it, but perhaps plotting will be enough to show PCU changes have taken place. If cutting, remove 32 seconds around the change.

As title suggests

How to find if everything's run well (ERROR, WARNING etc)

Relates to master_project/pc_ss/D_create_background.py, fixed in pc_hid

If using pcabackest to create backgrounds for std2 files - then fullspec should equal 'no'. Although if using pcabackest for other files which also need no gaincorrection, it should be 'yes'.

Simple task of adding a before and after Southern Atlantic Anomaly filter in the maketime filter creation.

The output of Phil's xtescan2 provides Standard2 lists, linking to original files. However, the list can change if you run it multiple times, in later runs including .bkg and .std2bkg files. I've quickly dealt with this by taking every third line - but it is only valid if you've run the pipeline multiple times. If you were to run the pipeline only once (over a new data set), you would need every line. The root of the problem lies in finding why and how those 'extra' files are being created. Only related to pc_ss from what I can tell.