datacarpentry / astronomy-python

Astronomy

Contributing

We welcome all contributions to improve the lesson! Maintainers will do their best to help you if you have any questions, concerns, or experience any difficulties along the way.

We'd like to ask you to familiarize yourself with our Contribution Guide and have a look at the more detailed guidelines on proper formatting, ways to render the lesson locally, and even how to write new episodes.

Please see the current list of issues for ideas for contributing to this repository. For making your contribution, we use the GitHub flow, which is nicely explained in the chapter Contributing to a Project in Pro Git by Scott Chacon. Look for the tag . This indicates that the maintainers will welcome a pull request fixing this issue.

Maintainer(s)

Current maintainers of this lesson are

• Azalee Bostroem
• Rodolfo Montez Jr.
• Ralf Kotulla

Authors

A list of contributors to the lesson can be found in AUTHORS

Citation

To cite this lesson, please consult with CITATION

astronomy-python's Issues

Python code chunks not showing with syntax highlighting in exercise

This is in lesson 5 (joins):

Problem with exercise syntax

I think all of the exercises are missing the line that marks the start of the solution:

> > ## Solution

Example:

Exercise

One of the other tables we'll use is
gaiadr2.panstarrs1_original_valid. Use load_table to get the
metadata for this table. How many columns are there and what are
their names?

meta2 = Gaia.load_table('gaiadr2.panstarrs1_original_valid')
print(meta2)

for column in meta2.columns:
    print(column.name)

{: .language-python}
{: .solution}
{: .challenge}

Needs to be:

Exercise

One of the other tables we'll use is
gaiadr2.panstarrs1_original_valid. Use load_table to get the
metadata for this table. How many columns are there and what are
their names?

Solution

meta2 = Gaia.load_table('gaiadr2.panstarrs1_original_valid')
print(meta2)

for column in meta2.columns:
    print(column.name)

{: .language-python}
{: .solution}
{: .challenge}

Fix Link to setup instructions from "Starting from this episode"

The link to the data in the "starting from this episode" section at the beginning of each episode doesn't work.

launch_job returns more than 2000 rows

I was just going over the first episode with a student and tried to show that launch_job wouldn't return 3000 rows .... but it did. We should investigate further and modify the text accordingly

Check on installation issues

remove wget from test notebook
check name for ipykernel note

Make "Starting from this episode" collapsable

The text under "Starting from this episode" at the start of each episode should be collapsed under the heading so that learners don't see it by default

Remove wget from test notebook (and environment file?)

The test notebook uses wget which I don't think we use anymore and it should it be removed.

We can put info about invert_yaxis in callout

Lesson 6 has "gotcha" about invert_yaxis that can go into a callout. Can also format the code chunks with python syntax highlighting within callout.

Photometry - we make cuts on Gaia photometry early on (bp_rp color cuts) - so not first time we see photometry - maybe highlight why we need

Instructions

Thanks for contributing! ❤️

If this contribution is for instructor training, please email the link to this contribution to
[email protected] so we can record your progress. You've completed your contribution
step for instructor checkout by submitting this contribution!

If this issue is about a specific episode within a lesson, please provide its link or filename.

Keep in mind that lesson maintainers are volunteers and it may take them some time to
respond to your contribution. Although not all contributions can be incorporated into the lesson
materials, we appreciate your time and effort to improve the curriculum. If you have any questions
about the lesson maintenance process or would like to volunteer your time as a contribution
reviewer, please contact The Carpentries Team at [email protected].

You may delete these instructions from your comment.

- The Carpentries

Episode 4: gd1_frame definition missing in new notebook

Dear maintainers,

At the top of Episode 4, an instruction is given which modules and variables need to be loaded if the episode is started with a new notebook. In this instruction, the definition of the variable gd1_frame is missing.

Later in the episode, in the section "Assembling the query", gd1_frame is being used in a call to SkyCoord(). If gd1_frame is not defined, this leads to an error.

Could you add the line gd1_frame = GD1Koposov10() to the instructions for new notebooks at the start of this Episode to fix the issue?

Thanks and cheers, Jelle de Plaa.

remove Pre-assessment git question

The question about use of git should be removed from the pre-assessment

Add parallax to output of Episode 1 query3

we have parallax in the columns but the output we show doesn't include the parallax column. Here are the updated columns:

'SELECT \nTOP 10 \nsource_id, ra, dec, pmra, pmdec, parallax\nFROM gaiadr2.gaia_source\nWHERE parallax < 1\n  AND bp_rp BETWEEN -0.75 AND 2\n'

SELECT 
TOP 10 
source_id, ra, dec, pmra, pmdec, parallax
FROM gaiadr2.gaia_source
WHERE parallax < 1
  AND bp_rp BETWEEN -0.75 AND 2

<Table length=10>
   name    dtype    unit                              description                            
--------- ------- -------- ------------------------------------------------------------------
source_id   int64          Unique source identifier (unique within a particular Data Release)
       ra float64      deg                                                    Right ascension
      dec float64      deg                                                        Declination
     pmra float64 mas / yr                         Proper motion in right ascension direction
    pmdec float64 mas / yr                             Proper motion in declination direction
 parallax float64      mas                                                           Parallax
Jobid: None
Phase: COMPLETED
Owner: None
Output file: 1668100376332O-result.vot.gz
Results: None

lines not formatted with python syntax highlighting

There are a few lines of code in lesson 7 where the code is showing up as plain text rather than with Python syntax highlighting. Two examples:

plt.gca().tick_params(direction='in')

plt.style.use('fivethirtyeight')

The second example is followed immediately by an empty code chunk.

Update astropy in environment.yml to v 5.0 and check with lessons

Version 5.0 of astropy was recently released. We should check that students are getting this version when using the environment.yml file for installation and that it doesn't break anything

Add example of plotting with custom style file

plot_cmd(candidate_df)

plt.plot(left_color, g, label='left color')
plt.plot(right_color, g, label='right color')

plt.legend();

Broken link to paper in episode 2

The URL and displayed text should be swapped in the link to Figure 1 of the Price-Whelan and Bonaca paper.

Change callout about starting from this episode to pre-reqs

Based on carpentries/trainers#171 - and conversation with Curriculum Team, makes more sense to have these callout boxes rendered as prereq boxes. I will do that today.

Self organized workshop description

Right now the curriculum stands out on the self organized workshop page as either not having a description or not being associated with DC

Clean up instructor notes

Instructor guide contains a few references that are out dated - clean up to match current lessons

Explicitly return to default style

In episode 8, before recreating Figure 1 we should return learners to the default style
plt.style.use('default')

Relative versus absolute links to figures

Throughout the lessons, figures are flipping between relative paths and absolute paths. What is the best-practice?

Checklist of actionable feedback from second beta pilot

Summarized from second beta pilot notes

Typos / errors: (all addressed in #108)

• Remove notebook directory
• the documentation link for the column names under the Pan-STARRS table is incorrect - it just links back to the JOIN episode - should link to: https://gea.esac.esa.int/archive/documentation/GDR2/Gaia_archive/chap_datamodel/sec_dm_external_catalogues/ssec_dm_panstarrs1_original_valid.html
• TYPO: Episode 8 "At the very least, those details (and the isochrone line) could be called out in the legend" (legend should be caption)
• TYPO: In plotting GD-1 with annotations exercise, "plot_cmd_function" should be "plot_cmd_selection"

Set-up and Instructor notes:

• include paper link in setup as recommended reading before workshop - #108
• Note in Instructor notes that "Checking the match" section can be skipped - #108
• Make the setup instructions more prominent on the workshop website. --> OUT OF SCOPE - requires edit to https://github.com/carpentries/workshop-template

Callouts and explanations to add:

• Add callout for asynchronous queries section about getting different values that instructor in the table when using Top Ten. ​'Top Ten' just grabs 10 results but that the system isn't organized in a specific way unless you code in something to ask for it to organize by. The fact the source_ids are all random and not sequential backs up that idea for me too - #108
• Add callout in working with units section - For arcmin: Make sure you use the single-quote ' on you keyboard that is beside the "Enter" key! The character up on the left hand side of the number row is a tick that looks similar, but doesn't work the same.
  • I don't see anywhere where learners have to use the single-quote to represent arcmin. The solutions have them use u.arcmin. @abostroem - is this callout neccessary?
• Style sheets, add callout about how to swap back to the default style. Answer plt.style.use('default') - #108
• #119
• Make sure GD-1 is introduced in lesson: "GD-1 is a stellar stream around the Milky Way. This means it is a collection of stars that we believe was once part of a bound clump, but the gravitational influence of the Milky Way has torn it apparent and spread it over an arc that traces out its orbit on the sky.
  This is interesting, because if the original bound clump was a dwarf galaxy, understanding its orbit with sufficient precision allows us to measure the mass of the Milky Way, which is very important for understanding the future and pass of the Milky Way as a whole. But that is much easier to do if we have a coordinate system aligned with the stream because that makes fitting the location of the stars much easier mathematically - it's a line instead of some weird funky curve.
  Additionally, this stream is especially interesting because it has "gaps", which have a natural interpretation as being caused by the influence of small clumps of dark matter passing near the stream. And knowing the typical rate of these gaps tells you about the typical size and density of these clumps, which turns out to be one of the best probes we have of the fine structure of dark matter."
• Add explanation for “which points are in the polygon” exercise: This exercise could use a little more context (e.g. we never actually say that contains_points is a method on the Polygon object). Perhaps “The results is a Polygon object which has a contains_points method built in. This allows us to pass contains_points a list of points and for each point this will tell us if the points are contained within the polygon. Then in the exercise: Your points list should be a list of tuples, where each tuple two elements representing an x,y pair. Call contains_points on the polygon we just created (e.g. polygon.contains_points), passing it the list of points you just created
• Write the data - This section intro could use a sentence calling out reproducibility
• plotting GD-1 with annotations exercise - specify creating one of the arrows on the plot (e.g. spur, gap)

Other:

• British vs American spelling of neighbor in queries (and possibly other things)
• Rename data directory to something explicit like back up data?
• Do we ever re use the front_to_back function? Could this just be code?
• At the end of episode 7, could we use plot_cmd to plot the winner_df points too? Maybe the only modification would be to not set the color in the function itself?
• Add link to onboarding Youtube video - #108

Add kernel installation instructions to setup?

In the setup instructions, learners are guided through a fresh install of anaconda.

https://datacarpentry.org/astronomy-python/setup.html

I came at it slightly differently since I have anaconda and other conda venvs present. Creating the new env with
conda env create -f environment.yml
left me unable to

Should we include the instructions to make a new kernel to jupyter, ie.

$ python -m ipykernel install --user --name=AstronomicalData

Fix link to and formatting for MIST Isochrone details

I moved the details of how we generated the MIST isochrone to _extras/calculating_MIST_isochrone.md but I think it needs a header to render correctly and the link to it from Lesson 6 (text: details here) doesn't work. We should fix this.

Tell learners to move student_download from download to desktop or another location

In setup, learners download the file student_download.zip. We have them upzip the file and start working with the component files, but don't tell them to move it out of their Downloads folder. We should tell them to put it somewhere else. 🙂

Finalize data files

files in student_download/data should be checked against final lesson

Plots not rendering in lesson 3

And others

Extraneous Python code block at end of lesson 3

One results to polygon_results left behind in 03-motion

One results variable was left behind in the rename to polygon_results in 03-motion.md, it is in the "Transform back" section. I can't do a PR now, but if you have a branch you are working on, please do make it and I can squash it.

Update astroquery version to 0.4.4

astroquery just released a new version which should be transparent to the lessons except that it instantiated the gaia catalog silently. We should check that students are getting this version and modify the lessons as needed (hopefully just the initial import)

Add versions to packages in environment.yml

The CAC discussed the fact that we've had instances of package updates that break this lesson. We recommend that the maintainers add version numbers to the environment.yml file that are updated (after testing is done on the new versions) periodically (quarterly? semi-annually?)

remove convex hull reference

Episode 8 has a reference to Convex Hull in Adding the Shaded Region which should be removed

Update Query Result numbers in Various Exercise Solutions

@yotam-stsci and I noticed some of the solutions have out of date numbers for the query results - will review and report here what the updated values are and can edit to fix after our workshop is over today.

Reintroduce POLYGON for proper motions, requires new operator POLYGON_PROPER_MOTIONS

The original Transform and Select chapter contained a beautiful application of the convex hull algorithm from Scipy.
This was used to derive a polygon which could be included in an ADQL query to select stars based on their proper motions.
However, POLYGON from ADQL is not meant to select based on proper motions, it should only be applied to positions on the sky.

Therefore it has recently been replaced by two BETWEEN operators

POLYGON for proper motions will still work in our case, with this trick

AND 1 = CONTAINS(POINT(-pmra, -pmdec),
                   POLYGON({negative_pm_point_list})) 

But that will not work in all cases, as @abostroem pointed out, so is not recommended.
At this point the two BETWEEN operators are the best we can do, it seems.

But I would like to reintroduce the convex hull and more elegant selection of stars based on proper motions as soon as something like a POLYGON_PROPER_MOTIONS operator has been built.

Hopefully VO people will build such an operator in the near future.

@eteq, @abostroem mentioned your name in the Slack channel, could you please keep us updated on any developments with respect to this matter?

modify variable in plot_cmd

The plot_cmd function in episode 7 takes table as input but we actually are always passing it a data frame so the variable should be something like df

Make sure figures match code

We should make sure that the figures we show match the code that students are typing

Episode 6: Join - issue with "starting from this episode"

When I was in episode 6 (Join) I got a null result when trying to run candidate_coord_pm_job = Gaia.launch_job_async(query=candidate_coord_pm_query).

We figured out that in the "Starting from this episode" section, the problem was with the initialization of sky_point_list and pm_point_list. They were set to point_series['sky_point_list'][0] and point_series['pm_point_list'][0] respectively, but each just gave the first character in the string instead of the full string.

Thus if you remove the [0] the later query could run.

Original initilization block

filename = 'gd1_data.hdf'
point_series = pd.read_hdf(filename, 'point_series')
sky_point_list = point_series['sky_point_list'][0]
pm_point_list = point_series['pm_point_list'][0]
point_series

If you do not modify the code then sky_point_list will be '1' and pm_point_list will be ' '.

Modified block

filename = 'gd1_data.hdf'
point_series = pd.read_hdf(filename, 'point_series')
sky_point_list = point_series['sky_point_list']
pm_point_list = point_series['pm_point_list']
point_series

If you remove the [0], then sky_point_list will produce '135.306, 8.39862, 126.51, 13.4449, 163.017, 54.2424, 172.933, 46.4726, 135.306, 8.39862' and pm_point_list will produce ' -4.05037121,-14.75623261, -3.41981085 ... -4.42442296,-14.74641176' which is what we want.

Section on Accessibility in Visualizations Episode

While the summary of episode 8 mentions that one should keep accessibility in mind it doesn't go into depth on what that means specifically. It would be great to add a section about this that covers issues such as colorblindness, being thoughtful regarding image size and annotation size, considering how things will look in print vs. on various electronic devices, etc.

Are there any resources that we can link to regarding this as well? Article or paper?

Incorrect plot image in Episode 7

Small issue - the image included as the example for the last plot in episode 7 near the bottom is the wrong one (the axis labels are incorrect, but the data looks fine). In this section, the plot has ra/dec instead of the phi labels.

Episode 1, Exercise 2 needs complete solution

The exercise:
Read the documentation of this table and choose a column that looks interesting to you. Add the column name to the query and run it again. What are the units of the column you selected? What is its data type?

Contains the query but not how to run it to get the results. We should add:

job1_with_rv = Gaia.launch_job(query1_with_rv)
results1_with_rv = job1_with_rv.get_results()
results1_with_rv

Update teaching time for episode 8: Visualization

In the episode 8: Visualization
https://datacarpentry.org/astronomy-python/08-plot/index.html
Both teaching time and exercises are listed as 65 minutes each. However, the episode is mostly exercises. Combinedly, it is might take less than 75 minutes. Should we update teaching time to may be something like 10 minutes?

Add intro text to episode 1

Somehow in the conversion from notebook to lesson template, the introduction to the workshop got lost. Here is text written by @eteq

GD-1 is a stellar stream around the Milky Way. This means it is a collection of stars that we believe was once part of a bound clump, but the gravitational influence of the Milky Way has torn it apparent and spread it over an arc that traces out its orbit on the sky.
This is interesting, because if the original bound clump was a dwarf galaxy, understanding its orbit with sufficient precision allows us to measure the mass of the Milky Way, which is very important for understanding the future and pass of the Milky Way as a whole. But that is much easier to do if we have a coordinate system aligned with the stream because that makes fitting the location of the stars much easier mathematically - it's a line instead of some weird funky curve.
Additionally, this stream is especially interesting because it has "gaps", which have a natural interpretation as being caused by the influence of small clumps of dark matter passing near the stream. And knowing the typical rate of these gaps tells you about the typical size and density of these clumps, which turns out to be one of the best probes we have of the fine structure of dark matter.

Style exercise should have solution

In episode 8 the style exercise should have a solution

plt.style.use('seaborn-bright')
plot_cmd(candidate_df)

plt.plot(left_color, g, label='left color')
plt.plot(right_color, g, label='right color')

plt.legend();

rcParams Exercise should have solution

In episode 8, rcParams Exercise should have solution

Episode 1: first exercise solution should actually answer the question

Exercise: One of the other tables we will use is gaiadr2.panstarrs1_original_valid. Use load_table to get the metadata for this table. How many columns are there and what are their names?

We should add:
There are 26 columns and their names are:
obj_name
obj_id
ra
dec
ra_error
dec_error
epoch_mean
g_mean_psf_mag
g_mean_psf_mag_error
g_flags
r_mean_psf_mag
r_mean_psf_mag_error
r_flags
i_mean_psf_mag
i_mean_psf_mag_error
i_flags
z_mean_psf_mag
z_mean_psf_mag_error
z_flags
y_mean_psf_mag
y_mean_psf_mag_error
y_flags
n_detections
zone_id
obj_info_flag
quality_flag

AUTHORS page unpopulated

The AUTHORS page linked from the README is unpopulated: https://github.com/datacarpentry/astronomy-python/blob/gh-pages/AUTHORS

Episode 2: Inconsistent variable name in "Selecting a region" section

Dear maintainers,

In Episode 2: Selecting a region, I found an inconsistency in the used variable name cone_query. The first code block defines cone_query:

cone_query = """SELECT 
TOP 10 
source_id
FROM gaiadr2.gaia_source
WHERE 1=CONTAINS(
  POINT(ra, dec),
  CIRCLE(88.8, 7.4, 0.08333333))
"""

but in the next code block, this variable is suddenly called query_cone in the Gaia.launch_job() call:

from astroquery.gaia import Gaia

cone_job = Gaia.launch_job(query_cone)
cone_job

Copying this code into a notebook obviously results in an error, so could you correct this by renaming one of the two to make them consistent?

Many thanks and cheers, Jelle.

Update episode_function.py

• Check episode numbers
• update plot_cmd to match lessons (not use invert_yaxis)

Move isochrone to discussion?

Right now there is an isochrone.md file, but it doesn't show up on the rendered website because it is lacking a page title (example below).

---
title: Prerequisites
---

Two options are:

1. Add a page title to that .md file. It will then appear as a separate page within the Extras drop-down menu.
2. Move the contents of that .md file to discussion.md (which is currently empty).

Remove gala from test_notebook (and maybe environment)

We removed the lesson dependencies on gala - we don't need to test it in the setup