astropy / astropy-v2.0-paper

Repository for the second Astropy paper

Scope of the Paper

The goal is to produce a brief and informative paper that covers major astropy principles not mentioned in the first paper, the core v2.0 package, and infrastructure in the astropy project to support development in python.

The draft of the paper can be found here.

Notes on writing

• We don't plan on including code in this paper, but if you think you will need to include code in your section, please add it to a separate python module (.py file) and include it in this repository.
• Use \sectionname not Section, \figurename not Figure
• Use \astropypkg for the astropy package, \astropy for the astropy project, \python not Python
• If your subpackage was included in Paper I, then please just include a note on what the package does, a reference to paper I, and any new major updates to your package
• If your subpackage was not included, then please describe the subpackage on level with what was in the first paper, and highlight any major features in it. Typical length should be equivalent to one page (single-column).

Journal

The paper will be submitted to the software section of the Astrophysical Journal.

Feedback

Comments on the paper are welcome as issues. Questions and comments can also be sent to [email protected].

Rules for Authorship

The form is still open, so if you are only seeing this now but do qualify, please sign up on the form

We would like you to become a co-author if any of the following applies to you:

You have an official role in the project, as defined on http://www.astropy.org/team.html
You have contributed code to the core package
You have contributed to one of the following pieces of infrastructure:
    Astropy-helpers
    Package-template
    Ci-helpers
    Sphinx-automodapi
    astropy tutorial
    Website
You have contributed to one of the following ‘core’ affiliated packages (packages planned and developed by the Astropy core team for key functionality that may be merged into core in the future):
    Photutils
    Specutils
    Regions
    Reproject
    ccdproc

If you would like to be a co-author, please complete the form here. If the above does not apply to you but you feel that you should still be considered for co-authorship, please complete the form and your application will be reviewed.

The coordination committee has determine that the author order will be 'The Astropy Collaboration' as the first author, followed by people who have contributed significantly to the paper, in order of contribution level (or alphabetically where contribution levels are similar), and all other authors will then be listed alphabetically. A note will be included to indicate the author list and how it was determined. Final decisions about author order will be made by Kelle Cruz (on behalf of the Astropy Coordination Committee) and Steve Crawford (on behalf of the Paper II coordination committee).

astropy-v2.0-paper's Issues

Figure 4 could include a link to source code so readers could practice/recreate the image

This figure is very nice. It would be of great use to the community (and maybe citable) if the code were available to make this plot. Is it possible to link to a github page or python notebook where the code is available to recreate this figure?

I understand if this is extra/unnecessary work.

glossary

We should consider adding a glossary of terms that are not common to the non-coding astronomer

Comment from Wolfgang

This was left on the overleaf draft (which is now hopefully locked), but here was a comment from @wkerzendorf left there before I locked it:

\inlinecomment{Wolfgang K}{I would get more quickly to the point here (I think we can also cut some more if needed): Software is a crucial component of modern research. The use of software ranges from single-use scripts used by individuals to extensive frameworks supported by large institutions. Despite their different scopes they share common algorithms. One of the core goal of \astropypkg is to provide such common algorithms for \textsc{python} packages. The other is to foster a community glued by open-source principals. }All astronomical research makes use of software in some way. Astronomy as a field has thus long supported the development of software tools

merge rights

Can someone with access give me merge rights on this repository? For right now, I'd be okay with the paper II coordinators having merge rights.

\farcs vs. \arcsec

Shouldn't one use the traditional "0\farcs 2" rather than "0.2\arcsec"?

Appendix

I noticed several issues in the Appendix in the draft I read:

• There is a giant blank area between Appendix section title and the tables.
• There are no paragraphs describing the tables in the Appendix. For example, we can explain why are the tables there.
• The affiliated package listing might not be up-to-date. Maybe the tables should be updated again after some PRs at https://github.com/astropy/astropy.github.com/pulls are merged? Or are we strictly listing only packages that were accepted when astropy 2.0 came out?

model definition and evaluation, paragraph 2

I found the second paragraph of section 3.7.2, on Model.inverse, difficult to understand. An example might help here.

Set up Travis with LaTeX?

Just to make sure the paper builds :)

Also didn't someone make a service to build PDFs in CI at #pyastro16?

Sec. 3.9.4 and figure 5

• Could you say in which sense the Lupton algorithm to produce colour images is optimal?
• Figure 5 should contain axes such as Fig. 4. It is not really clear to me what this figure should show. If we want to demonstrate the colour-figure capabilities of astropy.visualization, the example is probably not very good. A set of three or four panels which shows colour composite images with different scalings offered by astropy.visualization would be better in my opinion.

Address remaining inline comments?

I noticed that there are some remaining \inlinecomment comments in main.tex. Should they be addressed before submitting the paper?

add description to affiliated packages table

Should a description of each project be added to the affiliated package tables?

Mention ReadTheDocs in Package Template section

There is already a mention of Sphinx. And I see RTD related files in https://github.com/astropy/package-template . So, maybe we should give RTD a shout-out?

Decide if more code snippets are called for

From the referee:

More generally, given the technical nature of the work being described here, I frequently found myself wanting a small code snippet that demonstrated the new/improved functionality of the packages being described. Perhaps the authors could find a way to include short code examples where appropriate, either inline in the paper body or in the appendix.

I think our philosophy is that we don't want snippets that duplicate/are separate from the docs. So how do we want to finesse this? We could point to specific URLs with these snippets where the referee asks. Or just copy-and-paste them into the paper...? It's probably a good point that if there aren't examples in the docs, we should add them, though.

Plot of contributors over time

The referee has asked to include a plot of number of reviewers over time in the inline comments:

REVIEWER: What about the number of contributors over time? I’ve seen this
plot elsewhere and it might be useful to show here.

Avoid orphaned subsection title?

Not sure how particular journal editor would be about this... Note that Section 3.5.5 title is orphaned, i.e., last sentence of the page. Of course, this might change as the paper gets more edits, however know that such a thing is possible.

Section 5.1 "summer of 2018"

In section 5.1, you write "summer of 2018". I think this is northern-hemisphere-centric, and perhaps should list a range of months, or second-semester, or mid-2018 (to give a few examples).

Re-export 'convolution-example.png' as PDF with machine readable text

From @sladen on November 30, 2017 12:12

Currently 'figures/convolution-example.png' contains pre-rendered text. This could probably be easily re-exported as PDF containing bitmaps + machine readable/renderable text.

(Unfortunately this also appears to be the only image without source code to regenerate it, which possibly explains why this has not been done already).

Copied from original issue: astropy/astropy#6917

Warning for LS periodograms

From the referee's report:

Note from statistical editor. In sec 3.11.3, a sentence can be added warning users about difficulties in interpreting the statistical significance of Lomb-Scargle periodogram peaks. This is important because unreliable interpretation is common, and warnings are not given elsewhere (e.g. NASA Exoplanet Archive Periodogram Service). A good reference is Vanderplas 2017 (https://arxiv.org/abs/1703.09824) soon to appear in the ApJ Suppl.

I don't think this is a controversial suggestion, I just wanted to make an issue for it so we don't forget because it is listed with the other "major points".

Inconsistent figure captions handling

I see a mix of these styles in the paper:

• Caption has most of the explanation of the figure. While in the main text, it is very brief like, "See Figure X."
• Caption is very brief. While in the main text, it explains the figure in detail.

If you don't think this is worth addressing, feel free to close the issue.

Why some sentences have spaces around "/"?

I see a both ways "/" is used in the paper:

1. "blah / blah" (with spaces)
2. "blah/blah" (no spaces)

Should we choose one and stick with it? Does it matter?

Add code examples

The referee says

More generally, given the technical nature of the work being described here, I frequently found myself wanting a small code snippet that demonstrated the new/improved functionality of the packages being described. Perhaps the authors could find a way to include short code examples where appropriate, either inline in the paper body or in the appendix.

We discussed this in the paper telecons at least once and decided that we don't like code snippets too much because (i) then can be found in the docs, (ii) they risk being version specific and could be almost out of date already when the paper is published, and (iii) they are not necessarily instructive unless you are an astropy user already and know the context well enough to fill in the missing lines (or you get very long examples).

As a compromise, I suggest that we pick no more than 5 examples with no more than 5 lines of code each. I think that these do not have to be self-contains and executable, e.g. say data=Table.read(''sourcelist.fits") without downloading an actual file with a persistent identifier etc.

Note that scripts for exporting code examples with syntax highlighting to pdf (to be included as a figure) exist in the repro for paper I.

Doc link: stable -> 2.0?

While reading the draft, a thought occurred to me: When this paper is out, "stable" might already be pointing to 3.0 and not 2.0. Therefore, is it possible to make a "2.0" (or "2.x") URL for the doc now and then change all the "stable" links to something like http://docs.astropy.org/en/stable/v2.0/...?

Citations in Sphinx extensions

TPR commented:

We should put the releases of sphinx-automodapi on Zenodo and cite this

Citation rendered with question marks on PDF

I see a lot of "?" in https://github.com/astropy/astropy-v2.0-paper/blob/master-pdf/main.pdf when a citation is inserted.

accuaracy of coordinate transforms

NOVAS package is mentioned in text but is not in Table 1.
I also wonder if table 1 info might be better presented as a figure.

Citation for Aitoff projection?

I, for one, have no idea what it is, so a citation would be helpful if I want to read more about it.

p.s. I found https://en.wikipedia.org/wiki/Aitoff_projection but I don't think Wikipedia links look nice on a paper.

Classes, decorators, mixin, namespace

@perrygreenfield comments:
Agreeing with Tom below, I would avoid using terms like subclass, decorators, mixin, namespace, (and perhaps even class, instead using objects consistently; this is no place to be pendantic about the distinction between classes and objects; even to the extent of adding a qualifying sentence somewhere like: "In this paper we will use object when we mean object or classes; in software there are important distinctions between the two terms, but they are not relevant in this paper."), and the like and replace them with more generic descriptions, e.g., "extended", or "tools to make defining functions that can use quantities simple to write"

Intended audience?

The intro might benefit from a statement explaining who its intended audience is (or maybe who it isn't, eg - paper is not a "beginners' guide to astropy").

The paper also assumes some familiarity with Python and/or object-oriented programming. Some bits of terminology may be obscure for those without that background - e.g., "decorator", "instantiate."

Font

Hi, I noticed a small technical issue: when I open the PDF on my Mac in the Preview or Acrobat Reader app, the text appears very un-sharp. I've never seen this with other papers. Here's a screenshot:

Does anyone else see the same thing? Is it possible to change something to get a font that nicely renders on these common PDF viewers?

Update ape13 bibliography record when it is citable

Search: ape13 and fix in bibliography.bib when this APE is accepted / there is a zenodo record for this APE.

Figure 2

Figure may not be meaningful to those who don't know what all of the various acronyms mean: include a reference in caption? Need to explain significance of the arrows coming out of and going back into a given oval?

Address inline comments

This is an issue just to address the number of inline comments provided by the referee. If a PR is open and inline comments are not addressed in that PR, then an issue should be opened for that PR.

Update affiliated package table

The "provisionally accepted" table is no longer relevant as we don't have a provisional category anymore. Also, the main table of packages should be updated as I think there are a couple of recent additions.

Normalize plot styles

Right now, the plots have different fonts and styles. We should probably just use the matplotlib default settings.

more astronomy focused Modeling introduction?

The modelling capability is new for me, so I read it with interest. Here a suggestion to make it even more interesting to readers. (Feel free to ignore though, the section is already nice.)

Perhaps it can be elaborated on why the modelling is added to astropy at all? As in, what are the astronomical drivers to have modelling in an astronomy package? Why did we not use/wrap/wrote a general modelling package instead?

From the rest of the section I can guess, e.g. that a varying-over-the-FoV-PSF-model is a typical astronomy thing, or that we want to use Quantity objects, etc. Perhaps we can put the astronomy-specific 'why' at the start of the Modeling section?

(Also, it isn't really necessary to have a subsubsection (3.7.1) called 'Overview'. That text can just be in subsection 3.7 directly I think.)

Referee comment on Package development

 - A number of times I found myself wondering how the Astropy projects 
decides whether to build  their own package or leverage something available
in the scientific Python ecosystem? On the face of it it seems like every 
decision made by the Astropy project to build a new core package only 
increases the long term cost/overhead of maintaining this functionality. For 
example, Section 3.5.2 describes `astropy.table` and notes that there's 
similarities to the pandas package and numpy arrays. The functionality 
described in `astropy.nddata` also sounds somewhat generic. Some 
discussion as to why existing community tools (such as pandas) weren't used 
would be very welcome. In addition, if the answer to this question is that these 
other tools don't support all of the functionality the Astropy community needs 
(e.g. ability to handle Quantities with units in tables) are any efforts being made 
by the Astropy community to contribute these changes to packages such 
as Pandas? 

Level of Single Model Definition and Evaluation

@hamogu commented:

I tihnk this is too technical. I know it's a complex subpackage but I had trouble understanding this section even as a regular user.

Citations in section 5.1

Citations should be updated or added when they are available for packages in section 5.1

tables

The table section could be expanded:

@adrn: Need more introduction to the table subpackage!

@perrygreenfield: Some mention that some of these are inspired by Pandas?

Spell check!

Remember to run spell check prior to submission. Do not close until last step before submission

Table 1 as matrix

@perrygreenfield commented:

Wouldn't the table be better represented as a matrices? Or perhaps as a matrix as a crude image with colors or intensities representing the size of the quantity. Is the difference between median and mean significant enough to warrant showing both? A link to the actual numbers may satisfy readers that want the exact values, but I suspect the majority don't need to see that precision.

Plot styles

Check whether the style is consistent between different plots

citations in Fitting Models to Data

@adrn commented:

Add citations to the fitters in Fitting Models to Data section of the paper

Check consistent use of astropy

Is astropy used correction through the paper? Either \astropypkg or \astropy ?

Check prior to distributing paper.

Define "open development"

The referee says

The phase 'open development' is used a few times but isn't really defined or connected to the broader open source ecosystem. What other communities are working in this way? How novel is 'open development' by their assessment? For example, other communities have adopted policies that encourage community contribution - most notable is the Node.js Foundation's contribution policies designed to promote 'Healthy Open Source' https://medium.com/the-node-js-collection/healthy-open-source-967fa8be7951 . How does the Astropy community efforts compare to this?

Starting to comment from the bottom: I have reviewed the Node.js article linked by the referee. Compared to that, astropy is actually significantly more restrictive. In Node.js anybody is given commit rights after their first non-trivial PR based on the observations that most PRs make the code better, most people know their limits and won't merge things they don't understand, and that in system (like astropy) where there are few committers per sub-package, the time those people have for reviews becomes a bottleneck. In git, if someone with commit rights messes up, there is always a way to recover, so the risk is low. That is a policy astropy might want to look into as well (@astropy/coordinators hint, hint ;-) )

Otherwise, I think our approach to development is fairly standard across the open source world. We are more open (in term of how many people can make decisions) than the Linux kernel and less than Node.js. I don't think that what we do is a novelty, in the open source world, but it is a novelty in the astronomy software world. Maybe it is enough to just spell that out?

astropy's vision and philosophy, can you include a link/reference?

You write (page 4): "to ensure they are in keeping with astropy's vision and philosophy." Is there any place that the vision and philosophy are listed, that you can reference here?

Include software versions in Table 1

Even though Table 1 is claimed to be illustrative only, it's quantitative: numbers probably depend on some specific software versions, which should be quoted.

Add more discussion of inclusivity

Given the title of this paper, I expected to see more discussion by the authors about the inclusivity of their community. An uber-skeptical reader could make the assessment that the Astropy community has a code of conduct therefore believes it's inclusive. I suspect that the community is doing much more than this though... So, has any attempt been made by the authors to quantify the inclusivity of the Astropy community? There's a growing literature in the quantitative social sciences studying open source communities that the authors might want to look into: e.g. the work of Gousios http://www.gousios.gr/publications.html and Vasilescu https://cmustrudel.github.io/publications/ come to mind. Some discussion or attempt to quantify the inclusivity is recommended.

I have two responses to this:

• We can totally write more about our inclusive practices and give some numbers. This is 100% reasonable even though I'm not exactly sure where to get numbers from (mailing lists?, Python in Astronomy, FB group?) and which axes of inclusivity to focus on (gender, ethnicity, country of origin, current country, past IDL users, etc.). Let me know if the any of the primary authors would like me to help out with this.
• Unless we have social scientists knocking on our door wanting to do this study, I am very opposed to us studying ourselves. This activity is a serious undertaking and one which should be done by a professional, not by Astronomers. Also, I think any effort to study the demographics/inclusivity, should be done across all NumFocus projects, and not just Astropy. It would be much more powerful to get cross-project comparisons to empower us to suss-out quantifiably supported "best practices".

WCSaxis figure

Issue to discuss whether to continue to include the WCSaxis figure or if the RGB figure is sufficient on its own.

Back up performance increase claim with asv results?

In Section 3, "Beyond what is mentioned below, most subpackages have seen increased performance since the release of the v0.2 package."

Is it possible to back up the claim above with a footnote that is perhaps an URL to appropriate asv results?