baidakovil / inat-statistics

Utilities for working with data of iNaturalist.org. Particularly used in my project «Tsyurupy and its forests» 🌳

iNat-statistics

It is a common place for utilities to work with statisctics of iNaturalist.org data.
Now it consists of two scripts.

Table of contents

• Util №1: iNat-changes
• Util №2: iNat-treasures
• Built with
• Contributing

iNat-changes

This utility catch user activity changes in your iNat project or in list of any iNat observations. It creates html table, ready to publish to iNat journal post.
How it does actually look you can see in one of my own regular posts about half-year project changes.

Please wait up to 60 sec till page at inat-changes.onrender.com will be loaded. It is free hosted.

iNat-changes is a Flask online-app with core script inat_changes.py.

Features

• Specifying first and second date to see changes. It means, you can export observations for the whole period, but look for changes in interesting (current) period:
  

• List of most active users. Sorted by next order: Total count of observations -> Research grade count observations -> Most recent observations -> Username

• Observations count increase: both total or research. Look at 3rd, 4th column in first line:

• Position increase: above you can see, for current period user budetinetakoe became first after being twentieth. Position decrease is not shown

• «New user» marks: user anastasiiamerkulova above first appeared in current period

• Season of user's last observation: in forth column

• Specifying number of lines in the final table

• Links to observations: if you provide project name, app will generate links to see observations under every value in 3rd, 4th columns

• Ready-to-paste for iNat journal. Just save html file and open it in text editor. All the text can be pasted to post

Disadvantages — ideas to development

• Russian language of headers and seasons without choice

• No statistics about observations: current period / total users / total observations

iNat-treasures

inat_treasures.py

This utility that automatically load info about geographical distribution of the species and prepares result in handy table. All results of request to API are saved to text file for automatic reusing.

It work in two modes: rarest species and most popular species. These are almost same tables, but sorted in opposite order (the difference is: most popular consist only species AND subspecies, but no any other taxon levels: genus, kingdom and etc; rarest table consist all taxon levels).

Example of the most popular table from my post:

This list sorted from most popular to rarest species in radius of 200 km around of my small town. Take a look at first column, it show "position by rarity"

Explanation

• For each of modes script will prepare four tables:

  • 20 km
  • 200 km
  • 2000 km,
  • 0 km,

  i.e. eight tables in total. This numbers is the radiuses of the circle with center at some inputed geographical point. This point can be, for example, your home or your city main square. 0 km is whole world, i.e. no radius at all (yes, this number should be infinity or at least diameter of Earth, but zero is simpler to type!)

• Each of tables will include only the species that present in given csv file. So what app do? It look, which of these species are rarest or most popular in this circles.

  If your csv contains 1000 observations and 500 taxons, app will make this count of API requests:

  500 taxons x 4 radiuses x 2 dates = 4000 requests

  Each request will give information about how much observations of given taxon exist in given area at given date

• It includes 2 dates, because app can display how much new observations we have in the circles, i.e. dynamics of observations. If you're not interested in dinamics, just specify first date as first observation date

• Table will consist both research grade and non-research observations. Filter it by yourself before import if needed.

Features

• Display the number of research grade observations in the given list.

  Look at example: rarest table from my half-year report, in second column.

  

• Number of new observations of given species in that region. For example, for a given period of time (half of a year) there is only one new obs of Alchemilla conclobata in the whole world

• «New observation» marks: for example, we can see there was no any Alchemilla conclobata observations in project before this period of time thanks to NEW mark. By the way, in that particular case (NEW and ↑1)we can say that for given period of a time the single obs of this species is observation in our list.

• Taxon levels for all non-species taxons: genus, subspecies, and etc. You can see this in second line of table above

• Latin names and common names where it possible (some taxons have no common names)

• Iconic taxon icons between second and third columns. Iconic taxons are groups of organisms that iNaturalist define by yourself: birds, amhibians... You can see these icons in filter dialog on iNat

• Database of loaded data. Loaded date saves to csv with fields: taxon_id, radius, date, count. For example, if you monitor changes every month, you need no load info about observations counts for last month — they are already load and saved month ago and will be loaded automatically.
  Notice: database not saves geographical points that regards to counts, so you have to manage this on your own. It is subject of TODO (you can make pull request too).

• Human-readable format of counts (K for thousands, M for millions).

Disadvantages — ideas to development

• REALLY GOOD IDEA to end life of this script by

  • using pipeline where all info about all the observations in R20 and R200 areas are loaded into database. This requires multiple CSV by-hand requests from, iNat but gives much more flexibility
  • R2000 and R0 areas are used current pipeline, but HTML data prepared with EJS or another program logic, not replace

• It is really bad written code. It is awful and almost can't be debugged or updated

• Russian language in headings without choice

• No totals, no notes on current periods

• No variables of radiuses, coordinates in code

• Plain database without coordinates column

  P.S. Despite this, it works.

Built with

Online-services

iNaturalist.org - Online social network of people sharing biodiversity information since 2008 | MIT

Software

Python - Language to work quickly and integrate systems more effectively since 1991**|** GPL compatible

• For inat-changes:
  Flask - Web framework written in Python since 2004 | BSD
  Werkzeug - Utility library for Web Server Gateway Interface applications since 2007 | BSD
  Gunicorn - Python WSGI HTTP Server ported from Ruby’s Unicorn project since 2010 | Apache 2

• For inat-treasures:
  Requests - HTTP library for Python allowing to send HTTP requests since 2011 | Apache 2

Hosting service

Render - cloud platform to build and run apps and websites with autodeploy from GitHub | proprietary

Note: As a designer and programmer, I have nothing to do with any of these services
or softwares. As a user of the iNaturalist, I have a positive opinion about iNaturalist.

Contributing

If you wish to make code contributions to the project, you are welcome! Make pull requests, open issues, write me on iNat with ideas. This project certainly have it's future!

🪡🦄🦋🐌🐞🐜🐢🦂🕸🕷🦗🦟🪳🪲🪰🐡🐠🐏🐫🐖🐈🐀🎄🌵🍂🐢🦂🕸🕷💐🪨🐚🌾🌷🌳🐠🐏🐫🐖🍄🍄🍄🍄🍄🍄🦩