The aus_gbif_clean from traitecoevo

aus_gbif_clean's People

Watchers

aus_gbif_clean's Issues

Flag extreme outliers based on distance

When there's a 1-2 points a long away from the rest of the points, these are likely candidates for bad data. could be mis-id, or horticultural planting.

Here's an attempt to id outliers, based on distance from the centre of the centroid of points for that species.

Calculate the centroid used median of lat and long
Calculate the distance of all points from the centroid
Estimate a reasonable edge of distance, e.g. as 99% quintal of distances measures
Anything > 2 * this edge distance can be consider an outlier.

distance <- function(decimalLongitude, decimalLatitude) {
    sqrt(
        (decimalLongitude - median(decimalLongitude))^2 + 
            (decimalLatitude - median(decimalLatitude))^2
        )
}

is_outlier <- function(dist, f=2) {
    edge <- f*quantile(dist, probs=0.99)[1]
    dist > edge
}

library(tidyverse)

data <- read_csv("processed_data/filt_aus_limited_columns.csv")

map_australia <- map_data("world") %>% 
    filter(region == "Australia") %>% 
    filter(lat > -45 | long < 155)

for(sp in unique(data$species)) {

    data_sp <- 
        data %>% filter(species == sp) %>% 
        mutate(
            dist_center = distance(decimalLongitude, decimalLatitude),
            outlier = is_outlier(dist_center)
            )

    p1 <- 
        ggplot() + 
        geom_polygon(data = map_australia, mapping = aes(x = long, y = lat, group = group), fill = "white", colour = "black") +
        geom_point(data = data_sp, aes(x = decimalLongitude, y = decimalLatitude, col=outlier)) + coord_fixed() + 
        labs(title=sp)
    
    ggsave(sprintf("output2/%s.png", sp), p1, dpi=150)
}

Some example plots below, and many more here