wildtreetech / sentinel2-bot Goto Github PK

Sometimes the bot tweets the same image again. I have noticed/suspected it a couple of times, now here is hard evidence:
https://twitter.com/Sentinel2Bot/status/816264394502008840
https://twitter.com/Sentinel2Bot/status/814858131440336896
There are probably many more, because I must only notice it when it's a distinctive image and when it gets repeated quickly.
Is this understood?

• cycles through not enough images?
• falls back (when cloudy) to 'default' images?
• random number generator not random?

Could it be tested? eg, download all the word parts of the tweets and compare?

Hey @betatim. Nice work!

It would be nice if you were able to tag the countries (and maybe cities?) shown in the tweets. Would mean that your bot's tweets show up in tag-search results.

EDIT: I will try poking in the code to see where this needs to be added, but pointers are appreciated! :)

https://twitter.com/Sentinel2Bot/status/786480930508705792

Some images are very very dark. One way of improving on that is to use some kind of adaptive adjusting of colours/brightness.

Some images where the colours must be wrong:

https://twitter.com/Sentinel2Bot/status/937970081941852160
https://twitter.com/Sentinel2Bot/status/937929463110819841
https://twitter.com/Sentinel2Bot/status/937804008198213632
https://twitter.com/Sentinel2Bot/status/937728445110542336
https://twitter.com/Sentinel2Bot/status/937426429788217345

Currently it takes a long time/a lot of CPU to verify that the bot still works. Create tests that use mock to replace the slow running bits so that we can verify that the bot still works.

Steps:

• setup py.test
• create the simplest test possible that tests something
• test with a mock that picking a random location works
• test with a mock for downloading images
• create a test that mocks posting to twitter

Right now the bot does not delete images that it has already tweeted. Eventually this will fill up the hard drive. Teach the bot to delete unused images.

Some images, especially of sandy places seem to have a weird colour balance. is the ground in Algeria really that red/orange??

Compare (Laghouat, Algeria (33.0°N 2.4°E), 2 September 2016):

with (Tiris Zemmour, Mauritania (24.9°N 11.5°W), 21 March 2016):

Need to do better at counting characters. It is all about the unicode!

• https://dev.twitter.com/overview/api/counting-characters
• https://gist.github.com/mtigas/810514

For example 'Laâyoune-Sakia El Hamra ⵍⵄⵢⵓⵏ-ⵙⴰⵇⵢⴰ ⵍⵃⴰⵎⵔⴰ العيون-الساقية الحمراء, Western Sahara' is 81 characters but if you encode it properly: len(unicodedata.normalize("NFC", 'Laâyoune-Sakia El Hamra ⵍⵄⵢⵓⵏ-ⵙⴰⵇⵢⴰ ⵍⵃⴰⵎⵔⴰ العيون-الساقية الحمراء, Western Sahara').encode('utf-8')) it is 134 "characters"

Good yet simple algorithm for picking visually interesting pictures.

(This issue is about finding a super simple algorithm to make this decision, if you want to use something complicated like machine-learning open a new issue)

I saw the tweets because it sometimes says "Unknown location, do you know it? Tell @openstreetmap" e.g. https://twitter.com/Sentinel2Bot/status/785543415547191296
what does this mean? and what are we supposed to be telling OpenStreetMap?

...is my question. I guess the answer might be added to the README here

The roadmap. What is on the horizon for Sentinel bot?

Near term

• visually interesting images (#5)
• make it easier to explore area shown on the image (OSM link?)

Long term

• apply "bottom of atmosphere" correction, results in better images(?) (#19)
• support for other bands (sometimes post a near IR picture instead of just true colour)

Long long term

• create a partner bot that uses the images for machine-learning purpose
• use the retweets/likes to train an algorithm to determine what is "visually interesting" (related #5)

https://twitter.com/Sentinel2Bot/status/1161794634454065152?s=20
Particularly for images like this it would be cool to just zoom in and browse around

For images with a lot of clouds the ground is very dark. For example:

You could improve this by using a larger top end clip for scenes that have a lot of cloud.

As an example: the same image as above but by hand clipping a lot more at the top:

The clouds get clipped and you lose details there but you can see the ground.

To do: experiment with what a good clip value is. MGRS code for the example image: T47PNN_20190127T034021_rgb_small.jpg

cc @ozjtigger1

There is Py6S which can drive 6S to do atmospheric corrections. Step one would be to understand which parameters we need to know, how to look them up for our tiles, then compute big lookup tables for processing of images.

Py6S on github: https://github.com/robintw/py6s

Work ongoing for creating a LUT: robintw/Py6S#8