nicomandel / asdc_mwe

minimal working example of how to run code on asdc

Trying to get things to work

1. on asdc server infrastructure, for development users
2. for deployment users.

Main Structure

There is code provided for two types of users:

development users

are users who want to run a server infrastructure for further development and usage of models for research purposes

1. shell scripts are provided for wrapping yolov5 code to run multiple training and testing instances in:
   1. train_test_model.sh trains and tests multiple model combinations on multiple datasets
   2. test_model.sh only tests a single provided model on a range of specified datasets
2. Jupyter notebooks for running inference with the SAHI package and previously developed models:
   1. inference_for_yolov5 is a notebook which is dynamically updated from inference_for_yolov5.py. This is the basic notebook for running inference with SAHI and a previously developed model, which will only output visualisations
   2. inference_single_image is a notebook which is dynamically updated from inference_single_image.py which, despite its name, inferences each image individually and is thus capable of outputting label files!

deployment users

are users who want to run inference on their local structure and want to process folders of large images. The scripts have been developed according to the workflow documented in Issue#3. The general structure is that a site is split into multiple flights

1. process_flight.py is a script to run a single flight.
   1. provides command line utilities, so that it need only be run from the command line via python process_flight.py -i <input_folder> [OPTIONS]
   2. it contains the function run_single_flight, which runs a single flight directory and is in shared use with the second script
   3. includes (rudimentary) logging to skip images that have previously been processed
2. process_site.py is a script to run multiple flights from a single site.
   1. provides command line utilities, so that it need only be run from the command line via python process_site.py -i <input_folder> [OPTIONS]
   2. Looks for subdirectories containining the word flight (without spell-checking...) and will run a run_single_flight from process_flight for each of them.
   3. Includes (rudimentary) logging to skip already processed flights.
3. A config file provides configurations for visualisations, such as minimum IoU and bounding box thickness.

Shared code

Code which is of importance to both types of users is provided in files post-fixed with _utils.py, at the current stage:

• log_utils.py for utilities pertaining to writing logs
• model_utils.py for utilities concerning model loading, file format handling, processing and visualisation
• file_utils.py for utilities concerning file handling, such as loading folders and listing files

Default folder structure

.pt files are kept in the config subfolder. Default data directories should be stored as subdirectories in the data folder

Installation order

1. install miniconda (see online instructions)
2. install conda environment through conda env install -f environment.yml
3. install pip into conda environment through conda install pip
4. install rest of requirements through pip pip install -r pipreq.txt --default-timeout=100

asdc_mwe's Issues

Remove "Performing prediction on 77 number of slices." printout

Any chance we can remove the "Performing prediction on 77 number of slices." for each image? It just clutters up the command window when running 30,000 images. I looked around but couldn't find where the print statement is to comment out?

Line width of bounding box

Line width to 15px please

Work Flow

I'll explain the folder structure we use, then explain how I've handled it. Short answer off the bat, yes we definitely need it to accept ARW files, but also have a flag we can edit in the script to accept JPEGS as well (or make it automatically figure what file type it's taking in I suppose).

For each site we fly we create a site folder in the hard drive with the date it was flown then the site name, lets use "2023.12.21 - Fly Site Alpha". (As we are dealing with "normal people" there is always going to be spaces in the name strings). Into this site folder is a folder for each flight which contains the ARW files. These folders are generally named "Flight 1, Flight 2 etc... There is also another folder in there with the tracklog and camera log files usually called "tracklogs". (no images in this folder so need to handle that exception)

So when I want to run a site through my algorithm. I start the script which then opens a GUI and asks me if I want to process just 1 flight, or an entire site, assuming you click "entire site" it will ask if you want to process it from the first flight or from a particular site. This is due to the fact it can get interrupted and you dont want to have to process the entire site again, just to do the last 2 flights. It will then ask for the folder in a GUI folder picker i.e the root site folder (if you choose the entire site) or the flight folder you want to start on (if you choose a particular site to start from).

Once it starts running it makes a new folder in the site folder called "Detections - Fly Site Alpha" and into this folder makes a folder for each flight with the detections. All flights have to be separated into individual folders as we can run over the 10,000 number naming convention of the image files and so you end up with multiple ARW12324 files, if these aren't kept separate in different folders, you won't know which image was for which flight and so won't be able to accurately geotag.

I made it create and update a txt file in the site folder listing where it got up to and some stats on time taken, number of detections found etc.. This can be useful when you come back in the morning, the laptop has been shutdown or done an update or found an error, and exited the script.

So once it's run, you are just left with detection images (with the squares drawn on but no north arrow at this stage) in the correct flight folder in the detections folder, there are no other files created or stored as we dont need the converted JPEGS or anything, these will just take up extra space.

After that, you look through the detection images and use the geotag script to geotag any real detections, which we spoke about already. I used a GUI for that as well as it can sometimes be only 1 image image, so starting up a spreadsheet, saving it, pointing a script to it etc.. all for one image isn't usually worth it. The camera logs we are using now will be far easy to geotag images with and make an output file as they already have the image name, lat/long and heading (I used to have to work it out over the past/future seconds by looking through the lat/longs and working out direction of travel from change in position!).

As we are going to be running this on 3 laptops, mainy unattended, we need it to batch process an entire site, rather than 1 flight (folder) at a time. We also need to have as few stages as possible. Usually we can only run it overnight when the laptops are not being used (they are transferring and backing up data most of each day). I'm sure you can see how for this quantity of data, moving scripts around into the correct folders, running it by command line, editing scripts to point them to the correct folders and having multiple steps will be a nightmare. It also wouldn't be p[ossible to just have a processing folder with it all set up that we copy and paste the images into, as it can take hours to copy 10,000 images.

Also need to add, in case it makes a difference, all the images are on external hard drives, so it needs to be happy talking to external drives.

ARW to JPEG conversion script - you can ignore the bilatreral filter part and args, you wont need them

from PIL import Image, ImageDraw
from easygui import *
import os
import rawpy
import sys
import cv2

# get the desktop path of computer
# desktopPath = os.path.join(os.environ["HOMEPATH"], "Desktop")
pixelLogRootFolder = "C:\\Users\\littl\\Desktop\\OHW Testing Folder\\1cm Test Data\\Blurred Test Folder"
os.chdir(pixelLogRootFolder)

# Photo file picker GUI
PhotoName = fileopenbox(msg="Choose the Image ARW file", title=None)
PhotoNameStripped = PhotoName.rsplit('\\', 1)[-1]

#PhotoName = "C:\\OHW Testing Folder\\1cm Test Data\\Working On Folder\\DSC00829.ARW"
#PhotoNameStripped = PhotoName.rsplit('\\', 1)[-1]

raw = rawpy.imread(PhotoName)
rawRgbnpArrayPreFilter = raw.postprocess(use_camera_wb=True, output_bps=8)

# Apply Blur
# Bilateral filter arg - source, diameter, sigmaColour, sigmaSpace
# sigmaColor – A larger value means that farther colors within the pixel neighborhood will be mixed together
# resulting in larger areas of semi-equal color.
# sigmaSpace – A larger value of the parameter means that farther pixels will influence each other as long as their colors are close enough (see sigmaColor).
# When diameter>0, it specifies the neighborhood size regardless of sigmaSpace. Otherwise, d is proportional to sigmaSpace .
diameter = 0  # 0
sigCol = 25  # 75
sigSpace = 1  # 1

rawRgbnpArray = cv2.bilateralFilter(rawRgbnpArrayPreFilter, diameter, sigCol, sigSpace)

# Save blurred image for testing
blurredimage = Image.fromarray(rawRgbnpArray)
blurredimage.save('AA Blurred - ' + str(diameter) + ", " + str(sigCol) + ", " + str(sigSpace) + ", " + str(PhotoNameStripped) + '.jpg', quality=100, optimize=False)

Box Colours

Responding to:
Box Colours Makes sense. Since this is a scaling based on single input value, we can modify this to be anything. e.g. different shading of blue, see this line. Just let me know what you need

I think just one solid colour is best as shades might mean a lighter shades box gets missed? I used the following when I was testing confidence levels and these colours all stood out well in the terrain we are flying over. If possible:

• High confidence - Black
• Med confidence - Blue
• Low confidence - Yellow

If that's too hard, shading just blue will be fine.

Bounding Boxes

The bounding boxes are way too small and difficult to see. I found blue was the easiest colour with the width of the line 15px and sitting about 70 px of the flower. See attached image showing an output from mine. This can group over and cover flowers, but it looks like the bounding box automatically re-sizes around clumps, so probably less of an issue. If it does obscure flowers, we can go to the original image to see what they were. The boxes need to be very obvious as if there are multiple detections, in different parts of the image, you dont want to miss any.

Bounding box padding

Responding to:
Padding: Yes that's the lines. There needs to be some check on image boundaries though (so a rectangle in a 9568 x 6376 image cannot go beyond those values). Not too cumbersome, just sth to be aware of for me.

Padding to 70px in all directions please. This beautifully shows the difference between a pro (you) and amature (me). It took me hours to figure out the bug when my bounding box was exceeding the limit of the image and trying to draw a line there and throwing an error. You immediately picked up on that potential edge case.

Speed tests - 20/12/2023

Speed Tests
Laptop 1 (HS Laptop) GTX1080 - Torch wasn't found, so I used 'pip install torch', then re-ran and the result was False :-( . Doing some research this GPU isn't new enough and so will never work fast. I haven't run a test on this one yet.
Laptop 2 (NPWS Laptop) RTX 2070 - True. But this is the laptop I ran the test on! I just restarted the laptop and re-ran and took 1 hour, 39 mins 42 secs to do 528 images, so 11.33s per iteration. I ran these on JPEGS that were converted form the ARW's and were around 20-30mbs in size.
Laptop 3 (My personal one) RTX 3080 Ti - True. Ran a test on 50 and took 5 mins 52 secs, so 7.04s per iteration. I ran these on JPEGS that were converted form the ARW's and were around 30-40mbs in size (these were converted using a higher quality than on the ones I ran on laptop 2). I did notice it had "CUDA:0" at the top though?
To run the bounding box script over the 39 detections and it took 1:10 so 1.79s per iteration.

On all 3 I checked power mode was at max and battery mode was highest. Nothing else was running at the time.

Worst case, we can be running these on some flights that have 2000 images per flight, over 12 flights, that will mean 46 hours per site! For 0.9cm work, we are generally doing 8-10 flights per day of around 1000 per flight, call it 9000. That's = 17.6 hours per site. This is on my "fast" laptop, on the NPWS one it equates to 28 hours per site. For the 70 sites this season, that will mean over 82 days of just processing time. (And these times dont include the conversion or bounding box time)