jithin8mathew / tello_object_detection_demo_application

Building an Android Application to control Tello Drone and perform real-time object detection using YOLOv5

Tello drone-based object detection app for Android

This repository will walk you through android app development process for object detection using Tello Drone.

This project will demonstrate how to:

• Create a UI for controlling Tello Drone
• Handle drone using virtual controls
• Process H.264 (AVC) encoded video from Tello and display it on app
• Perform near real-time object detection from Tello video frame

Things to fix:

• 0. Video feed keeps getting stuck during flight (Temporary fix: turn the video switch at the top left of the screen on and off again). A permanent fix is needed.
• 1. I could not figure out a way to toggle between and SurfaceView and BitMap displaying for simply viewing video frames vs performing object detection
• 2. Some of the calculations like drone acceleration etc. needs more attention and correction.
• 3. Improve the efficiency of the code and simplify it.
• 4. Closing datagram sockets while not in use or going back to the main page is not working.
• 5. Capture image and record video buttons does not work at this point, but I will be adding these features soon.

How to install this application?

Open android studio> open demoApplication project > Build > run app (preferably connecting your phone)

Things to keep in mind:

• While testing on Pixel 4 (Android 12) I faced issues associated with UDP connection. After connecting to Tello's wifi, under internet> wifi> TelloNetworkName> Privacy, there are two options. 1) Use randomized MAC (default) 2) Use device MAC. After connecting to Tello netwok, swith between these options before using the app.
• The above step will disconnect from the wifi and reconnect again, only then will this app work on Pixel 4 (Android 12), this could be only a problem on my device Pixel 4 in general (I don't know it yet)
• There could be several bugs, things could be done in an easier and efficient way, possibly. Please report bugs and issues.
• If you face any issues during development or installation, please raise an issue in my GitHub page for this project

More importantly, I am excited to see cool application of this project. And finally, if you liked this project and find it interesting, please ⭐ the project so that it can reach more people. Thank you and hope you have some fun with this!.

This project is explained in detail at https://medium.com/@jithin8mathew/building-an-android-application-to-control-tello-drone-flight-and-perform-real-time-object-ab953f6c5f5b

LICENSE

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

tello_object_detection_demo_application's Issues

Video Stream doesn't work on android studio devices

Thank you for sharing your code.

When i extracted .apk and have used it(tested on android 12, 9), everything worked well! (control, video stream, object detection ... )

But when i was using any virtual devices on android studio, i couldn't see the video from the drone.
(White screen is kept displaying, I could control only)

Is there a specific device model to run code on android studio or do i have to adjust any options for devices?

If you know the difference between when actual devices are used and when android studio devices are used, please let me know why

Thank you again!

Video decode issue on API 26

Hi there,

I was working on replicating your code on my own Android Application decode the video stream and ran into an issue where there were green vertical lines with a green bar at the top. I thought this was an issue in my end, transferring the algorithm in this repo, but I cloned the application and found the exact same issue.

Note, that to run this app I had to change the minimum SDK from 27 to 26, as that is what my device was - so maybe that is what is causing both of these issues. I'm planning to see if theres anything related to MediaCodecList that might help.