knutoplandmoen / tic-tac-toe-computervision-robot

An autonomous Tic-Tac-Toe-playing robot powered by Arduino, programmed using C++ and python. Equipped with a camera, the robot captures real-time game data, processing it using the 'minimax algorithm' to calculate the optimal move. Once determined, the robot activates its robotic arm, maneuvering the designated game piece to the desired position.

Tic-Tac-Toe Computervision-robot

Introduction

A tic-tac-toe robot, that is fully autonomous. The robot sees via its camera, "thinks" by the help of the minimax algorithm and moves with the help of a servo motor, arduino based arm.

the projects ecosystem is then based on the information flow from camera -> computer -> arduino:

The minimax algorithm and some of the game functions is based on Cledersonbc's minimax algorithm for tic-tac-toe. too get a more in depth explenation about this see: https://github.com/Cledersonbc/tic-tac-toe-minimax

A brief program explenation

The camera is the only sensor used in this project. The image taken by the camera does not provide much information, but it is passed on to Python where the data in the image is analyzed and used to, for example, see that a chip has been moved. In Python, the camera information is used in two functions: seeboard(minutes) and playermove(board). in both of these functions, the cv2 library is used to find the color blue or red in the image. In playermove(board) this is used to find where a new move has been placed. Furthermore, in the playermove(board) function, the mediapipe library is also used to see hands in the image, to then stop search for the color red or blue. In seeboard(minutes) the cv2 library is used, to then draw an outline of found pieces to create a visual image of what the robot sees:

After the program has detected the pieces, found out the game state and used the minimax algorithm to calculate the next move, it sends a signal to the arduino. Since there are nine routes and five places the robot arm can pick up the pieces, there are 45 different values that can be sent. The program on the arduino converts this signal into two values, a value for which slot it should put the chip in and a value for where it should pick up the chip:

In the arduino file, we first translate the value we get from python (incomingByte) into two values, dataval is route and chip is where it should pick up the chip. Example if incomingByte = 'a' then the robot will pick up the chip in position 1 since chip = 1 and put it in position 1 since dataval = 1. to see how this is converted in python program see line 638 in functions.py file:

Arduino circuit:

pictures

the drawn vision of the finished robot from the start of the semester and finished robot: