Comments (56)
@husnan622 set normalize=False in ConfusionMatrix()
from yolov5.
π Hello @husnan622, thank you for your interest in YOLOv5 π! Please visit our βοΈ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.
If this is a π Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.
If this is a custom training β Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.
For business inquiries or professional support requests please visit https://ultralytics.com or email [email protected].
Requirements
Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:
git clone https://github.com/ultralytics/yolov5 # clone
cd yolov5
pip install -r requirements.txt # install
Environments
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
- Notebooks with free GPU:
- Google Cloud Deep Learning VM. See GCP Quickstart Guide
- Amazon Deep Learning AMI. See AWS Quickstart Guide
- Docker Image. See Docker Quickstart Guide
Status
If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on MacOS, Windows, and Ubuntu every 24 hours and on every commit.
from yolov5.
@glenn-jocher I got the confusion matrix as shown,
For this matrix the TP is nil(no value), FP is 1 and TN is 0.94 what these indicates exactly and may I know how to improve the values like TP has to get some value, FP has to decrease??
I really appreciate your help. Thanks in advance..
from yolov5.
@husnan622 val.py makes confusion matrices automatically. See val.py for usage examples.
from yolov5.
@dadin852 hello! Thank you for sharing your code for generating a confusion matrix with custom labels for YOLO. It's always helpful to have different options for analyzing the performance of our models.
Regarding the differences between the results of detect.py
and test.py
, keep in mind that detect.py
performs detection on a single image or directory of images, whereas test.py
evaluates the entire validation dataset and reports metrics such as mAP. It's possible that the differences you're observing are due to variations in the images that are being processed, or other factors such as the size of the batch being used during evaluation.
If you have any further questions or issues, feel free to reach out. We're here to help!
Best regards,
Glenn Jocher
from yolov5.
@glenn-jocher I've tried what you suggested regarding val.py but I haven't been able to bring up the confusion matrix image, I don't know what's missing from the command I made
and val.py is still scanning the valid dataset even though I have modified val.py to be task=test
from yolov5.
@husnan622 check your runs/val/exp2 directory, confusion matrix is in there.
from yolov5.
@husnan622 if your data.yaml has a test: key then yes you can run python val.py --task test to use your test split.
from yolov5.
Thanks @glenn-jocher
Are the weights that I use correct?
and usually the code in the confusion matrix is contained actual data path & prediction data path for example y_true and y_pred, where can i find it
from yolov5.
@husnan622 you can use any weights you want as long as they are trained on your --data data.yaml
You can access the confusion matrix code in utils/metrics.py:
Lines 126 to 220 in 7845cea
from yolov5.
Thank you so much for your help @glenn-jocher
from yolov5.
@glenn-jocher Previously I was able to generate a confusion matrix using val.py, the results are like the following image:
But I want a confusion matrix that only displays True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN), for example in the following image:
How to generate a confusion matrix that only displays True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN)?
from yolov5.
What is the background in the confusion matrix image?
How do I remove this section?
from yolov5.
π Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.
Access additional YOLOv5 π resources:
- Wiki β https://github.com/ultralytics/yolov5/wiki
- Tutorials β https://docs.ultralytics.com/yolov5
- Docs β https://docs.ultralytics.com
Access additional Ultralytics β‘ resources:
- Ultralytics HUB β https://ultralytics.com/hub
- Vision API β https://ultralytics.com/yolov5
- About Us β https://ultralytics.com/about
- Join Our Team β https://ultralytics.com/work
- Contact Us β https://ultralytics.com/contact
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!
Thank you for your contributions to YOLOv5 π and Vision AI β!
from yolov5.
@glenn-jocher : For YOLOv5, how do I increase the font size of numbers in the confusion matrix?
from yolov5.
@syamghali to increase the font size of the numbers in the confusion matrix in YOLOv5, you can modify the plot_confusion_matrix()
function in the utils/plots.py
file. Specifically, you can change the fontsize
parameter in the heatmap
function call on line 74. The default value is 14; you can increase it to the desired size. However, please note that while increasing the font size may make the numbers in the plot more readable, it may also reduce the space available for the plot, which could make the plot less visually informative.
from yolov5.
pls. What is the background mean in the confusion matrix image? and how can I remove it?
from yolov5.
@muhanadabdul the background area in the confusion matrix image represents values that are not part of the confusion matrix itself. This area is typically used to display legends or colorbars. To remove the background area, you can modify the plot_confusion_matrix()
function in the utils/plots.py
file of YOLOv5. Specifically, you can remove the code that generates the legend or colorbar or modify the relevant parameters to adjust their size and location. However, please note that removing or modifying these sections may make the plot less informative or harder to read, especially for users who are less familiar with the specific plot.
from yolov5.
Dear, thank you for explanation, but I am afraid I do not understand what the values in the background label mean to the validation phase
the values with the red circle in the image above
from yolov5.
@muhanadabdul hello, the values you have highlighted in the image represent the number of samples that were not considered in the validation phase. These samples may have been excluded from the validation set for a variety of reasons, such as missing annotations or insufficient image quality. The values can be useful to understand the size of the validation set relative to the full dataset and to identify any potential imbalances or biases in the dataset. However, they are generally not included in the confusion matrix or other validation metrics, as they do not represent true positive or negative results. Please let me know if you have any further questions or concerns.
from yolov5.
Dear, thanks for your explanation, now it's clear, but just have another question. the two highlighted values in the image below are related to the same class "mobile_use" Based on your explanation I am confused in understanding their meaning if we let the 0.08 value represent the number of samples that were not considered in the validation phase, what the other value (0.06) mean?
Is there any way to find which images the model excluded from the validation set?
Also, can we hide these two columns (V, H) and their values from the resulting confusion matrix
from yolov5.
@muhanadabdul hello! The value of 0.08 represents the number of samples that were not included in the validation set for the "mobile_use" class. The value of 0.06 represents the percentage of samples for this class that were not included in the validation set. This percentage is calculated as the number of excluded samples (0.08) divided by the total number of samples (1.34) for this class. Regarding whether it's possible to find which images were excluded from the validation set, this depends on how the data was split and stored. If you have access to the code or procedure that was used to split the data, you may be able to identify the excluded images. If the image filenames or IDs are included in the dataset, you may also be able to cross-reference them with the validation set to identify the excluded images. Finally, regarding the question of hiding the V and H columns and their values from the confusion matrix, you can modify the plot_confusion_matrix()
function in the utils/plots.py
file of YOLOv5. Specifically, you can remove the code that generates the V and H columns, or modify the relevant parameters to adjust their size and location. Please note that modifying the confusion matrix in this way may make it less informative or harder to read, especially for users who are less familiar with the plot.
from yolov5.
Fantastic, many thanks for your dear.
from yolov5.
@muhanadabdul hello! You're very welcome. If you have any further questions or concerns, please don't hesitate to ask. We're here to help!
from yolov5.
from yolov5.
@charanlsa dear user,
Thank you for reaching out. The confusion matrix you have shared represents the performance of your YOLOv5 model on the validation set. The rows correspond to the ground truth classes, and the columns correspond to the predicted classes. Each element in the matrix represents the number of validation samples that were assigned to a specific ground truth class and predicted class. The diagonal elements represent the number of correct predictions for each class, and the off-diagonal elements represent the number of incorrect predictions.
If you are noticing a low accuracy, one possible approach to increasing it is to adjust the hyperparameters of your YOLOv5 model. Some hyperparameters you might consider tuning include the learning rate, batch size, and number of training iterations. Additionally, you may want to consider increasing the size or diversity of your training set to boost overall performance.
Regarding the specific class where you are noticing a high number of false positives (i.e., ground truth class 1 and predicted class 0), you may want to consider adjusting the class weights or focal loss coefficients to emphasize this class during training. You may also want to examine the annotations for this class carefully to ensure that they are accurate and complete.
I hope this information is helpful. Please let me know if you have any further questions or concerns, and I'll be happy to assist you. Good luck with your project!
Best regards,
Glenn Jocher
from yolov5.
plot_confusion_matrix()
Dear, I did not find the plot_confusion_matrix() function in the utils/plots.py ?
from yolov5.
plot_confusion_matrix()
Dear, I did not find the plot_confusion_matrix() function in the utils/plots.py ?
from yolov5.
@muhanadabdul hi there! I'm sorry to hear that you were not able to locate the plot_confusion_matrix()
function in the utils/plots.py
file. Just to make sure, are you using the latest version of YOLOv5? If so, the function should be included in the file. One possibility is that the function was accidentally removed or modified during your customization of the code.
If you are still unable to find the function, a workaround could be to use a third-party library like scikit-learn to plot the confusion matrix. For example, you could use the confusion_matrix()
and plot_confusion_matrix()
functions from scikit-learn to generate and visualize the confusion matrix.
Let me know if this helps, or if you have any further questions or issues.
Best regards,
Glenn Jocher
from yolov5.
dear, yes I using the latest version of YOLOv5 by using the clone code
!git clone https://github.com/ultralytics/yolov5 # clone repo
the file ATTACH here you can check it
plots.zip
I need where the code responsible for plotting the confusion_matrix image, really I don't need hird-party library,
from yolov5.
@muhanadabdul Hi there! I apologize for the confusion. It looks like the plot_confusion_matrix()
function is indeed not included in the latest version of YOLOv5. I apologize for the misinformation in my previous response.
If you would like to plot the confusion matrix without using a third-party library like scikit-learn, you will need to implement your own function to generate and plot the matrix. Here is some sample code you can use as a starting point:
import numpy as np
import matplotlib.pyplot as plt
def plot_confusion_matrix(cm, classes, normalize=False, title='Confusion matrix', cmap=plt.cm.Blues):
"""
Plots a confusion matrix.
"""
if normalize:
cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
print("Normalized confusion matrix")
else:
print('Confusion matrix, without normalization')
plt.imshow(cm, interpolation='nearest', cmap=cmap)
plt.title(title)
plt.colorbar()
tick_marks = np.arange(len(classes))
plt.xticks(tick_marks, classes, rotation=45)
plt.yticks(tick_marks, classes)
fmt = '.2f' if normalize else 'd'
thresh = cm.max() / 2.
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
plt.text(j, i, format(cm[i, j], fmt),
horizontalalignment="center",
color="white" if cm[i, j] > thresh else "black")
plt.ylabel('True label')
plt.xlabel('Predicted label')
plt.tight_layout()
plt.show()
This code defines a function plot_confusion_matrix()
that takes as input a confusion matrix, a list of class names, and other optional parameters like whether or not to normalize the matrix and what title to give the plot.
You can then generate the confusion matrix as a NumPy array using the sklearn.metrics.confusion_matrix()
function (which you mentioned you would prefer not to use), and then use the plot_confusion_matrix()
function to plot the matrix:
import itertools
# Generate confusion matrix using scikit-learn (or your preferred method)
cm = confusion_matrix(true_labels, predicted_labels)
# Define list of class names
classes = ['class_1', 'class_2',
from yolov5.
Hello, here's a code made to generate Confusion Matrix with given .txt files :
https://github.com/dadin852/YOLO-Confusion-Matrix-with-custom-labels.git
I did this because the detection result between detect.py and test.py are different.
from yolov5.
I've already set all the factors (img-size, batch-size, thres...) between detect.py and test.py οΈ the same.
And the results are different.
from yolov5.
Hi Glenn: When the YOLOv5 creates a confusion matrix, what is the default confidence threshold that is used for classification for background areas (False Positive). For, example, in the above matrix, for mobile_use background = 0.08. This means the model predict background as mobile_use in 8% of the cases., right ? And what is the default confidence threshold that is considered here, is it 0.2?
from yolov5.
hi i tried to edit metrics.py to generate confusion matrix without background, but it still generate it with background, how can i create confusion matrix without background?
from yolov5.
Hello, are you suggesting that you'd like to remove the background FP/FN labels?
My approach involves saving the matrix data in metrics.py and then editing it to create the confusion_matrix.png.
In metrics.py :
def plot(self, save_dir='', names=()):
try:
import seaborn as sn
array = self.matrix / ((self.matrix.sum(0).reshape(1, self.nc + 1) + 1E-6) if False else 1 ) # normalize
# Save the matrix data
open(f'{my_save_dir}/matrix.txt', 'a+').write(np.array2string(array, separator=','))
array[array < 0.005] = np.nan # don't annotate (would appear as 0.00)
...
from yolov5.
from yolov5.
@ryecries no worries about the delay! After editing the metrics.py
file, you don't necessarily need to run val.py
again. The metrics.py
file is responsible for generating the confusion matrix based on the validation results that have already been computed.
Once you have made the desired changes in the metrics.py
file, you can run the plot()
method to generate the confusion matrix. This can be done independently from the validation process. The matrix will be saved as matrix.txt
, and you can use this data to create your custom confusion matrix visualization.
Let me know if you have any further questions or need any assistance!
Cheers!
from yolov5.
hello. I have a question: I trained the YOLOv5 and I got the confusion matrix result (confusion matrix picture). but I want to re-create the confusion matrix picture because I want to change the font size or text in the confusion matrix picture. I always run training again to get the new confusion matrix result. are there any ways to create a new confusion matrix without training? can I use the weight result to re-create the confusion matrix picture? Thank you!
from yolov5.
@RyanTNN hi there!
It is not necessary to run the training again to create a new confusion matrix with different font size or text. You can create a new confusion matrix using the existing model weights.
The confusion matrix is generated based on the predictions and ground truth labels during the evaluation process. If you want to change the font size or text in the confusion matrix picture, you can modify the code responsible for plotting the matrix.
In the metrics.py
file, you can adjust the font size or other visualization settings within the plot()
method. You can explore the matplotlib documentation for customizing the appearance of the confusion matrix.
Once you have made the desired modifications, you can run the evaluation again using the trained model weights by running the val.py
script. This will generate a new confusion matrix with the updated visualization settings.
I hope this helps! Let me know if you have any further questions or need any clarifications.
from yolov5.
@RyanTNN hi there!
It is not necessary to run the training again to create a new confusion matrix with different font size or text. You can create a new confusion matrix using the existing model weights.
The confusion matrix is generated based on the predictions and ground truth labels during the evaluation process. If you want to change the font size or text in the confusion matrix picture, you can modify the code responsible for plotting the matrix.
In the
metrics.py
file, you can adjust the font size or other visualization settings within theplot()
method. You can explore the matplotlib documentation for customizing the appearance of the confusion matrix.Once you have made the desired modifications, you can run the evaluation again using the trained model weights by running the
val.py
script. This will generate a new confusion matrix with the updated visualization settings.I hope this helps! Let me know if you have any further questions or need any clarifications.
okay. I got it. Thank you for your time.
from yolov5.
@RyanTNN You're welcome! I'm glad I could help. If you have any more questions or need further assistance, feel free to ask. Happy coding!
from yolov5.
@husnan622 val.py makes confusion matrices automatically. See val.py for usage examples.
what is the file used for plotting confusion matrix for yolov7
from yolov5.
@glenn-jocher . could you please help me get confusion matrix and accuracy score for yolov7 on custom dataset
from yolov5.
Why my confusion matrix is not showing values? the image is the from exp folder after training.
from yolov5.
@Joshnavarma It looks like the image link you provided is broken. To better assist you, could you please provide more details or a valid link for the confusion matrix image? Thank you!
from yolov5.
please find the attached image.
Apart from this i have one qestion that my yolov7 gave 85.2 % mAP on custom dataset fo 6k images(50 epochs, batch size 16). is it a acceptable mAP score and why it is giving that big vlaue. the reason i have the question is when i train my model with 100 epochs, batchsize=32 it gave around 64%mAP. now i doubt if i did anything wrong in my yolov7 training. i have used yolov7-training.pt file for trainng my custom data. am doing masters in data nalytics and this is my final year project i want to justify why the score has that big difference.
Appreciate your help. Thanks.
from yolov5.
@Joshnavarma Thank you for sharing the confusion matrix image. It seems that the provided link is not accessible. If you could upload the image to a public image hosting service (such as Imgur or PostImage) and share the new link, I would be happy to take a look.
Regarding your question about the mAP score discrepancies, achieving 85.2% mAP on a custom dataset with 6k images after 50 epochs is certainly a notable result. However, it is essential to investigate potential factors that may have contributed to this high mAP score, such as the data distribution, augmentation techniques, and the balance of classes in your dataset. Additionally, changes in hyperparameters like batch size and the number of epochs can also impact the training process.
For the varying mAP scores obtained with different training configurations, it's important to perform a thorough investigation of any alterations made during the training process, such as changes in the dataset, data preprocessing, augmentation, and the impact of different hyperparameters.
As you're working on your final year project, understanding and justifying the impact of these factors on your results will be valuable. I recommend thoroughly reviewing your training process and experimenting with different configurations to understand the effects on performance.
If you need further assistance troubleshooting the confusion matrix issue or analyzing the differences in mAP scores, please feel free to provide additional details. I'm here to help!
from yolov5.
https://postimg.cc/XX7t1B0G
Thanks for the reply. please find the image now. Thanks.
from yolov5.
@Joshnavarma thank you for sharing the image. However, the provided link still seems to be inaccessible. Could you double-check the link or provide an alternative method to access the confusion matrix image? It's important for me to review the confusion matrix to better assist you. Thank you!
from yolov5.
Hi @glenn-jocher, how can I only show the A, B and D classes for confusion matrix?
from yolov5.
Hello @wjlim-14, it seems there's still an issue with the image link you've provided. However, to address your question about showing only specific classes (A, B, and D) in the confusion matrix:
Currently, the YOLOv5 val.py
script generates a confusion matrix for all classes present in your dataset. To display a confusion matrix for specific classes, you would need to modify the code to filter out the classes you don't want to include.
Here's a general approach you could take:
- After running
val.py
, you'll have the predictions and ground truth labels. - Filter these predictions and labels to only include the classes of interest (A, B, and D in your case).
- Generate a new confusion matrix using only the filtered predictions and labels.
This would require custom coding on your part. You can refer to the val.py
script to understand how the confusion matrix is generated and then apply the necessary filters.
If you're not comfortable with modifying the code, another workaround is to temporarily remove the data for the classes you don't want to include from your dataset and then run val.py
. This will produce a confusion matrix only for the classes that remain.
Remember to backup your data and code before making any changes, and ensure you revert any temporary dataset changes after you're done.
If you manage to get the correct image link or upload the image to a different hosting service, I'll be happy to take a look at the confusion matrix issue you're facing.
from yolov5.
Anyone please explain this confusion matrix, what actually happened here.
from yolov5.
@jahid-coder hello! Given that the image link you've shared for the confusion matrix isn't accessible, I'm unable to view the specifics of your confusion matrix directly. However, Iβll explain generally what a confusion matrix represents.
A confusion matrix is a table often used to describe the performance of a classification model on a set of test data for which the true values are known. Each row of the matrix represents the instances in a predicted class, while each column represents the instances in an actual class (or vice versa). The diagonal elements represent correct predictions, whereas off-diagonal elements are misclassifications.
If you're able to provide a working image link or more specific details about your confusion matrix, I'd be more than happy to give a more tailored explanation! π
from yolov5.
@glenn-jocher thanks for your general explanation, I want to know specifically about this share confusion matrix. Actually i want to know explanation of this confusion matrix. What happened here and how to summarize about my model from this confusion matrix graph.
from yolov5.
Hello @jahid-coder! Unfortunately, the image link you've provided for the confusion matrix doesn't seem to be accessible, so I'm unable to view and discuss the specifics of your modelβs performance.
However, in general, you can interpret a confusion matrix by observing:
- Diagonal cells: which show correct predictions where the predicted class matches the true class.
- Off-diagonal cells: which indicate misclassifications, where the rows indicate the predicted class and the columns show the true class.
To summarize your model from the confusion matrix:
- High values along the diagonal in relation to the respective row and column totals indicate good performance.
- A lot of high values off the diagonal suggest areas where your model is confused and misclassifying.
Once the image becomes accessible, Iβd be happy to provide a more specific analysis. Make sure the image is properly uploaded or consider hosting it on a reliable image hosting platform for sharing. πΌοΈπ
from yolov5.
Related Issues (20)
- scale_masks fucntion HOT 1
- cls loss HOT 1
- Problem with training for a single class HOT 4
- Issue when try to validate openvino format model HOT 3
- Is there a problem with the way I fine-tuned the YOLOv5? HOT 3
- No module named 'models' HOT 2
- Roc curve /part 2 HOT 1
- REQUIREMENTS.TXT FILE ERROR WITHIN YOLOV5 HOT 2
- Custom object detection by retaining the original classes of yolo HOT 5
- Is yolov5 sensitive to the size of defects and what structural improvements are needed to increase its sensitivity to defects? HOT 1
- Inconsistency issue with single_cls functionality and dataset class count HOT 3
- A minor query about the image channel number check using `im.shape[0] < 5` HOT 5
- Questions about mosaic and affine transformation data augmentation. HOT 6
- Does YOLO perform object detection on jp2 image format? HOT 2
- Parameter performance indicators HOT 5
- How to reduce the size of best.pt HOT 2
- Confusion Matrix HOT 4
- π Feature Request: Simplified Method for Changing Label Names in YOLOv5 Model HOT 2
- where is yolov5 v7.0 --trian in export.py? HOT 2
- MESSES MY SYSTEM HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yolov5.