YOLOv5 was published just right after YOLOv4 has been released. While it seems great how fast our technology is progressing – is this even possible? Also, is YOLOv5 really comparable to previous versions, or is it different?
In this article, we will review the current controversy about YOLOv5. Particularly, you will read about:
- Who created YOLO?
- YOLOv5: What happened?
- Literary support of YOLO versions
- Misleading information about YOLOv5
- Is the name YOLOv5 accurate?
Who Created YOLO?
The versions of YOLOv1 to YOLOv3 were created by then-graduate student Joseph Redmon and advisor Ali Farhadi. However, YOLOv4 was introduced by Alexey Bochkovskiy, who continued the legacy since Redmon stopped computer vision research due to ethical concerns.
In a tweet, Redmon claimed that although he “loved the work, the military applications and privacy concerns eventually became impossible to ignore”.
If you want to read more about privacy concerns in deep learning, I recommend reading our Article about privacy-preserving deep learning.
YOLO up to YOLOv3 are considered to be large improvements of one another and have vastly different features, with each one being a significant improvement from the last. Joseph Redmon describes the efficiency and architecture of YOLO in his research papers, which provide legitimacy to its use and distinction.
YOLOv5: What Is Different?
YOLOv5 was released by a company called Ultralytics in 2020. It was published in a GitHub repository by Glenn Jocher, Founder & CEO at Ultralytics, and quickly gained traction soon after its publishing.
The YOLOv5 object detection model was also published on the iOS App Store under the app name “iDetection” and “Ultralytics LLC”. While the Ultralytics page on GitHub is supposedly said to be state of the art among all known YOLO implementations, the version’s legitimacy has remained questionable in the community.
YOLOv5: Limited Literary Support
Redmon’s YOLO versions 1-3 and Bochkovskiy’s YOLOv4 were published in peer-reviewed research papers that supported their performance and architecture use.
The image below is taken from Bochkovskiy’s YOLOv4 research paper. In the paper, he compares the efficiency of YOLOv4 with YOLOv3 and other common object detection algorithms. The accuracy is measured in terms of FPS and Average Precision (AP). YOLO versions 1-3 were discussed in a similar way in Redmon’s papers.
This kind of accuracy review is necessary for the community to understand the comparative efficiency of YOLO versus other algorithms. Unfortunately, YOLOv5 has not been discussed in a formal research paper yet.
When Ultralytics was asked about a potential research paper, Glenn Jocher replied in a GitHub thread in June 2020:
We are extremely limited in resources at the moment, and our priority is architecture and training experimentation to mature on a more stable set of both.
He also stated that Ultralytics has been to submit to the research paper platform arXiv by the end of the year. However, a paper has not been published as of today but may be underway.
The community has instead been resorting to third-party articles to gain information on YOLOv5’s performance. Consequently, this drives concerns because some articles may not be entirely credible since the authors did not create the architecture, and therefore do not know it as well as the original creators would.
Roboflow YOLOv5 Article Controversy
YOLOv5 was incorrectly discussed by Roboflow, who have thus published another article correcting their mistake. In the original article “YOLOv5 is Here: State-of-the-Art Object Detection at 140 FPS”, multiple facts were misconstrued.
As there aren’t many articles out there on YOLOv5’s performance, this article was looked at by thousands of people and relied on for correct information. However, because it’s not a published paper, there was a margin of error on the calculations found in the article. This caused mass confusion among people wanting to use YOLOv5 as it came out and stained YOLOv5’s credibility.
Alexey Bochkovskiy, the creator of YOLOv4, pointed out that the article had invalid comparison results. He explained in a GitHub thread that the latency shouldn’t be measured with a batch size of 32 but rather with a batch equal to 1. Latency is the time of a complete data processing cycle, and it cannot be less than processing a whole batch, which can take up to 1 second depending on batch size. The higher batch — the higher latency, so the real latency of YOLOv5 can be up to 1 second.
His other critiques were that the Roboflow article compared the speed of much smaller and less accurate versions of ultralytics-YOLOv5 with “very accurate and big YOLOv4”. He said that “they [Roboflow] did not provide the most critical details for comparison: what exactly YOLOv5 version was used s,l,x,… what training and testing resolutions were used, and what test batch was used for both YOLOv4 vs. ultralytics-YOLOv5”.
Moreover, Roboflow did not test it on the generally accepted Microsoft COCO dataset, with exactly the same settings. They did not test it on the Microsoft COCO CodaLab-evaluation server to “reduce the likelihood of manipulation”.
Naming YOLOv5?
Because Bochkovskiy and others are not the original authors of YOLO, the names of YOLO subsequent versions have been contested and remain controversial.
In the versions YOLOv4 and YOLOv5, Redmon’s novel feature extraction, Darknet-53, is heavily altered, and as is much of the YOLOv3 architecture in general. The use of YOLOv5 as Ultralytics’ model name has also been frowned upon in the Computer Vision community.
However, YOLOv4 has been recognized by the general community as a valid improvement over YOLOv3, and the name is not so controversial. YOLOv5, on the other hand, has unvalidated data backing its improvement over YOLOv4.
The editors of the original Roboflow article, Joseph Nelson and Jacob Solawetz, eventually published an article called “Responding to the Controversy about YOLOv5”, where they address the issues raised by Bochkovskiy on the GitHub thread. They begin the article by saying they “appreciate the machine learning community’s feedback, and are publishing additional details on our methodology.”
Roboflow, however, although the publishers of misinformation around YOLOv5, are not the publishers of YOLOv5 itself, so the name YOLOv5 remains unchanged within their article as it is at the discretion of Glenn Jocher (founder of Ultralytics) to acknowledge that they may need to change YOLOv5’s name to something more unique.
What’s Next?
If you enjoyed reading this article, we suggest having a look at:
- Real-Time Object-Detection with YOLOv3
- YOLOR – You Only Learn One Representation (What’s new, 2021)
- Computer Vision Applications in different industries
- Top Deep Learning Frameworks in 2021
- Read why Edge AI is the next big trend in AI
