YOLOv5 was published just right after YOLOv4 has been released. While it seems great how fast our technology is progressing – is this even possible? Is YOLOv5 really comparable to previous versions or is it different? In this article we will review the current controversy about YOLOv5. Particularly, you will read about:
- Who created YOLO?
- YOLOv5: What happened?
- Literary support of YOLO versions
- Misleading information about YOLOv5
- Is the name YOLOv5 accurate?
Who Created YOLO?
The versions of YOLOv1 to YOLOv3 were created by then-graduate student Joseph Redmon and advisor Ali Farhadi, while YOLOv4 was introduced by Alexey Bochkovskiy, who continued the legacy since Redmon stopped computer vision research due to ethical concerns. Redmon claimed in a tweet that although he “loved the work, the military applications and privacy concerns eventually became impossible to ignore”.
YOLO up to YOLOv3 are considered to be large improvements of one another and have vastly different features, with each one being a significant improvement from the last. Joseph Redmon describes the efficiency and architecture of YOLO in his research papers, which provide legitimacy to its use and distinction.
YOLOv5: What Is Different?
YOLOv5 was released by a company called Ultralytics in 2020. It was published in a GitHub repository by Glenn Jocher, Founder & CEO at Ultralytics, and quickly gained traction soon after it’s publishing. It was also published on the iOS App Store under the app name “iDetection” and “”Ultralytics LLC”. While the Ultralytics page on GitHub supposedly said to be the state of the art among all known YOLO implementations, the version’s legitimacy has remained questionable in the community.
YOLOv5: Limited Literary Support
The image below is taken from Bochkovskiy’s YOLOv4 research paper. In the paper, he compares the efficiency of YOLOv4 with YOLOv3 and other common object detection algorithms. The accuracy is measured in terms of FPS and Average Precision (AP). YOLO versions 1-3 were discussed in a similar way in Redmon’s papers.
This kind of accuracy review is necessary for the community to understand the comparative efficiency of YOLO versus other algorithms. YOLOv5 has not been discussed in a formal research paper yet.
When Ultralytics was asked about a potential research paper, Glenn Jocher replied in a GitHub thread in June 2020:
We are extremely limited in resources at the moment, and our priority is architecture and training experimentation to mature on a more stable set of both
He also stated that Ultralytics has been to submit to the research paper platform arXiv by the end of the year. A paper has not been published as of today, but may be underway.
The community has instead been resorting to third-party articles to gain information on YOLOv5’s performance. This causes issues because some articles may not be entirely credible since the authors did not create the architecture, and therefore do not know it as well as the original creators would.
Roboflow YOLOv5 Article Controversy
YOLOv5 was incorrectly discussed by Roboflow, who have thus published another article correcting their mistake. In the original article “YOLOv5 is Here: State-of-the-Art Object Detection at 140 FPS”, multiple facts were misconstrued.
As there aren’t many articles out there on YOLOv5’s performance, this article was looked at by thousands of people and relied on for correct information. However, because it’s not a published paper, there was a margin of error on the calculations found in the article. This caused mass confusion among people wanting to use YOLOv5 as it came out and stained YOLOv5’s credibility.
Alexey Bochkovskiy, the creator of YOLOv4, pointed out that the article had invalid comparison results. He explained in a GitHub thread that the latency shouldn’t be measured with a batch size of 32 but rather with a batch equal to 1. Latency is the time of a complete data processing cycle, and it cannot be less than processing a whole batch, which can take up to 1 second depending on batch-size. The higher batch — the higher latency, so the real latency of YOLOv5 can be up to 1 second.
His other critiques were that the Roboflow article compared the speed of much smaller and less accurate versions of ultralytics-YOLOv5 with “very accurate and big YOLOv4”. He said that “they [Roboflow] did not provide the most critical details for comparison: what exactly YOLOv5 version was used s,l,x,… what training and testing resolutions were used, and what test batch was used for both YOLOv4 vs ultralytics-YOLOv5.” Roboflow also did not test it on the generally accepted Microsoft COCO dataset, with exactly the same settings, and they did not test it on the Microsoft COCO CodaLab-evaluation server, to “reduce the likelihood of manipulation”.
Because Bochkovskiy and others are not the original authors of YOLO, the names of YOLO subsequent versions have been contested and remain controversial.
In versions 4 and 5, Redmon’s novel feature extraction, Darknet-53, is heavily altered and as is much of the YOLOv3 architecture in general. The use of YOLOv5 as Ultralytics’ model name has also been frowned upon in the Computer Vision community. YOLOv4 has been recognized by the general community as a valid improvement over YOLOv3, and the name is not so controversial. YOLOv5, on the other hand, has unvalidated data backing its improvement over YOLOv4.
The editors of the original Roboflow article, Joseph Nelson and Jacob Solawetz, eventually published an article called “Responding to the Controversy about YOLOv5” where they address the issues raised by Bochkovskiy on the GitHub thread. They begin the article by saying they “appreciate the machine learning community’s feedback, and are publishing additional details on our methodology.” Roboflow, however, although the publishers of misinformation around YOLOv5, are not the publishers of YOLOv5 itself, so the name YOLOv5 remains unchanged within their article as it is at the discretion of Glenn Jocher (founder of Ultralytics) to acknowledge that they may need to change YOLOv5’s name to something more unique.
If you enjoyed reading this article, we suggest having a look at: