Introduction

Object detection and instance segmentation are computer vision tasks that involve identifying and locating objects in images or videos. It is an important part of many applications, such as self-driving cars, robotics, and video surveillance.

Over the years, many methods and algorithms have been developed to find objects in images and their positions. The best quality in performing these tasks comes from using convolutional neural networks.

One of the most popular neural networks for this task is YOLO, created in 2015 by Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi in their famous research paper "You Only Look Once: Unified, Real-Time Object Detection".

Since that time, there have been quite a few versions of YOLO. Recent releases can do much more than object detection. During the last year, I wrote a series of articles about YOLOv8 for object detection and instance segmentation, focusing on practical usage in production. In this publication, I want to share them with a wider community. These guides show how to integrate the models, trained using this neural network to web services, written in different programming languages, including, but not limited to Python, Julia, Node.js, JavaScript, Go and Rust.

Welcome to read these guides by opening the links below.

1. How to detect objects on images using the YOLOv8 neural network.

In this article, I introduce the YOLOv8 network and show how to use it for object detection using standard API, based on PyTorch. I show how to install it, prepare a dataset, train the model and finally, how to create a web service with web interface, that can be used to upload the image and detect objects on it.

The final web service code available in this GitHub repository: https://github.com/AndreyGermanov/yolov8_pytorch_python.

2. How to create YOLOv8-based object detection web service using Python, Julia, Node.js, JavaScript, Go and Rust.

The default high level YOLOv8 API is based on PyTorch and limited to Python only. In this article, I show how to export the YOLOv8 model to ONNX format and work with it on low level, without standard API. I provide a step-by-step guide on how to prepare an input tensor from the image and how to process the output of the model to extract information about detected objects. The provided algorithm used to create an object detection web services on Python, Julia, Node.js, JavaScript, Go and Rust programming languages.

All source code available in the following repositories:

Python - https://github.com/AndreyGermanov/yolov8_onnx_python
Julia - https://github.com/AndreyGermanov/yolov8_onnx_julia
Node.js - https://github.com/AndreyGermanov/yolov8_onnx_nodejs
JavaScript - https://github.com/AndreyGermanov/yolov8_onnx_javascript
Go - https://github.com/AndreyGermanov/yolov8_onnx_go
Rust - https://github.com/AndreyGermanov/yolov8_onnx_rust

3. How to detect objects in videos in a web browser using YOLOv8 neural network and JavaScript.

Based on algorithms, defined in previous part, in this article I show how to implement object detection in videos right in a web browser, on client side, without using any backend. I guide how to integrate WebAssembly edition of ONNX library to a web interface on client side, how to convert each frame of the video to input tensor for the YOLOv8 model and how to run the object detection in parallel thread, using JavaScript WebWorkers API.

The source code of this service available in this repository: https://github.com/AndreyGermanov/yolov8_inference_video_javascript

4. How to implement instance segmentation using YOLOv8 neural network.

In this part, I describe how to use YOLOv8 for instance segmentation the same way, as it described for object detection in the previous parts. I show how to do this work using both standard YOLOv8 API and on low level, using ONNX. Finally, I explain how to create a web service, that can be used to do both object detection and instance segmentation for input image.

The source code of web service on Python and Rust provided in the following repositories:

Python: https://github.com/AndreyGermanov/yolov8_segmentation_python
Rust: https://github.com/AndreyGermanov/yolov8_onnx_rust_segmentation

Conclusion

The provided method and projects can be used as a base for more advanced object detection and instance segmentation solutions. Also, with some modifications, these algorithms are useful with any other neural networks, that can be converted to ONNX, including all models, created on TensorFlow and PyTorch.