Microscopic identification of leukocytes (white blood cells) is a common task in the hematology section of a hospital laboratory. Occasionally, laboratory personnel may lack confidence in their ability due to various factors and require assistance from someone more experienced, often by taking an image with their mobile device and sending it through an online messaging system. This project attempts to tackle this by mounting a small computer with camera and touchscreen onto a laboratory microscope for image capture and inference. The results of this project were the creation of multiple web-based links for submission of images for inference online, and various iterations of a laboratory microscope mounted computer containing a Raspberry Pi 4, Raspberry Pi HQ camera, and 5-inch touchscreen, within 3D-printed case.
The peripheral blood film is common procedure in any medical laboratory, which can be performed manually by laboratory personnel or with automated machines. It is prone to variations when viewing microscopically due to many different factors. Using a fixed position high quality camera is necessary to minimize image capture related factors.
During this project, a Raspberry Pi HQ camera was used and connected to an Olympus CX33 trinocular microscope using a camera adapter with C-mount connector. The camera was connected using varying lengths of CSI cable to the Raspberry Pi 4.
With the Raspberry Pi 4, image capture software was cloned from https://github.com/silvanmelchior/RPi_Cam_Web_Interface.git, which provides a user-friendly web interface for image capture and camera settings adjustment.
With the collected images, batches were uploaded to Roboflow, which provides a user-friendly interface for efficient image annotation, and provides use of Meta's Segment Anything model. The annotated dataset can be viewed here: https://universe.roboflow.com/nolilab/bloodsmearoilimmersion
With the dataset, object detection training / fine tuning was performed with various Roboflow Google Colab or Kaggle notebooks found at https://github.com/roboflow/notebooks. The notebooks make use of Ultralytics packages, and once training is completed, the model can be exported to various formats or uploaded to Roboflow for quick deployment. Export to TensorFlow lite and ONNX formats were chosen.
In following Roboflow's tutorials, there are multiple ways to deploy the custom model. With a Nvidia Jetson Nano Development Kit running inference as a server over the local network, the mounted Raspberry Pi device would capture images and display the inference result, as done in this repo: https://github.com/NoliAlonso/Roboflow_ObjectDetection
At this point, the project was adequate, but not optimal as it required multiple devices with multiple cables of many types. Inference was slow at 4 frames per second with the Nvidia Jetson, and 1 frame per second with a Raspberry Pi 4. It came to realization that leukocyte identification does not need to be a continuously running process, so the focus changed towards a mounted Raspberry Pi 4 that will run inference when needed.
The various iterations included the HQ camera, small touchscreen display, and a rechargeable 18650 power supply. A 3D printed cases were designed using Tinkercad include:
https://www.tinkercad.com/things/fOm3TMGSaBf-raspberry-pi-4b-ups-18650-lite-case
A final design included a 5-inch touchscreen display:
https://www.tinkercad.com/things/19mIJXxJul7-raspberry-pi-4b-5inch-dsi-hq-cam
Afterwards, simple software to run image capture and inference interactively on the Raspberry Pi was written in python. The repository can be found here: https://github.com/NoliAlonso/WBCdetector_YOLO-Roboflow
This project shows that there are multiple low-cost methods to perform microscopic identification of leukocytes. The inference output of the model varies, but it is very good at identifying neutrophils, and relatively good at identifying abnormalities such as blast cells due to their size. It was a very rewarding learning process involving computer programming, artificial intelligence, computer vision, and 3D printing.
There are no models linked
There are no models linked
