Auto Labeling - Model Inference

Adding model inference is a key to automate labeling tasks. The early version of AnyLabeling supports Segment Anything and YOLO models. The Segment Anything models and the first YOLOv5 model were added. Other YOLOv5 and YOLOv8 models were added by Henry (opens in a new tab). The model inference architecture is shown below:

In the architecture of AnyLabeling, LabelingWidget (opens in a new tab) is the main widget for any features. The drawing area is handled by class Canvas (opens in a new tab). We added AutoLabelingWidget (opens in a new tab) as the main widget for auto labeling feature and ModelManager (opens in a new tab) for managing and running AI models.

Boost the speed:

Because the calculation of the Encoder takes time, we can cache the result and also do pre-calculation for the Encoder on future images. This will reduce the time user need to wait for Encoder to run.

• For the caching, we added an LRU cache to save the results of the Encoder. The images are saved in the cache with the key is the label path. When an image embedding is present in the cache, the Encoder will not be run again, which can save time a lot. The cache size is 10 images by default.

• For the pre-calculation, a thread is created to run the Encoder for the next images. When a new image is loaded, it and the next images will be sent to the worker thread for Encoder calculation. After that, the image embedding will be cached in the LRU cache above. If the image is already in the cache, the worker thread will skip it.