Select and Inference with Focoos Models

This section covers how to perform inference using the Focoos Models on the cloud or locally using the focoos library.

As a reference, the following example demonstrates how to perform inference using the fai-rtdetr-m-obj365 model, but you can use any of the models listed in the models section.

In this guide, we will cover the following topics:

1. ☁️ Cloud Inference !
2. ☁️ Cloud Inference with Gradio
3. 🏠 Local Inference

Cloud inference for image processing

Running inference on the cloud is simple and efficient. Select the model you want to use and call the infer method on your image. The image will be securely uploaded to the Focoos AI platform, where the selected model processes it and returns the results.

To get the model reference you can refere to the Model Management section. Here the code to handle a single image inference:

from focoos import Focoos

focoos = Focoos(api_key="<YOUR-API-KEY>")

image_path = "<PATH-TO-YOUR-IMAGE>"
model = focoos.get_remote_model("<MODEL-REF>")
result, _ = model.infer(image_path, threshold=0.5, annotate=True)

for det in result.detections:
    print(f"Found {det.label} with confidence {det.conf:.2f}")
    print(f"Bounding box: {det.bbox}")
    if det.mask:
        print("Instance segmentation mask included")
        print(f"Mask shape: {det.mask.shape}")

result is a FocoosDetections object, containing a list of FocoosDet objects and optionally a dict of information about the latency of the inference. The FocoosDet object contains the following attributes:

• bbox: Bounding box coordinates in x1y1x2y2 absolute format.
• conf: Confidence score (from 0 to 1).
• cls_id: Class ID (0-indexed).
• label: Label (name of the class).
• mask: Mask (base64 encoded string having origin in the top left corner of bbox and the same width and height of the bbox).

Optional parameters are:

• threshold (default: 0.5) – Sets the minimum confidence score required for prediction to be considered valid. Predictions below this threshold are discarded.
• annotate (default: False) – If set to True, the method returns preview, an annotated image with the detected objects.

Image Preview

You can preview the results by passing the annotate parameter to the infer method:

from PIL import Image

output, preview = model.infer(image_path, threshold=0.5, annotate=True)
preview = Image.fromarray(preview)

Cloud inference with Gradio

You can easily create a web interface for your model using Gradio.

First, install the required dev dependencies:

pip install '.[dev]'

Set your Focoos API key as an environment variable and start the application. The model selection will be available from the UI:

export FOCOOS_API_KEY_GRADIO=<YOUR-API-KEY> && python gradio/app.py

Now, your model is accessible through a user-friendly web interface! 🚀

Local inference

Note

To perform local inference, you need to install the package with one of the extra modules ([cpu], [torch], [cuda], [tensorrt]). See the installation page for more details.

You can run inference locally by selecting a model and calling the infer method on your image. If this is the first time you are running the model locally, it will be downloaded from the cloud and stored on your machine. If you are using CUDA or TensorRT, the model will be optimized for your GPU before inference. This process may take a few seconds, especially for TensorRT.

Example code:

from focoos import Focoos

focoos = Focoos(api_key="<YOUR-API-KEY>")

model = focoos.get_local_model("<MODEL-REF>")

image_path = "<PATH-TO-YOUR-IMAGE>"
result, _ = model.infer(image_path, threshold=0.5, annotate=True)

for det in result.detections:
    print(f"Found {det.label} with confidence {det.conf:.2f}")
    print(f"Bounding box: {det.bbox}")
    if det.mask:
        print("Instance segmentation mask included")
        print(f"Mask shape: {det.mask.shape}")

result is a FocoosDetections object, containing a list of FocoosDet objects and optionally a dict of information about the latency of the inference. The FocoosDet object contains the following attributes:

• bbox: Bounding box coordinates in x1y1x2y2 absolute format.
• conf: Confidence score (from 0 to 1).
• cls_id: Class ID (0-indexed).
• label: Label (name of the class).
• mask: Mask (base64 encoded string having origin in the top left corner of bbox and the same width and height of the bbox).

As for remote inference, you can use the annotate parameter to return a preview of the prediction. Local inference provides faster results and reduces cloud dependency, making it ideal for real-time applications and edge deployments.