Getting Started Tutorial

This tutorial provides step-by-step instructions for using Latent Assisted Label with image datasets. By the end of this tutorial, you will have loaded a dataset, generated object proposals using both a Vision Language Model (VLM) and Segment Anything Model (SAM), clustered visually similar objects, and refined object detections.

1. Load dataset

For this tutorial, we're going to use the overhead dataset, which is available on the Sample datasets page. If you're running Latent Assisted Label via a Docker container, you can download the dataset directly to the working directory you mounted when you started the container:

curl -O https://docs.latentai.io/label/assets/datasets/overhead.zip

Now let's create a directory for the images and unzip the dataset:

mkdir -p overhead && unzip overhead.zip -d overhead

To load a dataset for the first time, use the Create Dataset from Directory operator.

Create Dataset from Directory

Provide a relative or absolute path to the dataset directory, and click Execute. If you downloaded and unzipped the dataset into the directory where you spun up the container, it should be available inside the container at /data/overhead/.

Once you've loaded a dataset, it will always be available to load in the Select Dataset menu.

Load Dataset

2. Run Unified Detection

Select the Unified Detection operator in the Browse Operations menu.

Select Unified Detection

Now we need to select the detection mode. For this tutorial, we'll select Both, which will detect the classes we prompt as well as all visible objects.

• Find Description – detect only prompted classes using a Vision Language Model (VLM).
• Find Everything – detect all visible objects using Segment Anything Model (SAM).
• Both (Find Description + Find Everything) – run both modes and merge results (recommended for general use).

Let's enter the following prompts to guide Find Description:

car, person, crosswalk

If needed, select your GPU device.

For this dataset, enter the following settings:

• max_area: 0.05
• min_area: 0.001
• nms_iou_threshold: 0.3
• sam_nms_threshold: 0.3

Adjust Parameters

Once you've made those adjustments, click Execute.

Execute Unified Detection

3. Cluster similar objects with Find similar

Before proceeding, confirm each detection contains a leip_embedding.

Inspect detections

Now select the Find Similar operator in the Browse Operations menu and click Execute.

Execute Find Similar

Now we're going to inspect the embedding visualization. Open the embedding visualization by selecting brain_key='find_similar_clusters'.

Open embedding visualization

Let's confirm the clusters are coherent. If you lasso around clusters, you should see similar objects grouped together.

Confirm cluster coherency

4. Refine detection with Modify Detections

Now let's assign labels to some objects currently marked unknown. Using the lasso tool we explored in the previous step, find a cluster of similar objects, then select Modify Detections in the Browse Operations menu and select change_label.

Unless you're confident about modifying an inaccurate label, it's recommended to select unknown_only when editing in bulk to avoid changing confirmed labels.

Click Execute and wait for the operator to finish.

Modify Detections

You may need to refresh your browser window. Then you can confirm the labels were updated. Finally you can rerun Find Similar to propagate new labels based on the modifications we just made.

And that's it! We generated object proposals, clustered visually similar objects, and used the two-dimensional embedding space to modify our detections.