Skip to content

Operator Parameters

Tip

Pick the default parameters for most cases, and tweak only if your dataset is unusually small, noisy, or large.

Unified Detection (FD + FE)

Parameter Type Default Description
target - dataset/view/selected/current sample Scope of operation
detection_mode - find_description find_description, find_everything, or both
GPU - - Device picker (cuda:0, cuda:1, etc)
prompts string - Comma- or newline-separated phrases for VLM
vlm_mode - both both, florence, or owl
owl_threshold float 0.30 Minimum confidence for OWL-v2 detections
iou_merge float 0.50 IoU threshold to merge Florence-2 + OWL outputs
nms_iou float 0.60 Non-max suppression threshold for merged VLM boxes
max_area_percentage float 0.20 Max object area fraction for SAM (shown when mode is find_everything or both)
min_area_percentage float 0.003 Min object area fraction for SAM (shown when mode is find_everything or both)
sam_nms_thresh float 0.70 NMS threshold for SAM boxes (shown when mode is find_everything or both)
points_per_batch int 32 SAM grid points per batch (shown when mode is find_everything or both)
overlap_iou_threshold float 0.50 Suppresses VLM boxes overlapping existing labeled ones
batch_size int 4 Operator processing batch size

Outputs

  • leip_embedding on samples (via SAM encoder)
  • vlm_detections (intermediate) when VLM runs
  • unified_detections (final):
    • includes new VLM detections (source='vlm')
    • includes new SAM detections (as label='unknown', source='sam')
    • retains existing detections depending on mode
  • bbox_area computed for every detection and stored as a field expression for patches
  • Per-detection leip_embedding extracted from the image embedding over each bounding box region

Merge semantics

  • In both mode, new VLM boxes that overlap too much (IoU > overlap_iou_threshold) with existing non-unknown detections are skipped to avoid conflicts.
  • SAM boxes that overlap an existing detection (per SAM NMS/overlap logic) are not added. Only novel boxes are added as unknown.

Memory management

  • Loads SAM/VLM models only as needed.
  • Between stages, aggressively frees GPU memory using an internal cleanup manager (torch.cuda.empty_cache(), GC, and object deletion). Cleanup summaries are logged.

Find Similar

Parameter Type Default Description
target - dataset/view/selected/current sample Scope of operation
clustering_method - hybrid hybrid = K-Means + majority + optional LDA; kmeans = K-Means only (no LDA)
target_objs_per_cluster int 15 Desired average cluster size
majority_threshold float 0.55 Fraction required to assign cluster majority label to unknowns
min_classes_for_lda int 3 Minimum distinct classes to enable LDA
min_examples_per_class int 5 Minimum examples per class for LDA training
n_neighbors int 15 UMAP neighbors for cluster visualization
batch_size int 4 Processing batch size

Notes

  • Requires unified_detections and leip_embedding.
  • Outputs: updated label and cluster_id; optional UMAP visualization.

Modify Detections

Parameter Type Default Description
action - change_label change_label or remove_detections
target - dataset/view/selected/current sample Scope of operation
unknown_only bool True Apply only to unknown detections (safer bulk ops)

Notes

  • Operates on both full samples and patch views.
  • Patch removal updates the parent sample's unified_detections accordingly.