ultralytics v8.4.128

v8.4.128

v8.4.128 - Use synchronous OpenVINO batch inference (#25921)

Added 4
  • NDJSON conversion now supports local image paths
  • Local Docker build instructions added to documentation
  • Task-specific K-Fold guidance expanded in documentation
  • Android ExecuTorch setup documented
Changed 7
  • OpenVINO now submits each input batch as a single synchronous request and consistently uses the LATENCY performance hint instead of asynchronous requests
  • TensorRT 7–10 now keeps only the detection-head Sigmoid layers in higher precision instead of affecting every matching activation
  • ONNX export now caps the opset at 18 to avoid CUDA execution falling back to CPU for unsupported operations
  • SAM auto-mask generation now encodes each crop once and reuses its features across point batches
  • Documentation now describes tracking as a mode that runs on detection, segmentation, pose, or OBB models
  • Augmentation behavior clarified in documentation
  • YOLO26n weights used for AMP compatibility checks are now stored in the user configuration directory and reused across projects
Fixed 7
  • RKNN INT8 export now clearly rejects unsupported non-detection tasks and recommends FP16 instead
  • Cached images are stored in a shared contiguous memory buffer to prevent DataLoader workers from duplicating the cache during forked training
  • FastSAM box and point prompts are clipped to image boundaries to prevent negative or out-of-range coordinates from selecting incorrect masks
  • DDP now ignores externally set RANK and LOCAL_RANK values unless a real multi-process environment is detected
  • Ray tuning correctly aggregates metrics across multiple datasets and reports the completed epoch
  • Scalar indexing now preserves bounding-box formats and instance dimensions
  • Ground-truth candidate selection is more consistent for very small boxes

From ultralytics

🌟 Summary

v8.4.128 improves OpenVINO reliability and batch inference, reduces RAM use during training, strengthens export behavior, and clarifies dataset, augmentation, and tracking workflows. 🚀

📊 Key Changes
  • Synchronous OpenVINO batch inference — priority update ⚡
    OpenVINO now submits each input batch as a single synchronous request and consistently uses the LATENCY performance hint. Throughput implementations remain available internally, but mode selection is forced to latency-oriented execution to avoid hangs in AsyncInferQueue, particularly for dynamic INT8 batches on CPU systems.
  • Improved OpenVINO efficiency and stability 🧠
    Benchmarks showed that one batched synchronous request was about twice as fast and used roughly one-third the RAM compared with splitting the batch into separate asynchronous requests. This should make OpenVINO exports more dependable in CI and production workloads, though throughput-focused applications may see different performance characteristics.
  • TensorRT INT8 optimizations 🔧
    TensorRT 7–10 now keeps only the detection-head Sigmoid layers in higher precision instead of affecting every matching activation. TensorRT 11 no longer applies the unnecessary Sigmoid exclusion. This reduces model size and improves inference speed while preserving confidence calibration.
  • More compatible ONNX exports 📦
    ONNX export now caps the opset at 18 to avoid CUDA execution falling back to CPU for unsupported operations. This prevents extra host-memory copies and should improve GPU inference consistency, especially for models such as RT-DETR.
  • Safer RKNN exports 📱
    RKNN INT8 export now clearly rejects unsupported non-detection tasks and recommends FP16 instead. Documentation also adds updated YOLO26 FP16 and INT8 benchmarks for Rockchip devices.
  • Lower RAM usage for cache='ram' 💾
    Cached images are stored in a shared contiguous memory buffer, preventing DataLoader workers from duplicating the cache during forked training. This should keep memory usage flatter when using multiple workers.
  • SAM and FastSAM improvements 🎯
    • SAM auto-mask generation now encodes each crop once and reuses its features across point batches, reducing repeated computation.
    • FastSAM box and point prompts are clipped to image boundaries, preventing negative or out-of-range coordinates from silently selecting incorrect masks.
  • More robust training and data utilities 🛠️
    • DDP now ignores externally set RANK and LOCAL_RANK values unless a real multi-process environment is detected.
    • Ray tuning correctly aggregates metrics across multiple datasets and reports the completed epoch.
    • Scalar indexing now preserves bounding-box formats and instance dimensions.
    • Ground-truth candidate selection is more consistent for very small boxes.
  • Improved dataset workflows 📚
    NDJSON conversion now supports local image paths, while COCO JSON training documentation adds clearer requirements, cache warnings, and validation guidance.
  • Documentation and usability updates ✍️
    Documentation now describes tracking as a mode that runs on detection, segmentation, pose, or OBB models; adds local Docker build instructions; expands task-specific K-Fold guidance; clarifies augmentation behavior; and documents Android ExecuTorch setup.
  • AMP check weights are cached globally 📥
    The YOLO26n weights used only for AMP compatibility checks are now stored in the user configuration directory and reused across projects instead of being downloaded into the current working directory.
🎯 Purpose & Impact
  • More reliable deployment: Synchronous OpenVINO execution reduces the risk of indefinite hangs and makes batch inference more predictable across Intel and AMD CPU environments. ✅
  • Better performance on supported hardware: TensorRT and ONNX changes reduce unnecessary precision constraints and CPU fallback operations, potentially improving speed, memory usage, and GPU utilization.
  • Lower training memory requirements: Shared RAM caching is especially beneficial for large datasets and multi-worker training.
  • Fewer silent prediction errors: Prompt clipping and format-preserving indexing ensure that invalid coordinates or scalar selections do not produce misleading masks or improperly interpreted boxes.
  • Clearer user guidance: Updated documentation makes it easier to choose the right task, prepare datasets, configure augmentations, build Docker images, and deploy models on mobile and edge hardware.
  • Compatibility note: OpenVINO now prioritizes latency-safe synchronous execution over automatic throughput mode selection. Users seeking maximum throughput should benchmark their specific hardware and workload after upgrading.
What's Changed
New Contributors

Full Changelog: https://github.com/ultralytics/ultralytics/compare/v8.4.127...v8.4.128

View original

Upgraded? How did it go?

Discussion