PiVoT

The non-homogeneous Poisson process (NHPP) is widely used to model high-resolution sensor data from extended objects, each of which can emit multiple measurements. Over the past decade, many multi-object trackers were developed based on this NHPP measurement model, yet they often struggle with accuracy and efficiency in large-scale tracking involving a multitude of objects, massive data, or heavy clutter. To overcome this, we introduce PiVoT (Poisson Measurements-based Variational Multi-object Detection and Tracking), an efficient variational inference framework for reliably detecting and tracking a large, varying number of objects, while simultaneously estimating their existence probabilities, shapes, measurement rates, detectabilities, and data association, even under dense clutter.

To achieve reliable and efficient approximate inference for this challenging task, PiVoT incorporates several technical innovations, including: 1) A theorem enabling convenient early identification of newly-born objects which are guaranteed to fail to associate with measurements upon convergence of variational inference; 2) a highly simplified yet equivalent global optimiser for computing existence probabilities at a fraction of the computational complexity; 3) a two-stage variational inference framework to overcome inference intractabilities.

Key Features of PiVoT

1. No gating or clustering required
   Instead, PiVoT inherently offers clutter-robust, parallelisable clustering for detecting an unknown number of probable new births through joint inference of all object states. This mechanism can also be used independently to enhance existing trackers or clustering methods.

2. Efficient and exhaustive new object detection
   PiVoT enables comprehensive detection of new objects across the entire surveillance region by initially allowing a large maximum number of births (thousands in our experiments). Crucially, at each iteration of variational inference, PiVoT early identifies and removes ineffective births—those that will ultimately fail to associate with measurements—thereby progressively improving efficiency and refining the birth model.

3. State-of-the-art empirical performance
   PiVoT demonstrates clear advantages in both accuracy and efficiency over existing Poisson measurement-based trackers across diverse tracking scenarios. More importantly, it enables reliable real-time tracking in challenging conditions, such as dense clutter or tracking a thousand closely spaced objects in under a second on a laptop without gating.

✨ Demonstrations of PiVoT

The following three demonstrations highlight PiVoT’s capabilities. The first two are simulated scenarios designed to be particularly challenging—one involving dense clutter, and another with a thousand moving objects. To our knowledge, as of the initial posting date (8 April 2025), no existing Poisson measurement-based trackers can achieve reasonable performance in either setting within a practical runtime. The final demo shows PiVoT applied to real radar data of ships near anchorage.

All tracking results and computational timings shown below were obtained using MATLAB on a test laptop (Apple M1, 16GB RAM), using only vectorisation and without employing CPU/GPU parallelisation or gating techniques.

Demo 1: Dense Clutter

In this demo, we showcase PiVoT’s capability of joint detection, tracking, and estimation of shape and measurement rates under dense clutter (Poisson rate 1575). Here, the number of objects increases from 50 to 100 and then decreases back to 50, with objects appearing and disappearing throughout the surveillance region. Two object types are present: smaller circular objects with a Poisson rate of 5 and larger circular ones with a rate of 8.

The measurements over 50 time steps are shown below. The primary challenge in this setting is the dense clutter—true objects are difficult to distinguish, even to the human eye.

We now apply the PiVoT to this task for detection, tracking and estimation of object shapes and rates. Below are the results.

Each ground-truth trajectory and large or small circle is closely matched by a red or green track and ellipse, demonstrating accurate detection, tracking, and shape estimation. The average processing time per time step is just 0.28 seconds, highlighting PiVoT’s efficiency.

Demo 2: Tracking 1000 closely-spaced objects

In this demo, we showcase PiVoT’s real-time detection and tracking performance in large-scale tracking. For simplicity, we assume relatively low clutter (Poisson rate 15.75), and known objects shape and rates. Here, the object number starts from 100, then with four sudden increases to a number up to 1036 objects, and finally decreases to a number around 450. All objects are with Poisson rate 5.

The measurements over 50 time steps are shown below. Key challenges in this scenario include efficiently handling massive data, promptly detecting numerous object births and deaths, and resolving frequent track coalescence.

Each ground-truth trajectory is closely matched by a red or green track, demonstrating ability to track a thousand objects simultaneously without excessive track loss and timely detection of object births and deaths. The average processing time per time step is only 0.67 seconds. This is particularly impressive considering that no gating is used, with up to 1036 objects and more than 5,000 measurements.

Further speed improvements could be expected through optimised implementation using CPU/GPU parallelisation and gating for a large surveillance area, making PiVoT a highly promising solution for real-time multi-object tracking in large-scale, cluttered environments.

Demo 3: Tracking ships with real radar data.

This final demo presents PiVoT applied to real radar data of ships navigating near an anchorage area along a river. Dozens of vessels of varying sizes enter and exit the scene, alongside static objects such as buoys and anchored ships. A bridge spans the river on the left side of the scene. Some objects may become occluded (i.e., temporarily stop generating measurements); PiVoT will identify such objects and display their previous estimates as yellow tracks and ellipses. In this dataset, a large ship can produce over 100 measurements, leading to approximately 4000 measurements per time step.

PiVoT is used for detection, tracking, and estimation of object shapes and measurement rates. The result is shown below:

PiVoT successfully tracks and estimates the shape of each ship. Notably, PiVoT can also re-detect objects that have been occluded for some time, as seen in tracks that transition from yellow to red. Interestingly, the bridge on the left is interpreted as two connected, elongated elliptical objects. This is due to the absence of a clutter map: any dense cluster of measurements may be treated as an object. However, such detections can be easily filtered out during post-processing by examining their velocity or shape.

The average processing time per time step is 0.171 seconds, making PiVoT well suited for real-time ship tracking in radar-based systems, where it is expected to remain robust against potentially large numbers of vessels entering and heavy clutter caused by extreme weather.

Stayed tuned for more

Technical details of PiVoT will be available soon, along with quantitative results demonstrating clear improvements in both accuracy and efficiency over existing benchmarks. These evaluations are conducted on a range of standard tracking scenarios—less challenging than those shown in Demo 1 and Demo 2 above—as PiVoT is currently the only Poisson measurement-based tracker capable of handling such difficult settings in practical time.