Paying full price for every frame is a choice
Two papers, three days apart
On 31 August 2026, a group including Carnegie Mellon's Katia Sycara posted "StreamScout: Learning When to Look Deeper for Streaming Video Understanding" (arXiv 2609.00291). Three days later, on 2 September, a separate paper appeared: "ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding" (arXiv 2609.02780), released under a CC-BY-4.0 licence and marked as work in progress. Neither paper cites the other; they were plainly written independently. They arrive at the same underlying idea from different directions, and that idea happens to be the constraint DHI has been building around since before either paper existed.
The shared idea
Both papers start from the same complaint: a video stream produces a continuous flood of frames, and most video-understanding systems process that flood at uniform cost, running the same expensive model over every frame whether or not that frame contains anything worth understanding. Both papers propose spending compute unevenly instead, doing something cheap almost everywhere and something expensive only where the evidence justifies it.
StreamScout implements this as a cascade. It first takes a glimpse at recent frames, cheaply. If that glimpse is not enough to answer the question at hand, it escalates to a historical lookback over frames further in the past. If that still is not enough, it escalates again to a salient retrieval step that searches more broadly for the relevant evidence. At each stage, if the evidence collected so far is sufficient, the cascade stops rather than running the more expensive stages anyway. The authors trained two variants, one distilled with LoRA and one trained with reinforcement learning. On the OVO-Bench benchmark, running Qwen3-VL-8B, they report a 14.65 point gain over the baseline, a 59 percent reduction in tokens processed compared to uniform sampling, and an average response time of 1.04 seconds.
ShallowStream attacks the same problem from inside the model rather than around it. Instead of a multi-stage cascade of separate glimpses, it uses the shallow layers of the model itself, specifically the KV cache and attention scores those layers produce, as a cheap index of which frames are actually relevant to the question. Deep processing, the expensive part, then runs only on the frames that index flags as worth a closer look. The reported numbers are large: up to 52.1 times reduction in prefill latency, and 11.9 times reduction in end-to-end latency, measured at 10 second granularity. The paper's own abstract names industrial monitoring, surveillance and early warning as target applications, which puts it, by its authors' own framing, in the same territory DHI operates in.
What DHI actually does
We did not arrive at selective compute through either of these techniques. We arrived at it through a hardware constraint: our production target is an NVIDIA Jetson Orin Nano, with 8 GB of memory shared between CPU and GPU. There is no headroom on that board to run a full, uniform cost model over every frame from every camera. The constraint forced a design decision before either of these papers gave that decision a name.
Our version of spending less on most frames happens mostly at the level of process architecture rather than inside a single model's attention mechanism. One shared inference service owns the single TensorRT execution context for the entire host. Individual camera worker processes never touch CUDA directly; they hand frames to the shared service instead of each holding their own model context. That is why adding another camera to a running host costs roughly 20 MB rather than roughly 260 MB, and it is the specific reason six concurrent camera streams fit on hardware where two separate full contexts would not. Video decode uses the hardware NVDEC block where the platform supports it, and falls back to software decoding through FFmpeg, per camera, where it does not. The Orin Nano has NVDEC for decode but no NVENC for encode, which shapes what we can and cannot do cheaply on the output side as well.
Why convergence is a signal worth taking seriously
Two research groups, working independently, publishing three days apart, both concluded that uniform per-frame cost is the wrong default for streaming video understanding, and both built a mechanism to spend compute where the evidence warrants it and skip it where it does not. That is a stronger signal than either paper alone. When one team makes an unusual architectural bet, it might be idiosyncratic. When two unconnected teams make the same bet in the same week, the constraint they are both responding to is probably real and probably durable, not a quirk of one lab's benchmark. We read this as validation that the direction we were pushed into by an 8 GB memory budget is not a compromise we will grow out of. It looks like where the field is heading anyway.
The honest gap
We want to be specific about what we have not built, rather than let the comparison above imply more than it should. Neither StreamScout's cascade nor ShallowStream's shallow-layer relevance index is implemented in DHI today. Our selective-compute story operates mostly at the coarser grain of shared execution context and camera-level decode fallback: we control how many camera streams a host can sustain and how memory is shared between them, not, within a single stream, which specific frames deserve a deeper look the way ShallowStream's attention-score index does, or which of three escalating evidence-gathering stages a question needs the way StreamScout's cascade does.
Our approach is a coarser, process-level version of the idea both papers implement at the level of model internals. That is a meaningful difference in sophistication. Closing it, moving selective compute down from which camera gets a shared context to which frame in this stream deserves a deeper pass, is a real direction for future work, not something we can claim credit for today. What we can say is that the constraint that shaped our architecture and the constraint two independent research groups chose to solve for in the same week are the same constraint, and that gives us more confidence in the shape of our system than in any single benchmark number could.