Zero-Latency Segment Splicing: Architecting Seamless Fragment Skipping in Modern Android Media Engines

The Latency Penalty of Naive Segment Seeking

Modern client-side media applications increasingly require dynamic segment skipping. Whether implementing programmatic ad-stripping, skipping sponsor fragments based on crowd-sourced metadata (as popularized by tools like SponsorBlock and mobile forks like PipePipe), or clipping dynamic intervals for preview streams, video engineers face an identical architectural challenge: how do you jump past an arbitrary segment of an active media stream without stalling the playback pipeline?

In standard Android media development using androidx.media3.exoplayer, the intuitive reaction to encountering an excluded timestamp window [T_start, T_end] is invoking player.seekTo(targetPosition) when the presentation clock reaches T_start. In practice, this naive seek wreaks havoc on media rendering pipelines.

// The Anti-Pattern: Triggering seeks inside an analytics/position poller
player.addListener(object : Player.Listener {
    override fun onPositionDiscontinuity(
        oldPosition: Player.PositionInfo,
        newPosition: Player.PositionInfo,
        reason: Int
    ) {
        // Naive seek produces buffer flushes, surface blanking, and frame stutter
        if (isInExcludedRange(newPosition.positionMs)) {
            player.seekTo(getRangeEnd(newPosition.positionMs))
        }
    }
})

A standard seek triggers an abrupt chain reaction across the player stack:

Decoder Flushes: The underlying MediaCodec instances for both audio and video receive a flush signal (flush()), discarding decoded output buffers and pending input bitstream buffers.
Demuxer Desynchronization: The Extractor must seek backwards or forwards to the nearest preceding Keyframe / Independent Decoder Refresh (IDR) frame before T_end to re-initialize inter-frame prediction contexts.
Preroll Decode Stalls: The video hardware decoder must decode all non-displayed intermediate P- and B-frames from the nearest keyframe up to T_end before surfacing a displayable frame.
Audio Sink Desync & Popping: The AudioTrack output stream encounters a discontinuity, leading to micro-clicks, buffer under-runs, or temporary muting.

The end result is an unacceptable 150ms to 400ms freeze, visual tearing, and brief audio pops. To achieve true frame-accurate, gapless segment bridging, we must avoid seeking entirely. Instead, we must mutate the player's underlying presentation topology by virtualizing the media timeline.

Deconstructing the Media3 Pipeline Topology

To engineer a zero-latency skipping engine, we must understand how Android Media3 decouples playback time from real media presentation timestamps (PTS). ExoPlayer operates on three primary abstractions:

MediaSource: The provider of stream chunks, manifests, and demuxed bitstreams. It emits a Timeline object describing the media structure.
Timeline: An immutable snapshot of the media layout, broken into Window (a high-level playable entity, like a track or episode) and Period (an atomic chunk of continuous media containing a single presentation timeline).
Renderer (MediaCodecVideoRenderer, MediaCodecAudioRenderer): The consumers that read demuxed samples, feed hardware decoders, and synchronize output presentation to an internal master clock (MediaClock).

[ Raw Media Source ] 
         │
         ▼
[ Virtualized Segment Timeline (Transforms PTS space) ]
         │
         ├──► Audio Stream (Filtered via PCM AudioProcessor)
         │         │
         │         ▼
         │    [ AudioSink ] ──► Hardware DAC
         │
         └──► Video Stream (Bitstream-level AVPacket bridge)
                   │
                   ▼
              [ MediaCodec ] ──► Surface (Zero-copy)

When a standard playback engine plays through a media period, its position is continuous. However, if we feed the Renderer an augmented Timeline that natively maps non-contiguous byte/sample ranges into a seamless virtual interval, the decoder does not undergo a state reset. The media clock continues ticking seamlessly across segment boundaries.

Engineering the Virtualized Splicing Timeline

Instead of watching the playback cursor hit a boundary and reacting, we intercept the media source definition prior to pipeline preparation and wrap it inside a custom ForwardingTimeline and MaskingMediaSource.

Consider an asset spanning 0 to 600,000 ms, with an excluded segment between 120,000 ms and 180,000 ms. Rather than presenting a single 10-minute continuous period, we construct a compound virtual timeline containing two consecutive synthetic periods:

Period 0: Maps virtual range [0, 120,000) to physical PTS [0, 120,000)
Period 1: Maps virtual range [120,000, 540,000) to physical PTS [180,000, 600,000)

Here is how we subclass ForwardingTimeline to rewire window durations and period offsets:

package dev.media.engine.timeline

import androidx.media3.common.Timeline
import androidx.media3.common.util.UnstableApi

@UnstableApi
class SplicedVirtualTimeline(
    delegate: Timeline,
    private val excludedIntervals: List<ClosedRange<Long>>
) : Timeline() {

    private val delegateTimeline = delegate
    private val durationOffsetMs: Long

    init {
        // Calculate total excluded duration across all intervals
        durationOffsetMs = excludedIntervals.sumOf { (it.endInclusive - it.start) }
    }

    override fun getWindowCount(): Int = delegateTimeline.windowCount

    override fun getWindow(windowIndex: Int, window: Window, defaultPositionProjectionUs: Long): Window {
        delegateTimeline.getWindow(windowIndex, window, defaultPositionProjectionUs)
        // Compress the virtual window presentation duration
        window.durationUs -= durationOffsetMs * 1000L
        return window
    }

    override fun getPeriodCount(): Int {
        // Each segment gap partitions the presentation into distinct periods
        return delegateTimeline.periodCount + excludedIntervals.size
    }

    override fun getPeriod(periodIndex: Int, period: Period, setIds: Boolean): Period {
        // Map period transitions to bypass physical time intervals
        delegateTimeline.getPeriod(0, period, setIds)
        
        val interval = excludedIntervals.getOrNull(periodIndex)
        if (interval != null) {
            val gapDurationUs = (interval.endInclusive - interval.start) * 1000L
            period.set(
                period.id,
                period.uid,
                periodIndex,
                /* durationUs = */ period.durationUs - gapDurationUs,
                /* positionInWindowUs = */ calculatePeriodOffsetUs(periodIndex)
            )
        }
        return period
    }

    private fun calculatePeriodOffsetUs(periodIndex: Int): Long {
        if (periodIndex == 0) return 0L
        return excludedIntervals.take(periodIndex).sumOf { (it.endInclusive - it.start) * 1000L }
    }
}

Solving the GOP Boundary Problem: Frame-Accurate Bridging

While wrapping the Timeline works well when segment boundaries align strictly with intra-coded keyframes (IDRs), video encoders almost never place keyframes at the exact millisecond where a content boundary or sponsor block starts or ends.

If the skip point T_end (e.g., 180,000 ms) lands on a non-IDR frame (such as a bidirectional predictive B-frame), the decoder cannot begin reconstructing pixels immediately at T_end without the reference I/P-frame from which it derives motion vectors. If you force the demuxer to send samples starting strictly at 180,000 ms, MediaCodec will output corrupt pixel macroblocks until the next keyframe arrives, or fail entirely.

The Dual-Track Forward-Decode Technique

To guarantee deterministic frame accuracy without stalling playback, implement a low-level MediaSource wrapper that handles decoder pre-rolling:

Pre-roll Extraction: When the stream crosses the splice boundary T_start, the demuxer continues reading packets for Period 1 starting at the nearest preceding keyframe before T_end (e.g., keyframe at 176,500 ms).
Surface Presentation Muting: The audio sink drops PCM output for decoded samples between 176,500 ms and 180,000 ms.
Video Buffer Dropping: For frames decoded between 176,500 ms and 180,000 ms, MediaCodecVideoRenderer invokes codec.releaseOutputBuffer(index, /* render = */ false) instead of passing them to the hardware Surface.
Resumed Presentation: As soon as the presentation timestamp reaches 180,000 ms, the renderer sets render = true and updates the native presentation time directly against the master clock.

// Intercepting output frame release inside a custom MediaCodecVideoRenderer
override fun processOutputBuffer(
    positionUs: Long,
    elapsedRealtimeUs: Long,
    codec: MediaCodecAdapter,
    buffer: ByteBuffer?,
    bufferIndex: Int,
    bufferFlags: Int,
    sampleCount: Int,
    bufferPresentationTimeUs: Long,
    isDecodeOnlyBuffer: Boolean,
    isLastBuffer: Boolean,
    format: Format
): Boolean {
    // Check if the current frame falls within an excluded pre-roll interval
    if (isPreRollDiscardRange(bufferPresentationTimeUs)) {
        // Discard without rendering to the Android Surface
        codec.releaseOutputBuffer(bufferIndex, false)
        return true
    }
    
    // Standard rendering logic continues down the pipeline
    return super.processOutputBuffer(
        positionUs,
        elapsedRealtimeUs,
        codec,
        buffer,
        bufferIndex,
        bufferFlags,
        sampleCount,
        bufferPresentationTimeUs,
        isDecodeOnlyBuffer,
        isLastBuffer,
        format
    )
}

By executing decoder pre-roll in the background ahead of the presentation boundary, frames are pre-calculated. When the playback cursor hits 120,000 ms, the very next frame submitted to the compositor is the fully decoded frame at 180,000 ms. Total perceived latency: 0 milliseconds.

Mitigating Audio Discontinuities with a Custom AudioProcessor

Video can tolerate simple frame switches, but human hearing is hypersensitive to phase misalignments. If an audio stream is abruptly joined across non-zero crossings, a loud acoustic click or pop occurs.

To resolve this, we place a custom AudioProcessor into the audio pipeline behind the decoder but upstream from the AudioSink (DefaultAudioSink).

package dev.media.engine.audio

import androidx.media3.common.audio.AudioProcessor
import androidx.media3.common.audio.BaseAudioProcessor
import java.nio.ByteBuffer
import java.nio.ByteOrder
import kotlin.math.sin

class CrossfadeSpliceAudioProcessor : BaseAudioProcessor() {

    private var isSpliceImminent = false
    private val crossfadeWindowSamples = 256 // ~5.8ms at 44.1kHz

    fun notifySplicePending() {
        isSpliceImminent = true
    }

    override fun queueInput(inputBuffer: ByteBuffer) {
        if (!isSpliceImminent || !inputBuffer.hasRemaining()) {
            val remaining = inputBuffer.remaining()
            val outputBuffer = replaceOutputBuffer(remaining)
            outputBuffer.put(inputBuffer)
            outputBuffer.flip()
            return
        }

        // Apply micro-crossfade: smooth logarithmic fade-out to prevent pop
        val samples = inputBuffer.remaining() / 2 // 16-bit PCM
        val outBuffer = replaceOutputBuffer(inputBuffer.remaining())
        val shortBuffer = inputBuffer.order(ByteOrder.LITTLE_ENDIAN).asShortBuffer()

        for (i in 0 until samples) {
            val sample = shortBuffer.get()
            val gain = 1.0 - (i.toDouble() / crossfadeWindowSamples.coerceAtLeast(1))
            val attenuated = (sample * gain.coerceIn(0.0, 1.0)).toInt().toShort()
            outBuffer.putShort(attenuated)
            if (i >= crossfadeWindowSamples) break
        }
        
        isSpliceImminent = false
        outBuffer.flip()
    }
}

This micro-fade-out attenuates the waveform amplitude to zero within a 5-millisecond window directly preceding the splice cut. When the stream resumes at the target timestamp, the reverse logarithmic fade-in occurs, eliminating transient clicks without introducing perceptual delay.

Verifying Zero Pipeline Stalls with Perfetto

To ensure your timeline virtualization does not inadvertently trigger garbage collector pauses or underlying codec resets, profile your application using Android's system tracing tool (Perfetto).

Look for the following signals in the trace view:

MediaCodec.dequeueOutputBuffer continuity: The thread driving the video decoder must not transition into an extended Sleeping or Blocked state across the skip boundary.
Choreographer#doFrame alignment: The frame render callback should consistently hit display refresh deadlines (e.g., 8.33ms for 120Hz displays, 16.66ms for 60Hz displays) throughout the boundary crossing.
Absence of AudioTrack.write underrun: Verify that AudioTrack.write() calls maintain their cadence without draining the platform buffer to zero (BufferUnderflowException).

Conclusion: Architectural Determinism Over Reactive Hacks

Seamless media engineering requires shifting your mindset from reactive state checking to declarative pipeline construction. When building client-side dynamic media applications on Android, treating video playback as a fluid, editable coordinate system—rather than a static timeline you jump around in—is the key to unlocking cinema-grade, zero-latency transitions.

By leveraging ForwardingTimeline, orchestrating pre-roll discard passes in custom renderers, and micro-crossfading PCM buffers in an upstream AudioProcessor, you construct a playback architecture capable of jumping across any stream discontinuity without dropping a single frame.