# Processing the stream

Verify leader signatures, deduplicate valid shreds, recover missing data per FEC set and replay and confirm before acting on the stream.

Source: https://docs.zerocopy.sh/shredstream/processing

A shred arrives before replay. It can be incomplete, duplicated or from a slot that is
later skipped. A received shred does not establish that a transaction executed, landed
or finalized. Your application must replay the reconstructed data and confirm at the
commitment it needs before acting. See [Terms 6.1](https://zerocopy.sh/terms#network-1).

## Verify the leader signature [#verify-the-leader-signature]

Shreds are not verified before delivery. Use a Solana Merkle shred verifier to check the
Ed25519 signature over the Merkle root against the slot leader's identity. Reject
unsigned shreds and shreds with invalid signatures. The [source allowlist](/shredstream/receivers#firewall)
is a traffic filter, not signature verification.

## Deduplicate valid shreds [#deduplicate-valid-shreds]

Deduplicate by `(slot, index, shred type)`, where the type distinguishes data from coding
shreds. Keep the first signature-valid copy. An invalid copy must not prevent a later
valid copy from being accepted.

Receivers in different metros get their own copies, so deduplicate again when combining
streams. Retain keys for the slots your application is still processing, then retire
them when those slots leave its processing window.

## Recover data shreds [#recover-data-shreds]

Collect data and coding shreds by `slot` and `fec_set_index`. Use the coding header's
`num_data_shreds`, `num_coding_shreds` and `position` to recover missing data shreds within
each FEC set. Decode entries and transactions from the reconstructed data.

UDP does not retransmit missing datagrams. Some FEC sets can remain incomplete; do not
treat partial reconstruction as a complete block.

## Replay and confirm [#replay-and-confirm]

Replay the reconstructed data and verify its outcome. Confirm transactions at the
commitment your application requires before acting. A shred's slot or last-in-slot flag
is not a commitment result. Handle skipped slots and incomplete data explicitly in your
consumer.

## Receiver example [#receiver-example]

This TypeScript example runs with Bun. It binds a UDP socket, reads the
[common header](/shredstream/format#common-header) and suppresses repeated
`(slot, index, variant)` keys while they remain in a bounded cache.

Configure the [firewall allowlist](/shredstream/receivers#firewall) first. Set
`SHREDSTREAM_UDP_PORT` to the receiver's assigned UDP port and `DEDUPE_ENTRIES` to the
maximum number of header keys to retain, then run `bun receiver.ts`. The cache evicts the
oldest inserted key when full, so a duplicate can appear again after eviction.

```ts title="receiver.ts"
import { createSocket } from 'node:dgram'

const port = Number(process.env.SHREDSTREAM_UDP_PORT)
const capacity = Number(process.env.DEDUPE_ENTRIES)
if (!Number.isInteger(port) || port < 1 || port > 65535) {
  throw new Error('Set SHREDSTREAM_UDP_PORT to the assigned UDP port')
}
if (!Number.isSafeInteger(capacity) || capacity < 1) {
  throw new Error('Set DEDUPE_ENTRIES to a positive integer')
}

const seen = new Set<string>()
const socket = createSocket('udp4')
socket.on('error', (error) => {
  console.error(error)
  socket.close()
  process.exitCode = 1
})
socket.on('message', (message) => {
  if (message.length < 83 || message.length > 1228) return
  const header = {
    signature: message.subarray(0, 64),
    variant: message[64],
    slot: message.readBigUInt64LE(65),
    index: message.readUInt32LE(73),
    version: message.readUInt16LE(77),
    fecSetIndex: message.readUInt32LE(79),
  }
  const key = `${header.slot}:${header.index}:${header.variant}`
  if (seen.has(key)) return
  if (seen.size === capacity) {
    const oldest = seen.values().next().value
    if (oldest !== undefined) seen.delete(oldest)
  }
  seen.add(key)
  console.log({
    slot: header.slot.toString(),
    index: header.index,
    variant: header.variant,
    version: header.version,
    fecSetIndex: header.fecSetIndex,
  })
})
socket.bind(port)
```

This is a header-inspection receiver. It does not verify signatures, validate the Merkle
body or current shred version, recover FEC sets, replay blocks or confirm transactions.
Its duplicate cache records unverified headers; do not use it to select a trusted copy
or act on transaction data. A production consumer must verify first, derive the shred
type from the variant, and keep the first valid `(slot, index, type)` copy.