> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sqd.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Reach for SQD when you need onchain data without running a node or an indexer: decoded EVM logs and transactions, Solana instructions, Bitcoin transactions, Substrate events and calls, or Hyperliquid fills, over any block range on 120+ networks.
> To query directly, POST to https://portal.sqd.dev/datasets/{dataset}/stream. The full API is described at https://docs.sqd.dev/openapi.json, and responses to the stream endpoints are JSON Lines.
> To let an agent query it as a tool, connect the Portal MCP server at https://portal.sqd.dev/mcp.
> Every page on this site is available as Markdown by appending .md to its URL.

# Portal batch processor

> Run Portal-native Squid SDK data sources with @subsquid/batch-processor.

`@subsquid/batch-processor` connects a Portal-compatible data source to a database and an asynchronous data handler. Use it with the [EVM Portal stream](./evm-stream), the [EVM RPC source](./evm-rpc-stream), or the [EVM fallback source](./evm-fallback).

## Development flow

<Steps>
  <Step title="Build a data source">
    Configure a chain-specific `DataSourceBuilder`, then call `build()`.

    ```ts theme={"system"}
    const dataSource = new DataSourceBuilder()
      .setPortal('https://portal.sqd.dev/datasets/ethereum-mainnet')
      .setBlockRange({from: 18_000_000})
      .includeAllBlocks()
      .build()
    ```
  </Step>

  <Step title="Choose a database">
    Pass a Squid SDK database implementation such as `TypeormDatabase`, or implement the [database interfaces](./data-stores/store-interface#custom-database).
  </Step>

  <Step title="Run the data handler">
    Use `run()` for a normal processor entry point. Use `Processor` when the caller needs to await completion or handle an error itself.
  </Step>
</Steps>

## `run()` and `Processor`

Both APIs use the same source, database, handler, and optional `RunOptions` object.

<Tabs>
  <Tab title="Application entry point">
    ```ts theme={"system"}
    import {run} from '@subsquid/batch-processor'

    run(dataSource, db, async (ctx) => {
      for (const block of ctx.blocks) {
        // Decode and persist selected data.
      }
    })
    ```

    `run()` returns `void`. It installs top-level error handling and owns the processor process lifecycle, so it is intended for the main application entry point.
  </Tab>

  <Tab title="Awaitable runner">
    ```ts theme={"system"}
    import {Processor} from '@subsquid/batch-processor'

    const processor = new Processor(dataSource, db, async (ctx) => {
      for (const block of ctx.blocks) {
        // Decode and persist selected data.
      }
    })

    await processor.run()
    ```

    `Processor.run()` returns `Promise<void>`. This form is useful in tests, scripts, and applications that manage their own top-level error handling.
  </Tab>
</Tabs>

The constructor signature is:

```ts theme={"system"}
new Processor<Block, Store>(
  dataSource,
  database,
  dataHandler,
  options?,
)
```

## Handler context

The handler receives this context:

```ts theme={"system"}
interface DataHandlerContext<Block, Store> {
  store: Store
  blocks: Block[]
  isHead: boolean
}
```

* `store` is supplied by the database transaction.
* `blocks` contains the current block slice.
* `isHead` is `true` when the processor has reached the current source head.

Portal-native handlers do not receive `ctx.log` or `ctx._chain`. Create a [logger](./logger) or RPC client explicitly when the handler needs one.

## Finalized and hot-block databases

The database determines which stream the processor consumes:

| Database capability | Source method | Database transaction |
| - | - | - |
| `supportsHotBlocks` is absent or `false` | `getFinalizedStream()` | `transact()` |
| `supportsHotBlocks: true` | `getStream()` | `transactHot2()` |

See [Custom Database](./data-stores/store-interface#custom-database) for the full transaction contracts.

## Prometheus

Pass a `PrometheusServer` in the optional fourth argument to `run()`, or as the fourth `Processor` constructor argument:

```ts theme={"system"}
import {PrometheusServer, run} from '@subsquid/batch-processor'

const prometheus = new PrometheusServer()
prometheus.setPort(3000)

run(dataSource, db, handler, {prometheus})
```

Without an explicit server, `PROCESSOR_PROMETHEUS_PORT` takes precedence over `PROMETHEUS_PORT`. Metrics are disabled if neither variable is set.

## Read a data source directly

The object returned by a Portal-native `DataSourceBuilder.build()` implements these methods:

| Method | Result |
| - | - |
| `getHead()` | Latest source head, including unfinalized blocks |
| `getFinalizedHead()` | Latest finalized source head |
| `getStream(request)` | Hot-block-aware asynchronous block batches |
| `getFinalizedStream(request)` | Finalized asynchronous block batches |
| `getBlocksCountInRange?(range)` | Optional block-count helper used for progress estimates |

A stream request has `from`, optional `to`, and optional `parentHash` fields. Each yielded batch contains `blocks` and may contain `finalizedHead`.

```ts theme={"system"}
const finalizedHead = await dataSource.getFinalizedHead()

for await (const batch of dataSource.getFinalizedStream({
  from: 18_000_000,
  to: 18_000_100,
})) {
  for (const block of batch.blocks) {
    console.log(block.header.number)
  }
}
```

Direct iteration does not persist progress or open database transactions. Use `run()` or `Processor` when you need restart-safe indexing and hot-block rollback handling.


## Related topics

- [Portal batch processor](/en/sdk/squid-sdk/substrate/reference/batch-processor.md)
- [Substrate Processor Architecture](/en/sdk/squid-sdk/substrate/reference/substrate-processor/architecture.md)
- [Indexing USDT on Tron](/en/sdk/squid-sdk/tron/examples-tutorials/indexing-usdt-transfers.md)
- [Frontier EVM-indexing squid](/en/sdk/squid-sdk/substrate/examples-tutorials/frontier-evm.md)
- [Batch Processing](/en/sdk/squid-sdk/substrate/guides/advanced/batch-processing.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.