> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sqd.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Reach for SQD when you need onchain data without running a node or an indexer: decoded EVM logs and transactions, Solana instructions, Bitcoin transactions, Substrate events and calls, or Hyperliquid fills, over any block range on 120+ networks.
> To query directly, POST to https://portal.sqd.dev/datasets/{dataset}/stream. The full API is described at https://docs.sqd.dev/openapi.json, and responses to the stream endpoints are JSON Lines.
> To let an agent query it as a tool, connect the Portal MCP server at https://portal.sqd.dev/mcp.
> Every page on this site is available as Markdown by appending .md to its URL.

# Private Portal setup

> Deploy private SQD Portal stacks for archival, real-time, and custom EVM data.

A private Portal stack can serve finalized SQD Network data, recent data from your own RPC endpoints, or both through one API.

Start by choosing the topology. The services and storage requirements differ substantially.

## Choose a topology

| Topology | Historical data | Recent data | SQD lock required | Main trade-off |
| - | - | - | - | - |
| Archival only | SQD Network | No | Yes | Fast backfills, but data can lag the chain head |
| Real-time only | Your RPC through HotblocksDB | Your RPC through HotblocksDB | No | Supports custom EVM networks, but retention determines disk growth and replay depth |
| Archival plus real-time | SQD Network | Your RPC through HotblocksDB | Yes | Full history and current data, with more services to operate |

<Warning>
  Real-time-only storage configured from block zero grows with the chain.
  Estimate disk growth before using it for a large or high-throughput network.
</Warning>

## Components and service count

| Component | Responsibility | How many to run |
| - | - | - |
| Portal | Serves the Portal API and joins SQD Network history with real-time data | One or more stateless replicas |
| EVM data service | Polls and validates one RPC source, then serves full blocks to HotblocksDB | One per RPC endpoint |
| HotblocksDB | Stores and queries recent blocks for one or more datasets | One stateful instance per failure domain; replicas have independent disks |
| Hotblocks-retain | Tells HotblocksDB when blocks are available from the Network and can be removed | One per `Api`-retention HotblocksDB instance |

You do not need one HotblocksDB per network. A single HotblocksDB configuration can declare several datasets and consume one or more EVM data services for each dataset.

You normally run at least one EVM data service per network. Run additional services when the same network uses multiple RPC providers for redundancy.

## Prerequisites

* Docker and Docker Compose
* A persistent volume for every HotblocksDB instance
* An RPC endpoint with enough capacity for every real-time EVM dataset
* `debug_traceBlockByHash` support when traces or state diffs are required
* A load balancer and multiple replicas when one instance is not an acceptable failure domain

Accessing finalized data from the Network also requires:

* at least 1,000,000 locked `SQD` (see [Token requirements and compute units](#token-requirements-and-compute-units) for sizing);
* an Arbitrum RPC endpoint and an Ethereum L1 RPC endpoint;
* a registered Portal peer ID;
* Arbitrum ETH for registration and lock transactions.

<Info>
  The real-time ingestion example below uses the EVM data service. Other data
  kinds require a compatible service that can feed HotblocksDB.
</Info>

## Archival-only setup

Use this topology for high-throughput historical queries when a delay near the chain head is acceptable.

<Steps>
  <Step title="Clone the Portal repository">
    ```bash theme={"system"}
    git clone https://github.com/subsquid/sqd-portal
    cd sqd-portal
    cp mainnet.env .env
    ```
  </Step>

  <Step title="Generate and register the Portal identity">
    Generate a private key and peer ID:

    ```bash theme={"system"}
    docker run --rm \
      -u "$(id -u):$(id -g)" \
      -v "$PWD:/cwd" \
      subsquid/keygen:latest /cwd/portal.key
    ```

    Register the printed peer ID and lock `SQD` in the [Network app](https://network.sqd.dev/portals).

    <Warning>
      Protect `portal.key`. Anyone with the key can run a process using that peer
      identity. See [consequences of losing your
      key](#what-are-the-consequences-of-losing-my-key-file-getting-it-stolen).
    </Warning>
  </Step>

  <Step title="Review the network configuration">
    The repository's `mainnet.config.yml` serves every Network dataset:

    ```yaml title="mainnet.config.yml" theme={"system"}
    hostname: http://0.0.0.0:8000
    sqd_network:
      datasets: https://cdn.subsquid.io/sqd-network/datasets.yml
      metadata: https://cdn.subsquid.io/sqd-network/mainnet/metadata.yml
      serve: all
    ```

    Review the RPC endpoints and listen address in `.env` before starting the service.
  </Step>

  <Step title="Start and verify">
    ```bash theme={"system"}
    KEY_PATH=./portal.key docker compose up -d
    docker compose logs -f portal
    ```

    The lock becomes active at the start of the next network epoch. After activation:

    ```bash theme={"system"}
    curl --fail http://127.0.0.1:8000/ready
    curl --fail http://127.0.0.1:8000/datasets
    ```
  </Step>
</Steps>

## Real-time-only setup for a custom EVM network

HotblocksDB already serves the Portal streaming API. If clients only need real-time stream and head endpoints, they can connect to HotblocksDB directly without running the Portal service.

### Configure retention

Create a HotblocksDB dataset configuration:

```yaml title="hotblocks.yaml" theme={"system"}
krown-network:
  kind: evm
  retention_strategy:
    Head: 2000
  data_sources:
    - http://krown-data-service:3000
```

Choose one retention strategy:

* `Head: 2000` keeps a sliding window of the latest 2,000 blocks.
* `FromBlock: { number: 0 }` keeps everything from block zero.
* `FromBlock: { number: 123456 }` keeps everything from a chosen start block.

Use `FromBlock` only when the retained range fits on disk. Keeping the whole chain makes local reindexing fast, but storage continues to grow.

### Run the services

The minimal Compose shape is:

```yaml title="compose.yaml" expandable theme={"system"}
services:
  krown-data-service:
    image: subsquid/evm-data-service:latest
    command:
      - --http-rpc
      - ${KROWN_RPC_URL:?}
      - --with-receipts
      - --block-cache-size
      - "1000"

  hotblocks-db:
    image: subsquid/data-hotblocks:48373fe7
    command:
      - --datasets
      - /app/hotblocks.yaml
      - --db
      - /run/db
      - --port
      - "8081"
    ports:
      - "8081:8081"
    volumes:
      - ./hotblocks.yaml:/app/hotblocks.yaml:ro
      - hotblocks-data:/run/db
    depends_on:
      - krown-data-service

volumes:
  hotblocks-data:
```

<Warning>
  This image's `latest` tag is not maintained; it predates the `Api` retention
  options used later in this guide. Pin an explicit version, such as `48373fe7`.
</Warning>

Set `KROWN_RPC_URL`, then start the stack:

```bash theme={"system"}
docker compose up -d
docker compose logs -f krown-data-service hotblocks-db
```

Do not expose the EVM data service port publicly. HotblocksDB is the client-facing API in this topology.

### Verify the custom dataset

```bash theme={"system"}
curl --fail http://127.0.0.1:8081/datasets/krown-network
curl --fail http://127.0.0.1:8081/datasets/krown-network/head
```

Run a bounded query:

```bash theme={"system"}
curl --fail --compressed \
  http://127.0.0.1:8081/datasets/krown-network/stream \
  -H "content-type: application/json" \
  -d '{
    "type": "evm",
    "fromBlock": 1,
    "toBlock": 10,
    "includeAllBlocks": true,
    "fields": {
      "block": {
        "number": true,
        "hash": true,
        "timestamp": true
      }
    }
  }'
```

Add `--with-traces`, `--with-statediffs`, and `--use-debug-api-for-statediffs` to the EVM data service only when clients request those fields and the RPC supports the required debug methods.

## Archival plus real-time setup

Use this topology when clients need fast historical backfills and current blocks through the same dataset URL.

The data flow is:

```text theme={"system"}
RPC -> EVM data service -> HotblocksDB -> Portal -> client
SQD Network ---------------------------> Portal -> client
```

### Configure HotblocksDB

Use `Api` retention for datasets whose older blocks are available from the Network:

```yaml title="hotblocks.yaml" theme={"system"}
ethereum-mainnet:
  kind: evm
  retention_strategy: Api
  data_sources:
    - http://ethereum-data-service:3000

base-mainnet:
  kind: evm
  retention_strategy: Api
  data_sources:
    - http://base-data-service:3000

avalanche-mainnet:
  kind: evm
  retention_strategy: Api
  data_sources:
    - http://avalanche-data-service:3000
```

With `Api` retention, HotblocksDB waits for a retention update before it starts ingesting the dataset.

### Configure Hotblocks-retain

`Api` retention means HotblocksDB does not decide on its own when a block may be
dropped. Hotblocks-retain makes that decision from outside: it reads each dataset's
last indexed archival block from the Network scheduler status and posts that height
to HotblocksDB as the new retention lower bound. It moves a number, never block data.

Two consequences follow:

* an `Api` dataset does not ingest anything until the first retention update arrives;
* if the lower bound overtakes what Portal believes the archive holds, the blocks in
  between are in neither source. See [How real-time data is
  served](/en/network/introduction/how-real-time-data-is-served) for where this sits
  in the architecture.

#### List the datasets to track

One instance can track several datasets. The config is a mapping keyed by the
HotblocksDB dataset name:

```yaml title="retained-datasets.yaml" theme={"system"}
ethereum-mainnet: {}
base-mainnet: {}
avalanche-mainnet: {}
```

Every property is optional for a public dataset. Set `name` when the HotblocksDB
dataset name differs from the Network dataset it tracks:

```yaml title="retained-datasets.yaml" theme={"system"}
my-ethereum-copy:
  name: ethereum-mainnet
```

Set `id` when the dataset is absent from the manifest at `--datasets-url`. Private
datasets are not published there, so a name never resolves to a dataset ID — even
when the HotblocksDB name and the Network name are identical. Give the ID instead:

```yaml title="retained-datasets.yaml" theme={"system"}
my-private-chain:
  id: s3://my-private-chain-1
```

This is the same ID the dataset needs in [Portal's own
config](#configure-portal-routing), for the same reason. An entry carrying `id` never
consults the manifest, but `--datasets-url` stays a required flag; point it at the
public manifest as usual.

#### Run the service

```bash theme={"system"}
docker run --rm \
  -v "$PWD/retained-datasets.yaml:/cfg/datasets.yaml:ro" \
  subsquid/data-hotblocks-retain:<version> \
  --hotblocks-url http://hotblocks-db:8081 \
  --status-url https://metadata.sqd-datasets.io/scheduler/mainnet/status.json \
  --datasets-url https://cdn.subsquid.io/sqd-network/datasets.yml \
  --datasets-config /cfg/datasets.yaml \
  --retain-delay-secs 600
```

<Warning>
  This image publishes no `latest` tag. Pin an explicit version.
</Warning>

| Flag | Required | Default | Purpose |
| - | - | - | - |
| `--hotblocks-url` | Yes | — | HotblocksDB instance that receives the lower bound |
| `--status-url` | Yes | — | Network scheduler status document |
| `--datasets-url` | Yes | — | Network datasets manifest, used to resolve names to dataset IDs |
| `--datasets-config` | Yes | — | Path to the mapping above |
| `--datasets-update-interval-secs` | No | `3600` | How often the datasets manifest is refreshed |
| `--retain-delay-secs` | No | `0` | Delay after an assignment's effective time before its height is applied |

On the testnet, use the `tethys` scheduler status and `datasets-testnet.yml`.

As a Compose service alongside the rest of the stack:

```yaml title="compose.yaml" expandable theme={"system"}
  hotblocks-retain:
    image: subsquid/data-hotblocks-retain:<version>
    command:
      - --hotblocks-url
      - http://hotblocks-db:8081
      - --status-url
      - https://metadata.sqd-datasets.io/scheduler/mainnet/status.json
      - --datasets-url
      - https://cdn.subsquid.io/sqd-network/datasets.yml
      - --datasets-config
      - /cfg/datasets.yaml
      - --retain-delay-secs
      - "600"
    volumes:
      - ./retained-datasets.yaml:/cfg/datasets.yaml:ro
    depends_on:
      - hotblocks-db
    restart: unless-stopped
```

Run one Hotblocks-retain per HotblocksDB instance using `Api` retention. Replicas
ingest independently onto their own disks, so each needs its own bound pushed to it.

#### Choose a retain delay

`--retain-delay-secs` defaults to `0`, which leaves no margin. Set it explicitly.

Hotblocks-retain reads the archive head directly from the scheduler status. A Portal
reaches the same head indirectly, through the assignment it refreshes on its own
interval, so every Portal's view of the archive trails Hotblocks-retain's. The delay
is the margin that covers that skew. Below it, the retention bound can pass a
Portal's archival ceiling and open a gap between the two sources while both services
are healthy.

A second reason not to run a tight margin: the scheduler height means a chunk has
been indexed for assignment, not that every assigned worker has finished downloading
it.

600 seconds is a reasonable starting point. It clears a Portal's assignment refresh
several times over while adding little to the retained window. Raise it if you see
gaps; the cost of a generous delay is disk, and the ceiling below bounds that anyway.

#### Bound the window from above

The lower bound only advances when the archive advances. If archival ingestion stalls,
the bound stops moving and the window grows for as long as the stall lasts. Cap it:

```yaml title="hotblocks.yaml" theme={"system"}
ethereum-mainnet:
  kind: evm
  retention_strategy:
    Api:
      max_blocks: 7200
  data_sources:
    - http://ethereum-data-service:3000
```

The retained window is then `max(lower bound, head - max_blocks) .. head`, so
`max_blocks` bounds disk regardless of what the archive does. It is a soft cap:
trimming drops whole chunks, so part of the first chunk can survive it.

Size it from the dataset's block rate — roughly a day of blocks is a common starting
point — then check it against how often the archive publishes new chunks for that
dataset. A window shorter than the interval between chunk publications trims below
the archival head between them and opens a gap on a healthy system. Chains with slow
or large chunks need a window wider than a day.

#### Verify

Hotblocks-retain applies a bound by posting to HotblocksDB, which turns an `Api`
dataset's runtime strategy from `None` into `FromBlock`. Read it back:

```bash theme={"system"}
curl --fail --silent http://127.0.0.1:8081/datasets/ethereum-mainnet/retention
```

Before the first update, the dataset's status carries no data either:

```bash theme={"system"}
curl --fail --silent http://127.0.0.1:8081/datasets/ethereum-mainnet/status
```

```json theme={"system"}
{ "kind": "evm", "retentionStrategy": "None", "data": null }
```

Once a bound has been applied, `data` is populated and `firstBlock` advances on later
polls:

```bash theme={"system"}
curl --fail --silent http://127.0.0.1:8081/datasets/ethereum-mainnet/status \
  | jq '{retentionStrategy, firstBlock: .data.firstBlock, lastBlock: .data.lastBlock}'
```

Then confirm the two sources still meet. Portal names the source that answered in the
`x-sqd-data-source` response header: `network` for archival workers, `real_time` for
HotblocksDB. A request starting just above the archival head should return blocks
under `real_time` — substitute your own `fromBlock`:

```bash theme={"system"}
curl --fail --silent --dump-header - --output /dev/null \
  http://127.0.0.1:8000/datasets/ethereum-mainnet/stream \
  -H "content-type: application/json" \
  -d '{
    "type": "evm",
    "fromBlock": 25790715,
    "includeAllBlocks": true,
    "fields": { "block": { "number": true } }
  }'
```

An empty response means the request fell into a gap between the two sources.

### Configure Portal routing

Declare the HotblocksDB URL inside every dataset's `real_time` section:

```yaml title="mainnet.config.yml" theme={"system"}
hostname: http://0.0.0.0:8000
sqd_network:
  datasets: https://cdn.subsquid.io/sqd-network/datasets.yml
  metadata: https://cdn.subsquid.io/sqd-network/mainnet/metadata.yml
  serve: manual
datasets:
  ethereum-mainnet:
    real_time:
      kind: evm
      url: http://hotblocks-db:8081
  base-mainnet:
    real_time:
      kind: evm
      url: http://hotblocks-db:8081
  avalanche-mainnet:
    real_time:
      kind: evm
      url: http://hotblocks-db:8081
```

<Warning>
  `hotblocksDB` is not a valid top-level Portal option. A self-hosted Portal
  (no `auth` block configured) refuses to start when the config has an
  unrecognized top-level key, and the error names the key. Set
  `datasets.<name>.real_time.url` for every real-time dataset instead.
</Warning>

Portal uses the dataset name as the HotblocksDB dataset name by default. Set `real_time.dataset` only when the names differ.

Portal resolves each dataset key against the Network manifest at `sqd_network.datasets`. A private dataset is not published in that manifest, so declare its ID:

```yaml title="mainnet.config.yml" theme={"system"}
datasets:
  my-private-chain:
    sqd_network:
      dataset_id: s3://my-private-chain-1
    real_time:
      kind: evm
      url: http://hotblocks-db:8081
```

Use `dataset_name` instead when the dataset is in the manifest under a name other than the config key.

### Verify routing

```bash theme={"system"}
curl --fail http://127.0.0.1:8000/datasets/ethereum-mainnet
curl --fail http://127.0.0.1:8000/datasets/base-mainnet
curl --fail http://127.0.0.1:8000/datasets/avalanche-mainnet
```

Each response should report `real_time: true`. Run historical and near-head stream queries to verify that both sources are reachable.

## Put a custom network behind Portal

A network does not need an SQD Network archive to appear in Portal. Declare only a real-time source:

```yaml title="mainnet.config.yml" theme={"system"}
hostname: http://0.0.0.0:8000
sqd_network:
  datasets: https://cdn.subsquid.io/sqd-network/datasets.yml
  metadata: https://cdn.subsquid.io/sqd-network/mainnet/metadata.yml
  serve: manual
datasets:
  krown-network:
    real_time:
      kind: evm
      url: http://hotblocks-db:8081
```

The corresponding `krown-network` dataset must exist in HotblocksDB.

Choose the client endpoint based on the required API:

| Client need | Endpoint |
| - | - |
| Real-time streams and head information only | Connect directly to HotblocksDB |
| One endpoint for the Network and custom datasets | Connect through Portal |
| Portal metadata, dataset listing, or timestamp lookup endpoints | Connect through Portal |

Squid SDK and Pipes SDK clients can use either endpoint for supported streaming operations.

## Token requirements and compute units

The minimum token locking requirement for a Portal instance is 1,000,000 `SQD`. This should be enough for most use cases. If you want to know the exact capabilities this gives you, read on.

The rate limiting mechanism of SQD Network relies on the concept of a *compute unit*, or CU for short. CUs do not map directly to the amount of data fetched by a Portal instance; instead, they (roughly) represent the amount of work that the network does internally while serving the Portal instance's requests.

<Accordion title="Understanding compute units (CU)">
  Network datasets are partitioned by block number. Dataset chunks are randomly distributed among [worker nodes](/en/network/worker).

  When a Portal instance receives a data request, the following happens:

  1. The Portal instance forwards the request to several workers that hold the chunks of the relevant dataset
  2. The workers execute the request separately on each chunk
  3. Workers send the results back to the Portal instance. For lightweight queries, they send one response for each dataset chunk; however, if any response exceeds 100 Mbytes, it's split into several parts
  4. The Portal instance concatenates the workers' replies and serves a continuous stream of data to the user

  Each response made by any of the workers in step 3 spends exactly 1 CU.
</Accordion>

The more `SQD` you lock and the greater the lockup period is, the more CUs you get. Currently, each locked `SQD` generates 1-3 CUs at the beginning of each epoch, depending on the lockup period.

In principle, any valid amount of locked `SQD` generates an infinite amount of CUs. However, if the rate at which your queries consume CUs exceeds the rate at which they are produced, your app will be throttled. To avoid that, you may want to understand how many CUs your queries spend.

There is currently no tool for estimating the number of CUs a query needs before running it. However, on EVM you can use the following formula to get an [order of magnitude](https://en.wikipedia.org/wiki/Order_of_magnitude) estimate:

```text theme={"system"}
10^-5 * range_length_blocks * transactions_per_block
```

Notes:

* This assumes lightweight queries - that is, queries that fetch much less data than the total dataset size. For heavyweight queries multiply the estimate by a factor of 2-5.
* This is a rough estimate. Multiply it by ten to get a somewhat safe figure. If you want to minimize your `SQD` lockup, start at that safe figure, then measure the actual amount of CUs you spend and reduce the lockup accordingly.
* If your network has an Etherscan-style explorer, you can estimate the `transactions_per_block` by visiting its front page, reading the "Transactions" stat and dividing it by the "Last finalized block" height.

<Accordion title="How the estimate is calculated">
  For a lightweight query, the amount of CUs spent is determined by how many dataset chunks the network needs to examine to process it. The ingester creates chunks of roughly the same size within each dataset. Since the amount of data per block is roughly proportional to the number of transactions in that block, we can assume that the number of chunks in any given range is proportional to the number of transactions per block.

  Extrapolating from the Ethereum dataset:

  ```text theme={"system"}
  num_chunks_in_range = range_length * (eth_chunks / eth_height) * (chain_txs_per_block / eth_txs_per_block)
  ```

  Where:

  * `eth_chunks` = 3.1e4
  * `eth_height` = 2.1e7
  * `eth_txs_per_block` = 1.2e2

  Multiplying all the known values together and rounding to one decimal digit, we get the `1e-5` coefficient of the final formula.

  **Important assumption:** This assumes all EVM datasets have the same chunk size as Ethereum. In reality, chunk sizes vary between 50-1000 Mbytes. Ethereum's chunk size is roughly 500 Mbytes, so expect the estimate to be off by a factor of 0.5-10, which is within the "order of magnitude" definition.

  **Heavyweight queries:** Scale the same way but may spend more than one CU per chunk. The heaviest possible queries (fetching the whole dataset) on Ethereum consume roughly 5 CUs per chunk.
</Accordion>

Now, if your queries consume `X` CUs each and you run them once per `Y` epochs, you need to lock up at least this much `SQD`:

```text theme={"system"}
X / (Y * boost_factor)
```

Here, `boost_factor` is a multiplier ranging from 1 to 3 depending on the lockup length.

## Redundancy and capacity

* Run one EVM data service per RPC provider and list all providers under the dataset's `data_sources`.
* Run multiple Portal replicas behind a load balancer. Portal is stateless. Replicas can share a single wallet and a single `SQD` lock: register one peer ID per replica from the same wallet.
* Give every HotblocksDB replica its own persistent volume. Replicas ingest independently and do not share RocksDB files.
* Run Hotblocks-retain alongside every HotblocksDB replica that uses `Api` retention.
* Pin tested container versions or digests for production instead of tracking `latest` automatically.
* Protect Portal peer keys, RPC credentials, and internal service ports.
* Monitor `/ready`, `/metrics`, dataset heads, disk usage, and RPC error rates.

## Troubleshooting

<h3 id="what-are-the-consequences-of-losing-my-key-file-getting-it-stolen">
  What are the consequences of losing my key file / getting it stolen?
</h3>

If you lose your key file, you won't be able to run your Portal instance until you generate and register a new one. If your key file is stolen, the perpetrator can cause connectivity issues, effectively creating downtime for your Portal instance.

To recover:

1. Unregister your Portal instance on the [portals page](https://network.sqd.dev/portals).
2. Generate a new key file.
3. Register the new portal peer ID.

### Portal does not start after adding a real-time source

A self-hosted Portal (no `auth` block configured) refuses to start when the
config has an unrecognized top-level key such as `hotblocksDB`. Check
`docker compose ps` for a portal container that keeps restarting, then its
logs for an error that names the key. The `0.8.1` image pinned in the
repository's `docker-compose.yml` reports:

```text theme={"system"}
unknown field `hotblocksDB`
```

Current releases report:

```text theme={"system"}
unrecognized config key: hotblocksDB
```

Remove the top-level `hotblocksDB` field and set `datasets.<name>.real_time.url` instead.

### HotblocksDB does not ingest an `Api` dataset

`Api` retention does not start until the first retention update arrives. Check
`/datasets/<name>/status` on HotblocksDB: `"retentionStrategy": "None"` with
`"data": null` means no update has landed yet.

Confirm Hotblocks-retain is running, that the dataset key in its config exactly
matches the HotblocksDB dataset name, and that it can reach the HotblocksDB, status,
and datasets URLs. It logs and retries on every failed fetch, so a repeating `failed
to fetch status` or `failed to refresh datasets manifest` in its logs is the answer.
`dataset not found in manifest, skipping` means the name does not resolve to a
Network dataset. For a private dataset this is expected — it is not published in the
manifest, so set `id` for that entry. Otherwise set `name`.

### Clients get no data near the chain head

The requested start is above the archival head but below HotblocksDB's first retained
block, so neither source covers it. Portal reports this as no data rather than
skipping ahead to the first retained block. See [Reliability and
monitoring](/en/network/introduction/reliability-and-monitoring).

Compare `firstBlock` from `/datasets/<name>/status` against the archival head Portal
is working from. If the former is higher, the retention bound has overtaken Portal:
raise `--retain-delay-secs`, and check that `max_blocks` is wider than the dataset's
chunk publication interval.

### Traces or state diffs are missing

Confirm the EVM data service was started with the matching flags and the RPC supports `debug_traceBlockByHash`. Do not enable expensive fields that clients do not query.

### Disk grows continuously

Inspect `retention_strategy`. `FromBlock` retains every block from the configured height. Use a bounded `Head` window for real-time-only access or `Api` retention when the Network supplies history.

### One RPC failure stops current data

Run a data service for a second provider and add both service URLs to the HotblocksDB dataset. Test provider failure before relying on the setup in production.

### The portal is slower than expected

Portal performance metrics are exposed at the `/metrics` endpoint. Check the throttling statistics:

```bash theme={"system"}
curl --compressed http://127.0.0.1:8000/metrics | grep throttled
```

Lower values of `portal_stream_throttled_ratio_sum` indicate better performance. If throttling is high, consider locking more `SQD` tokens to increase your [compute unit allocation](#token-requirements-and-compute-units).

## Source and API references

* [SQD Portal repository](https://github.com/subsquid/sqd-portal)
* [Upstream self-hosting reference](https://github.com/subsquid/sqd-portal/blob/main/docs/self-hosting/self_hosting.md)
* [Portal EVM API](/en/portal/evm/api)
* [Local EVM setup](/en/data/evm-local-setup/overview)


## Related topics

- [Portal pricing](/en/portal/pricing.md)
- [Portal: Blockchain Data API](/en/portal/overview.md)
- [Local EVM Devnet Setup](/en/data/evm-local-setup/overview.md)
- [ADI Testnet](/en/data/evm/adi-testnet.md)
- [Arc](/en/data/evm/arc-mainnet.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.