---
name: SQD
description: Use when building blockchain data indexers, querying on-chain data
  via HTTP API, or deploying data pipelines to production. Reach for SQD when
  working with Portal API for raw blockchain data access, Pipes SDK for custom
  TypeScript data pipelines, or Squid SDK for full-featured indexers with
  GraphQL APIs.
metadata:
  mintlify-proj: sqd
  version: "1.0"
---

# SQD Skill

## Product summary

SQD is a blockchain data platform with three layers: **Portal API** (HTTP interface for querying 120+ networks), **Pipes SDK** (lightweight TypeScript toolkit for custom pipelines), and **Squid SDK** (full indexing framework with PostgreSQL and auto-generated GraphQL). All three share the same underlying Portal data source. Use Portal for simple queries from any language, Pipes for embedding data pipelines in existing apps, and Squid for self-contained indexing services. Deploy Squid indexers to **SQD Cloud** with `sqd deploy` for managed hosting. Key files: `squid.yaml` (deployment manifest), `schema.graphql` (entity definitions), `src/main.ts` or `src/index.ts` (processor/pipe logic), `package.json` (dependencies). CLI: `sqd init`, `sqd build`, `sqd deploy`, `sqd logs`. Primary docs: https://docs.sqd.dev

## When to use

- **Portal API**: Query blockchain data over HTTP without setup. Use for backfills, wallet history, contract event extraction, or any language (Python, Go, Rust, etc.).
- **Pipes SDK**: Build TypeScript data pipelines that stream Portal data into ClickHouse, PostgreSQL, BigQuery, or Parquet. Embed in existing apps; no GraphQL layer.
- **Squid SDK**: Create a self-contained indexing service with PostgreSQL storage and auto-generated GraphQL API. Deploy to SQD Cloud for production indexing.
- **SQD Cloud**: Deploy Squid indexers for managed hosting, monitoring, scaling, and zero-downtime updates. Use slots and tags for production workflows.

Trigger conditions: User asks to "index blockchain data," "query on-chain events," "build a data pipeline," "migrate from The Graph," or "deploy an indexer to production."

## Quick reference

### Portal API (HTTP)

| Task | Command |
|------|---------|
| Query EVM blocks | `POST https://portal.sqd.dev/datasets/ethereum-mainnet/stream` with `type: "evm"`, `fromBlock`, `toBlock`, `fields` |
| Query Solana slots | `POST https://portal.sqd.dev/datasets/solana-mainnet/stream` with `type: "solana"`, `fromBlock`, `fields` |
| Stream finalized data | Use `/finalized-stream` endpoint instead of `/stream` |
| Get dataset list | `GET https://portal.sqd.dev/datasets` |

### Pipes SDK (TypeScript)

| Task | Command |
|------|---------|
| Scaffold project | `pnpx @subsquid/pipes-cli@beta init` |
| Run locally | `docker compose up -d && pnpm run dev` |
| Decode EVM events | Use `evmEventDecoder()` with contract ABI and event filters |
| Write to PostgreSQL | Use `drizzleTarget()` with Drizzle table definitions |
| Write to ClickHouse | Use `clickhouseTarget()` |

### Squid SDK (TypeScript)

| Task | Command |
|------|---------|
| Scaffold project | `sqd init hello-squid -t evm` |
| Build | `sqd build` |
| Start database | `sqd up` |
| Run locally | `sqd run .` |
| Generate TypeORM models | `sqd codegen` |
| Create migration | `sqd migration:generate` |
| Deploy to Cloud | `sqd deploy .` |
| View deployment | `sqd view -n <name> -s <slot>` |
| View logs | `sqd logs -n <name> -s <slot>` |

### SQD Cloud Manifest (squid.yaml)

```yaml
manifest_version: subsquid.io/v0.1
name: my-squid
build:
  node_version: "20"
deploy:
  addons:
    postgres:
      version: "18"
  processor:
    cmd: ["sqd", "process:prod"]
  api:
    cmd: ["sqd", "serve:prod"]
scale:
  dedicated: true
  addons:
    postgres:
      profile: medium
      storage: 100Gi
```

### Schema Definition (schema.graphql)

```graphql
type Transfer @entity {
  id: ID!
  from: String! @index
  to: String! @index
  value: BigInt!
  timestamp: DateTime!
  blockNumber: Int!
}
```

## Decision guidance

| Scenario | Use Portal API | Use Pipes SDK | Use Squid SDK |
|----------|---|---|---|
| Simple one-off queries | ✓ | ✗ | ✗ |
| Non-TypeScript language | ✓ | ✗ | ✗ |
| Embed in existing app | ✗ | ✓ | ✗ |
| Custom database (ClickHouse, BigQuery) | ✗ | ✓ | ✗ |
| Need GraphQL API | ✗ | ✗ | ✓ |
| Deploy to SQD Cloud | ✗ | ✗ | ✓ |
| Self-contained indexing service | ✗ | ✗ | ✓ |
| Lightweight, no framework overhead | ✗ | ✓ | ✗ |

| Deployment Choice | When to Use |
|---|---|
| Managed Portal endpoints | Getting started, dev/test, shared capacity |
| Self-hosted Portal | Private networks, data sovereignty, custom chains |
| SQD Cloud (Squid SDK) | Production indexers, managed infrastructure, monitoring |
| Self-host Pipes SDK | Custom infrastructure, non-Node.js hosts, Railway/Docker |

## Workflow

### Build a Squid SDK indexer

1. **Scaffold**: Run `sqd init hello-squid -t evm` to create a project from the EVM template.
2. **Define schema**: Edit `schema.graphql` to define entities (e.g., `Transfer`, `Account`). Use `@entity` and `@index` decorators.
3. **Generate models**: Run `sqd codegen` to generate TypeORM entity classes in `src/model/generated/`.
4. **Configure processor**: Edit `src/main.ts` to set up the data source (Portal or RPC), select fields, and add filters for logs/transactions/traces.
5. **Write handler**: Implement the batch handler to decode events, transform data, and save entities via `ctx.store.save()`.
6. **Test locally**: Run `sqd up` to start PostgreSQL, then `sqd run .` to run processor and GraphQL server.
7. **Query GraphQL**: Open `http://localhost:4350/graphql` and test queries.
8. **Deploy**: Create `squid.yaml` with manifest, run `sqd deploy .` to push to SQD Cloud.
9. **Monitor**: Use `sqd logs` and `sqd view` to check sync progress and errors.

### Build a Pipes SDK pipeline

1. **Scaffold**: Run `pnpx @subsquid/pipes-cli@beta init` and answer prompts for network, target database, and templates.
2. **Define schema**: Edit `src/schemas.ts` to define Drizzle table structure.
3. **Configure decoder**: Edit `src/index.ts` to set up event decoder (e.g., `evmEventDecoder()`) with contract addresses and event filters.
4. **Transform data**: Use `.pipe()` to reshape decoded events into table rows.
5. **Set target**: Wire decoder to `drizzleTarget()` or `clickhouseTarget()` with insert logic.
6. **Test locally**: Run `docker compose up -d` for database, then `pnpm run dev` to start pipeline.
7. **Verify data**: Query the target database to confirm rows are landing.
8. **Deploy**: Push to Railway, Docker, or any Node.js host with environment variables for database connection.

### Query with Portal API

1. **Identify dataset**: Check `https://portal.sqd.dev/datasets` for your network (e.g., `ethereum-mainnet`).
2. **Construct request**: POST to `https://portal.sqd.dev/datasets/<dataset>/stream` with JSON body specifying `type`, `fromBlock`, `toBlock`, `fields`, and filters.
3. **Handle response**: Read NDJSON response line-by-line; each line is one block.
4. **Handle reorgs**: Use `/finalized-stream` for production; it includes rollback events on chain reorganization.
5. **Retry on 409**: If you get HTTP 409, the chain reorged; reset your cursor and resume from the last finalized block.

## Common gotchas

- **Batch processing is mandatory**: Always accumulate entities and save in one `ctx.store.save()` call per batch, not per event. Saving per event causes 10x slowdown.
- **Schema changes require migration**: After editing `schema.graphql`, run `sqd codegen` and `sqd migration:generate`, then recreate the database locally with `sqd down && sqd up`.
- **Field selection must match handler**: If your handler reads `log.address`, add `address: true` to the `fields.log` in the processor config. Missing fields silently return null.
- **Indexes are critical for GraphQL**: Add `@index` decorators to fields you query frequently (e.g., `from`, `to` in transfers). Without indexes, queries slow down as data grows.
- **Portal streams are NDJSON, not JSON**: Each line is a separate JSON object. Do not try to parse the entire response as a single JSON array.
- **Reorg handling is automatic in Squid SDK**: The processor handles forks automatically. In Pipes SDK, use `onRollback` callbacks for ClickHouse; PostgreSQL targets handle it automatically.
- **Secrets must use `${{ secrets.NAME }}`**: Never hardcode API keys in `squid.yaml`. Use the `secrets` context and store values in SQD Cloud.
- **Migrations are applied at startup**: The processor applies pending migrations before indexing starts. If `deploy.init` is omitted, migrations run as a dependency of the processor command.
- **Slot vs tag confusion**: A `slot` is a deployment instance; a `tag` is a label pointing to a slot. Use tags for production URLs so you can move traffic between slots without changing client code.
- **Database connection pooling**: Set `GQL_DB_CONNECTION_POOL_SIZE` and `DB_CONNECTION_POOL_SIZE` appropriately for your workload. Default is 5; production may need 10–20.
- **Unfinalized blocks can reorg**: By default, Squid SDK processes unfinalized blocks. Use `hotBlocks: false` if you need only finalized data, but this increases latency.
- **TypeORM codegen overwrites models**: Do not edit generated files in `src/model/generated/`. Always regenerate after schema changes.

## Verification checklist

Before submitting work:

- [ ] Schema is defined in `schema.graphql` with `@entity` and `@index` decorators where needed.
- [ ] TypeORM models are generated: `sqd codegen` ran without errors.
- [ ] Processor/pipe configuration matches the data you're extracting (logs, transactions, traces, etc.).
- [ ] Batch handler accumulates entities and saves once per batch, not per event.
- [ ] Local test passes: `sqd run .` (Squid) or `pnpm run dev` (Pipes) indexes data without errors.
- [ ] GraphQL queries work (Squid only): Test at `http://localhost:4350/graphql`.
- [ ] `squid.yaml` is present and valid (Squid Cloud deployments).
- [ ] Secrets are stored in SQD Cloud, not hardcoded in code or manifest.
- [ ] Deployment manifest specifies correct Node.js version, package manager, and resource profiles.
- [ ] No TypeScript errors: `sqd build` completes successfully.
- [ ] Logs are checked for warnings or unexpected behavior: `sqd logs -n <name>`.
- [ ] Sync progress is monitored: processor height is advancing toward chain head.

## Resources

- **Comprehensive navigation**: https://docs.sqd.dev/llms.txt (page index) and https://docs.sqd.dev/llms-full.txt (full content)
- **Portal API quickstart**: https://docs.sqd.dev/en/portal/evm/quickstart
- **Squid SDK quickstart**: https://docs.sqd.dev/en/sdk/squid-sdk/evm/quickstart
- **Pipes SDK quickstart**: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/quickstart
- **SQD Cloud deployment guide**: https://docs.sqd.dev/en/cloud/deployment-guide
- **Deployment manifest reference**: https://docs.sqd.dev/en/cloud/reference/manifest
- **Best practices**: https://docs.sqd.dev/en/cloud/resources/best-practices
- **Troubleshooting**: https://docs.sqd.dev/en/cloud/troubleshooting
- **Migrate from The Graph**: https://docs.sqd.dev/en/sdk/squid-sdk/evm/guides/migration/migrate-from-thegraph

---

> For additional documentation and navigation, see: https://docs.sqd.dev/llms.txt