Dubhe Rust Indexer for Sui
The Dubhe Rust Indexer (dubhe-indexer
) is a high-performance indexing solution for Sui blockchain data, built in Rust for optimal performance and reliability. It provides real-time data indexing with PostgreSQL database storage and integrates seamlessly with the GraphQL server for querying capabilities.
Getting Started
Installation
The Dubhe Rust Indexer is distributed as a pre-compiled binary. You can install it using the Dubhe CLI:
# Check and install required tools including dubhe-indexer
npx @0xobelisk/sui-cli doctor
Alternatively, you can download the binary directly from the GitHub releases.
Prerequisites
Before using the indexer, ensure you have:
- PostgreSQL Database: The indexer requires a PostgreSQL database for data storage
- Dubhe Configuration: A
dubhe.config.json
file generated from your Dubhe project - Network Access: Connection to the Sui network you want to index
Database Setup
Set up your PostgreSQL database and configure the connection:
# Set environment variables for database connection
export DATABASE_URL="postgresql://username:password@localhost:5432/dubhe_indexer"
Basic Usage
Command Line Interface
The indexer is configured through command-line arguments:
dubhe-indexer [OPTIONS]
Options:
-c, --config <CONFIG> Path to the configuration file [default: dubhe.config.json]
-w, --worker-pool-number <NUMBER> Worker pool number [default: 1]
--start-checkpoint <NUMBER> Start checkpoint [default: 0]
--force Force restart: clear indexer database (only for local nodes) [default: false]
--network <NETWORK> Sui network [default: localnet]
--with-graphql Enable GraphQL server integration [default: false]
Network Configuration
The indexer supports different Sui networks:
- localnet: Local development network
- testnet: Sui testnet
- mainnet: Sui mainnet
Basic Examples
- Start indexing on localnet:
dubhe-indexer --config dubhe.config.json --network localnet --worker-pool-number 3
- Start indexing on testnet:
dubhe-indexer --config dubhe.config.json --network testnet --worker-pool-number 3 --start-checkpoint 1000
- Force restart for local development:
dubhe-indexer --config dubhe.config.json --network localnet --force
Configuration
Dubhe Configuration File
The indexer requires a dubhe.config.json
file that defines your contract’s schemas and events. This file is generated from your dubhe.config.ts
:
# Generate JSON configuration from TypeScript config
npx @0xobelisk/sui-cli convert-json --config-path dubhe.config.ts --output-path dubhe.config.json
Example configuration structure:
{
"name": "my_game",
"packageId": "0x123...",
"schemas": {
"player": {
"structure": {
"health": "u64",
"position": "Position"
}
},
"position": {
"structure": {
"x": "u64",
"y": "u64"
}
}
},
"events": {
"player_moved": {
"fields": {
"player_id": "address",
"new_position": "Position"
}
}
}
}
Environment Variables
Configure the indexer using environment variables:
# Database connection
export DATABASE_URL="postgresql://username:password@localhost:5432/dubhe_indexer"
# Optional: Custom RPC endpoints
export SUI_RPC_URL="https://custom-rpc-endpoint.com"
# Optional: Logging level
export RUST_LOG="error"
Integration with GraphQL Server
The indexer works seamlessly with the Dubhe GraphQL Server for querying indexed data:
# Start indexer with GraphQL integration
dubhe-indexer --config dubhe.config.json --network localnet --with-graphql
When GraphQL integration is enabled, the indexed data becomes available through the GraphQL API endpoints. See the GraphQL Server documentation for more details.
Querying Indexed Data
Using GraphQL Client
For more advanced querying capabilities, use the GraphQL client:
import { createDubheGraphqlClient } from "@0xobelisk/graphql-client";
const client = createDubheGraphqlClient({
endpoint: "http://localhost:4000/graphql",
subscriptionEndpoint: "ws://localhost:4000/graphql",
});
// Query with advanced filtering
const players = await client.queryTable("player", {
where: {
health: { greaterThan: 50 },
},
orderBy: [{ field: "health", direction: "DESC" }],
first: 10,
});
Performance Optimization
Worker Pool Configuration
Adjust the worker pool size based on your system resources:
# For high-throughput indexing
dubhe-indexer --worker-pool-number 5 --network mainnet
# For development (lower resource usage)
dubhe-indexer --worker-pool-number 1 --network localnet
Checkpoint Management
Control indexing start point for efficiency:
# Start from a specific checkpoint (useful for partial re-indexing)
dubhe-indexer --start-checkpoint 1000000 --network mainnet
# Start from latest checkpoint (default behavior when start-checkpoint is 0)
dubhe-indexer --start-checkpoint 0 --network mainnet
Monitoring and Troubleshooting
Logging
The indexer provides detailed logging for monitoring:
# Enable debug logging
RUST_LOG=debug dubhe-indexer --config dubhe.config.json
# Enable info logging (recommended for production)
RUST_LOG=info dubhe-indexer --config dubhe.config.json
Common Issues
-
Database Connection Errors:
- Verify PostgreSQL is running and accessible
- Check DATABASE_URL environment variable
- Ensure database user has proper permissions
-
Network Connection Issues:
- Verify network connectivity to Sui RPC endpoints
- Check firewall settings
- For localnet, ensure local Sui node is running
-
Configuration Errors:
- Validate
dubhe.config.json
format - Ensure package ID matches deployed contract
- Verify schema definitions match contract structure
- Validate
Force Restart
For local development, you can force restart the indexer:
# This clears the indexer database and starts fresh (localnet only)
dubhe-indexer --config dubhe.config.json --network localnet --force
Warning: The
--force
flag is only supported for localnet and will clear all indexed data.
Production Deployment
Docker Deployment
For production deployments, consider using Docker:
FROM ubuntu:22.04
# Install dependencies
RUN apt-get update && apt-get install -y \
postgresql-client \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# Copy dubhe-indexer binary
COPY dubhe-indexer /usr/local/bin/
RUN chmod +x /usr/local/bin/dubhe-indexer
# Copy configuration
COPY dubhe.config.json /app/
WORKDIR /app
CMD ["dubhe-indexer", "--config", "dubhe.config.json", "--network", "mainnet"]
Environment Configuration
For production, use environment variables for sensitive configuration:
# Database connection
export DATABASE_URL="postgresql://user:password@db-host:5432/dubhe_prod"
# Network configuration
export SUI_RPC_URL="https://fullnode.mainnet.sui.io:443"
# Logging
export RUST_LOG="info"
Integration Examples
Full Stack Setup
Complete setup with all components:
# 1. Start PostgreSQL database
docker run -d \
--name dubhe-postgres \
-e POSTGRES_DB=dubhe_indexer \
-e POSTGRES_USER=dubhe \
-e POSTGRES_PASSWORD=password \
-p 5432:5432 \
postgres:15
# 2. Set database URL
export DATABASE_URL="postgresql://dubhe:password@localhost:5432/dubhe_indexer"
# 3. Start indexer
dubhe-indexer --config dubhe.config.json --network testnet --with-graphql
# 4. Start GraphQL server (in another terminal)
# See GraphQL Server documentation for details
Development Workflow
Typical development workflow:
# 1. Generate configuration
npx @0xobelisk/sui-cli convert-json --config-path dubhe.config.ts
# 2. Start local services
npx @0xobelisk/sui-cli wait --localnet
# 3. Start indexer
dubhe-indexer --config dubhe.config.json --network localnet --force
# 4. Develop and test your application