Backfilling
Last updated
Last updated
Backfill offers an unprecedentedly swift method to sift through the entire history of the Solana blockchain to pinpoint and retrieve specific transactions necessary for your application. Imagine possessing the functionality of command-line tools like cat and grep, but supercharged to operate across our robust infrastructure. By harnessing our system, you can execute these operations without the overhead of significant resource consumption on your end. Combing through transactional data has never been more efficient, saving time and computational expense, ensuring that you can focus on what matters most—analyzing the data, not struggling to acquire it.
Our backfilling tasks are under constant surveillance, with metrics refreshing at one-second intervals. This ensures you have the most up-to-date insights into the performance and progress of your jobs. Developers have two options for accessing these metrics:
On-Demand Access: At any time during the backfilling process, you can request the latest metrics directly, allowing you to integrate this data into your monitoring systems on your schedule.
Webhook Integration: Set up a dedicated webhook URL when you create your job, and our system will push metrics regularly to your endpoint. This seamless integration is designed for developers looking to incorporate metrics into their existing infrastructure without the need for polling.
Whichever method you choose, you'll be equipped with the information you need to optimize your application's performance and monitor your backfilling processes effectively.
Available metrics:
txt_forward
Transactions that got sent to webhook
txt_scanned
Amount of transactions scanned
txt_per_sec
Scan speed
completed_percentage
Progress in percentage
Throughout the execution of your backfilling job, you can expect the job status to transition through several states. To keep track of your job's current status, you have a couple of convenient options:
Fetch the Latest Status: Query our API to retrieve the most recent status information for your job. This is perfect for those who prefer pulling data on demand.
Webhook Notifications: Provide a webhook URL when setting up your job, and our system will proactively send status updates to your endpoint. This method suits those who prefer an automated, push-based approach to stay informed.
The job status moves through the following stages during backfilling operations:
CREATED: Your job is set up and in the queue, awaiting its start.
STARTING: The job is initiated. This stage includes preparing the necessary resources and configurations.
RUNNING: Your backfilling process is active. You can check its progress via real-time metrics.
STOPPING: The job is winding down gracefully, saving its current state.
PAUSED: The job is on hold. You can resume at a future time from this state.
COMPLETED: The job has finished processing the specified range of blocks.
Stay in tune with your backfilling progress by opting for either of these methods for job status updates.
When using webhooks, your endpoint will receive a POST request containing a payload field in the JSON body. This payload will always be an array that holds transaction data, even if batch processing is disabled.
Example Payload:
The transactions are represented in raw, unprocessed format, similar to what you would receive from an RPC node. For advanced parsing, we offer an internal parser capable of interpreting well-known Interface Definition Languages (IDLs). More details on the parsing feature are available in the section.
Starts a backfilling process. If the time range is big, it could take up to a few minutes to start.
44f469559d4cc55170a26031a4a5123eb2Successful response
Metrics can be received on demand via this call or by getting full job information. Also you can defined metrics web hook callback when creating job.
44f469559d4cc55170a26031a4a5123eb2Successful response
Creates a backfilling task that retrieves transactions based on specified filters and subsequently submits them to a predefined URL in the hook's data field.
f469559d4cc55170a26031a4a5123eb2The name of job.
An array of filters to apply. Check the Transaction Filter for details.
Batch webhook calls. Specify the number of transactions to be batched in one call. Defaults to 1
Limit CPU for JOB, one CPU does ~20 000 transaction rescans. Speed can vary based on how many records are picked up and sent back. You can look at it as concurrency or threads, one CPU will always send one hook at the moment. Defaults to 1
Successful response