Algorand Indexer 2.3.4

Highlights

This release adds support for an upcoming protocol upgrade, as well as
fixes the following migration issues for people upgrading from 2.2.x to 2.3.x:

• The migration may have exited with the following error. update account asset, pq: null value in column \"frozen\" violates not-null constraint
• The migration may have completed, but be stuck in error status, reporting error during migration 7 (Compute cumulative account rewards for all accounts.): m7: len(assetDataMap): 77 \u003e 0

Changes

Enhancement

• Update consensus params to support V27

Bug Fixes

• Fix frozen null-constraint during import
• Delete assets from asssetDataMap in migration when updating accounts

Algorand Indexer 2.3.3

Highlights

This is a bug fix release to address an issue where a few blocks on TestNet were erroring due to no deltas for the sender. No migrations are necessary for this change and can be applied directly to 2.3.2.

Changes

Bug Fix

• Make sure there are deltas before attaching the delta object in REST layer

Algorand Indexer 2.3.2

Highlights

This release includes the following:

• Cumulative rewards support is now available in indexer! This is complicated to calculate and requires a non-blocking migration to implement (see upgrade notes below)
• Add REST API endpoints for create/delete fields
• Support for upcoming protocol upgrade
• Many optimizations and bug fixes

Important Changes

Cumulative rewards are now computed and stored in the account rewards field.
Round at which things are added or removed from the database is now reported. There is also a deleted flag for each of these indicating whether the record is currently deleted from the blockchain:
account: created-at-round, closed-at-round
asset: created-at-round, destroyed-at-round
asset holding: opted-in-at-round, opted-out-at-round
application: created-at-round, deleted-at-round
account application: opted-in-at-round, closed-out-at-round

Major Bugs Addressed

• Fix for negative asset integer bug seen on BetaNet
• JSON encoding import issue
• Asset holding accounting did not handle default-frozen = true correctly. This is now fixed.

Known Issues

• This release is not compatible with the other 2.3.0-beta releases due to migration refactoring. A full indexer database rebuild is required to upgrade from 2.3.0-betas.
• Application state rewind has not been implemented and returns a generic error. We are implementing a better error message for this state.

Upgrade Notes (Please Read)

Full Rebuild Required For BetaNet and TestNet indexers

Due to improvements in block importing, a full rebuild is necessary for BetaNet and TestNet indexers. This is not required for MainNet.

Mandatory Migration

Adding cumulative rewards support and some of the data cleanups requires a migration. However, this was written to run asynchronously and will not require downtime. There is minimal impact to query performance while running, but the migration is expected to take around twelve hours to complete and may consume a few GB of RAM. The server can be restarted at any time, and the migration will pick up where it left off. You can observe if the migration has finished by checking the /health REST API endpoint. The percent progress is available in the log file or from standard out if no log file is configured.

Configuration File Name Change

If you were previously using a configuration file with indexer, you will need to rename it. If you had previously been using a file named algorand-indexer, algorand-indexer.yml, or algorand-indexer.yaml, you should rename this file to indexer, indexer.yml, or indexer.yaml. The location of the configuration file has not changed.

Changes

New Features

• Add support for cumulative rewards
• Capture create and close rounds for things that can be created and closed
• Add Create/Delete to REST Response

Enhancements

• Switch to algod v2 client
• Degrade ReadOnly Node Service When Migration is Required
• IndexerDb should block importer until db is available
• Logging Improvements
• Naming cleanup and append to log file instead of overwriting
• remove support for e2etest.py and use e2elive.py
• Pipeline - add mule build docs
• Pipeline - parameterize STAGING in build task
• Added github templates for internal ticketing
• Include txn type into panic message
• Add 'deleted' flag to the database and API
• Add basic benchmark queries against a database to help prevent query performance regression
• Move db.CommitRoundAccounting out of the accounting package
• Better rewind error message
• Modify GetMaxRound to return the accounting_round instead of the max block round
• Validation scripts to verify accounting
• Change GetBlock to retrieve content in a single transaction
• Update to support upcoming consensus protocols
• Support databases behind load balancers
• Don't return special accounts in results
• Add pending rewards to rewards in account APIs

Bug Fixes

• Config file name change
• Fix golint
• Fixing typo in README.md postgress connection string (thanks PSJoe)
• Make sure GetMaxRound returns a value if there are no rounds processed
• Fix asset close accounting issue
• Fix handling of some transactions whose JSON encoding would fail to be received by PostgreSQL
• On account close, delete a bunch of fields out of account.account_data because they'd be cleared by algod
• When local state delta had a single key deletion did not happen
• Round writer check to prevent advancing on import errors
• Special case if multiple writers are accidentally run
• Always send json to PostgreSQL as a Go string not []byte
• Small fixes to pipeline code
• Adding line to copy mdhash to prevent overwriting (#366)
• Many improvements to accounting logic

Algorand Indexer 2.2.2

Increased traffic impacted indexer query performance. This hotfix adds indices to the postgres database tables.

Changes

Enhancement

• Add indexes to speed queries

Algorand Indexer 2.3.1-beta3

2.3.0-beta releases are incompatible with these versions due to migration refactoring, a full indexer database rebuild is required to upgrade from 2.3.0-betas.

Changes

Enhancement

• Modify GetMaxRound to return the accounting_round instead of the max block round

Bug Fix

• Small fixes to pipeline code

Algorand Indexer 2.3.1-beta2

2.3.0-beta releases are incompatible with these versions due to migration refactoring, a full indexer database rebuild is required to upgrade from 2.3.0-betas.

Changes

Enhancement

• Better rewind error message

Bug Fixes

• Round writer check to prevent advancing on import errors
• Special case if multiple writers are accidentally run
• Always send json to PostgreSQL as a Go string not []byte

Algorand Indexer 2.3.1-beta1

Highlights

2.3.0-beta releases are incompatible with these versions due to migration refactoring, a full indexer database rebuild is required to upgrade from 2.3.0-betas.

This release is focused on minor enhancements and bug fixes, including:

Enhancements

• Cumulative rewards are now computed and stored in the account "Rewards" field.
• Round at which things are added or removed from the database are now reported. There is also a 'deleted' flag for each of these indicating whether the record is currently deleted:
  • account: 'created-at-round', 'closed-at-round'
  • asset: 'created-at-round', 'destroyed-at-round'
  • asset holding: 'opted-in-at-round', 'opted-out-at-round'
  • application: 'created-at-round', 'deleted-at-round'
  • account application: 'opted-in-at-round', 'closed-out-at-round'

Bug Fixes

• Fix for negative asset integer bug seen on BetaNet
• JSON encoding import issue

Known Issues

• This release is not compatible with the other 2.3.0-beta releases due to migration refactoring, a full indexer database rebuild is required to upgrade from 2.3.0-betas.
• Application state rewind has not been implemented, and returns a generic error. We are implementing a better error message for this state.
• Multiple writer daemons will cause double accounting leading to invalid data. The only fix for this is a full indexer database rebuild.
• Asset holding accounting does not currently handle default-frozen = true correctly. This currently only effects a single asset on betanet.

Changes

Enhancements

• Add 'deleted' flag to the database and API
• Add basic benchmark queries against a database to help prevent query performance regression
• Move db.CommitRoundAccounting out of the accounting package

Bug Fixes

• Fix asset close accounting issue
• Fix handling of some transactions whose JSON encoding would fail to be received by PostgreSQL
• On account close, delete a bunch of fields out of account.account_data because they'd be cleared by algod
• When local state delta had a single key deletion did not happen

Algorand Indexer 2.3.0-beta2

Highlights

Add REST API endpoints for previously implemented create/delete. If upgrading from pre-2.3.0, please see upgrade notes in 2.3.0-beta1.

Changes

New Features

• Add Create/Delete to REST Response

Enhancements

• Include txn type into panic message

Algorand Indexer 2.3.0-beta1

Highlights

Cumulative rewards support is now available in indexer! This is complicated to calculate, and requires a non-blocking migration to implement (see upgrade notes below).

Upgrade Notes (Please Read)

Mandatory Migration

Adding cumulative rewards support requires a migration. However, this was written to run asynchronously and will not require downtime. There is minimal impact to query performance while running, but the migration is expected to take around five days to complete. The server can be restarted at any time, and the migration will pick up where it left off. You can observe if the migration has finished by checking the /health REST API endpoint. The percent progress is available in the log file or from standard out if no log file is configured.

Configuration File Name Change

If you were previously using a configuration file with indexer, you will need to rename it. If you had previously been using a file named algorand-indexer, algorand-indexer.yml, or algorand-indexer.yaml, you should rename this file to indexer, indexer.yml, or indexer.yaml. The location of the configuration file has not changed.

Changes

New Features

• Add support for cumulative rewards (#166)
• Capture create and close rounds for things that can be created and closed

Enhancements

• Switch to algod v2 client
• Degrade ReadOnly Node Service When Migration is Required
• IndexerDb should block importer until db is available
• Logging Improvements
• Naming cleanup and append to log file instead of overwriting
• remove support for e2etest.py and use e2elive.py
• Pipeline - add mule build docs
• Pipeline - parameterize STAGING in build task
• Added github templates for internal ticketing

Bug Fixes

• Config file name change
• Fix golint
• Fixing typo in README.md postgress connection string (thanks PSJoe)
• Make sure GetMaxRound returns a value if there are no rounds processed

Algorand Indexer 2.2.1

Highlights

A performance issue was identifies in queries to the /v2/accounts endpoint, so the query was optimized.

Changes

• Bug Fixes
  • Fix for long /v2/accounts queries (#248)
  • Typo in README.md postgress connection string (#245)