Search by group-id to get all Transactions within Group

[ryanRfox]

As an algorand-indexer user, I'd like the ability to search by "group-id" so I may receive the complete set of confirmed transactions matching this group-id.

[ryanRfox]

Possible solution:
Within /v2/transactions define "group-id" as a searchable parameter. Return each "transaction" within the "transactions" array.

[brianolson]

The currently expected workaround is to fetch all the transactions for a round and then filter them by group id in client code.

[ian-algorand]

Some design is needed still on this.

[brianolson]

question 0: how did you get the group-id such that you want to search for it?

If you submitted the group you can have followed it through to being committed and then know the round it was committed in and do the above workaround of fetching the txns for a round and filtering client side on group-id. We could add server side filtering; it would be pretty efficient to query for in current infrastructure if a round number was required to be submitted when querying to filter by group-id.

The expensive full service option is to add another index on the txn table comparable to the index by txid. I will habitually push back against this solution because I worked on a product that became cost-prohibitive and too slow because of all the indexes; so I want to be very cautious about any new index added. In my opinion, group-id, like txid, is not a good primary key to fetch transactions by. The best way to look up transactions is by round or by address.

[brianolson]

Alternate possible implementation: if a query returns a single txn or some txns all of the same group-id, have a flag ?expand-to-group=1 or similar that would result in the server internally doing the second query for everything by group-id. Already having a txn to refer to the system would have everything it needs for an efficient query of neighboring txns in the same group. expand-to-group=1 would do nothing if the return set includes multiple group-ids or any txns with empty group-id.

[jeapostrophe]

I have a few questions and comments on @brianolson 's proposed workaround.

First, I do not know if the Indexer returns the group txn index. The spec --- https://developer.algorand.org/docs/reference/rest-apis/indexer/#transaction --- includes the "intra-round-offset" field. Is that intended to be the same thing as the "GroupIndex"?

Second, it would be very useful for the indexer to have the invariant that the transactions would always be returned in the group order, so that when I do response.transactions[i], I get transaction number i.

Third, is true that all rounds include only one transaction group? I was under the impression that a round was a block and a block could contain many transaction groups.

Fourth, in my scenario, I do not know what the group index is, I only know that it happened between round A and B and that it involves a transfer from X to Y. Because of #210 I can only include either X or Y in my query and then once I get it, only then do I know what round it happened in or what the group index is. Under @brianolson 's proposal, I now have to make two round-trips to the Indexer to answer my query. If all we care about is performance and not the developer experience, even then I don't see why the indexer should prefer that I sent it two requests rather than one.

[Blackglade]

Was there ever any update on this or if it's still being considered?

[aorumbayev]

Any updates on this issue?
Despite being a very basic use case, having that option available on indexer would be superb!

[gidonkatten]

This would be very useful to have!

[algoanne]

would this be useful as the following?
you can search by group-id if you also provide the block that the group was confirmed in.

@jeapostrophe @Blackglade @gidonkatten @aorumbayev

[jeapostrophe]

I don't need this anymore, but in my original use, I didn't know the block it was in either.

[gidonkatten]

@algoanne it will be useful in the sense that I would need two queries to go from a given transaction to its whole transaction group:

1. Get the block number of one of the transactions in the group using txId
2. Get all transactions in group using groupId and block number

[Blackglade]

you can search by group-id if you also provide the block that the group was confirmed in.

If i know the block number, it's a fairly straight forward process of extracting the groupID from it as I can just filter on the txns in a block and only pick the one's with a matching group id. What i was really looking for was an index on the groupID allowing me to search for any without knowing the specific block

[chrisdewa]

I just can't believe this is not yet featured. Filter by transaction group ID is a must for lots of projects like algoexplorer, defly, Algoscout, etcetera.

Please make it possible search by group-id

[brianolson]

I have to go back to my question of Sep 2020; Why group id? The best way to refer to a committed transaction is by round and offset within round. A group is best as a range of offsets within a round.

[intelli-scripts]

+1 for this.
we have a use case for the explorer. given a group-id, we need to show a list of all the transactions in that group.

[eodgooch]

+1

I have to go back to my question of Sep 2020; Why group id? The best way to refer to a committed transaction is by round and offset within round. A group is best as a range of offsets within a round.

This is quite an obtuse opinion. There are 2 years worth of people requesting this feature. In my case I need to get all the transactions from a Tinyman LP. Currently I need to query for the transactions by assetId, then for each transaction I need to get the block & make another query then filter results. All the while ensuring I don't rate limit myself querying purestake api because now I need to make many more api requests to get all the data I need. That's a lot of unnecessary bytes. What would 1980 Bill Gates think!

I just can't believe this is not yet featured. Filter by transaction group ID is a must for lots of projects like algoexplorer, defly, Algoscout, etcetera.

Please make it possible search by group-id

Yes, pretty please!

[tamimehsas]

Try getting the transaction id using txn.get_txid() while looping through the group of transactions.

from algosdk.account import generate_account
from algosdk.future.transaction import PaymentTxn, assign_group_id
from algosdk.v2client.algod import AlgodClient

algod_token = 'a' * 64
algod_server = "http://127.0.0.1:4001"
algod_client = AlgodClient(algod_token, algod_server)

sk, pk = generate_account()

group = []

sp = algod_client.suggested_params()
sp.flat_fee = True
sp.fee = 1_000

for n in range(10):
    group.append(
        PaymentTxn(
            pk,
            sp,
            pk,
            0,
            note=str(n),
        )
    )

assign_group_id(group)

for tx in group:
    print(tx.get_txid())

[Argonaut5000]

Any work on this?