feat(store-indexer,store-sync): filter findAll by tableIds #1572
Conversation
|
holic
changed the title
|
we should probably default-include all store/world tables since weird things happen if these are ignored/excluded |
|
can also filter events by table IDs |
|
can't filter logs by topics/table IDs yet: #1658 |
| const includedTableIds = new Set(tableIds.length ? [...internalTableIds, ...tableIds] : []); | ||
| const tables = (await getTables(database)) | ||
| .filter((table) => address == null || getAddress(address) === getAddress(table.address)) | ||
| .filter((table) => !includedTableIds.size || includedTableIds.has(table.tableId)); |
There was a problem hiding this comment.
general q, not related to this PR: should we combine this adapter with the one defined in sqlite?
On the same note, a couple lines below this we have const sqliteTable = builtTable(table), should that be postgresTable? Is it the same object type? (Can't comment directly on it bc it didn't change in this PR)
There was a problem hiding this comment.
I would like to, but there's some API differences between drizzle's postgres and sqlite adapters that makes it hard to combine these.
Also the sqlite patterns here are a bit behind - I cleaned things up in the postgres, but didn't go back to refactor sqlite to match.
| startBlock: initialStartBlock = 0n, | ||
| maxBlockRange, | ||
| initialState, | ||
| indexerUrl, | ||
| }: CreateStoreSyncOptions<TConfig>): Promise<SyncResult> { | ||
| const includedTableIds = new Set(tableIds.length ? [...internalTableIds, ...tableIds] : []); |
There was a problem hiding this comment.
it looks like we actively pass internalTableIds here, but also always include it internally in the query adapter - doesn't really hurt bc it's a set, but feels like it could be confusing (e.g. someone removing internalTableIds here and then wondering why they still show up in the results). What are your thoughts on only putting it in one of the two places and making it an explicit assumption? (e.g. "findAll always returns internal tables", or "findAll returns only the tables passed in the query, it is recommended internal tables are always included in the query")
There was a problem hiding this comment.
I think making the indexer dumber (uses whatever you pass in for table IDs) makes sense, and only the store-sync includes internal table IDs.
|
|
||
| export const storeTableIds = Object.keys(storeConfig.tables).map((name) => | ||
| resourceIdToHex({ | ||
| type: storeConfig.tables[name as keyof typeof storeConfig.tables].offchainOnly ? "offchainTable" : "table", |
There was a problem hiding this comment.
as keyof typeof storeConfig.table sad that typescript can't infer that from Object.keys(storeConfig.tables) 😢
There was a problem hiding this comment.
agreed! I think lodash has a thing for this, just waiting for them to finish their TS rewrite
pulled out of #1571