Helper functions to simplify writing social indexers. #381
Conversation
gabehamilton
commented
Nov 9, 2023
•
gabehamilton
commented
- See linked issue for the proposal that this implements.
- See Test Indexer
…ith decoded args.
|
Please weigh in on naming. One alternative would be to add the extraction methods to the |
| .filter((operation) => operation?.methodName === method) | ||
| .map((functionCallOperation) => ({ | ||
| ...functionCallOperation, | ||
| args: base64DecodeFunction(functionCallOperation.args), |
There was a problem hiding this comment.
It might be possible for the decode to fail depending on the structure of args. I vaguely recall a block which failed there. I would wrap this in a try catch and return an empty result. The user is not able to control the contents of a block or this function code. So, if it throws errors, they need to be aware that some blocks will crash their code and stop their indexer if they don't handle the error. Or we can just handle it, which I feel is the better option.
| db: this.buildDatabaseContext(account, schemaName, schema, blockHeight), | ||
| base64Decode: base64DecodeFunction, | ||
| getFunctionCalls: getFunctionCallsFunction, | ||
| getSocialOperations: (block, operation) => { |
There was a problem hiding this comment.
Fortunately, I do actually have a test block for this piece of code. The block 100505912 has social posts with malformed data. As a result, the below filter and map functions will actually fail, throwing an uncaught error that will stop your indexer. You can run that block against the indexer you made and see that it does in fact fail. The fix I did was to simply wrap the calls in a try catch. You can see the fix here: https://github.com/near/near-discovery-components/blob/develop/indexers/social_feed/social_feed.js#L271-L283
| const contract = 'social.near'; | ||
| const method = 'set'; | ||
| return getFunctionCallsFunction(block, contract, method) |
There was a problem hiding this comment.
nit: Can't we simply put the contract and method in the function call, since they're constants anyway? Otherwise, if we want to keep the variables to declare the constants, we should place them as a class level constant with the names SOCIAL_CONTRACT and SET_METHOD. Or at least rename the two you have here to that.
|
Regarding the naming, they look fine to me. For the socialOperations one, I have a question. The name implies there are different operations you can execute on the social.near contract. I have only seen the set method. What other methods are used commonly? I think I would like to keep socialOperations in this. But decode and get functions could possibly go in Block. I'd say we can keep the PR as is for now and migrate if we want to, when we look into supporting things like other Actions. |
Maybe this needs a naming discussion as well. It's not the method it's the top-level data element being stored. Maybe this should be called path and arguments like |
|
Gotcha. My comment was mainly around if it should be socialOperations or socialPosts (as social post is what its called in the social feed indexer). But, from what you just explained, socialOperations is probably better. I think path would make it seem more like an API. I think going as far as paths is reaching a bit too far. We would need to keep that up to date. I would say its better to let the user handle parsing the data (such as seeing if a post contains a like, or a post) themselves. We just strip out parts of the block that, if you call the function, we know you don't want/need. |
|
Adding these methods to For socialOperations, I think this deserves to be a separate library altogether, because this is not generic enough and does not adhere to any NEP standard, and therefore subject to change. So maintaining it inside near-primitives will involve version management, migrations, etc. Perhaps for now you can keep them as tidy snippet of code you keep in gist and docs, so that users can copy-paste it into their indexers? |
|
To be moved to |
