DPLT-980 Tests for wildcard matching #86
Conversation
…exer rules matching with wildcards.
indexer/test_blocks/fetchBlocks.js
Outdated
| @@ -0,0 +1,34 @@ | |||
| // fetch test blocks from block server | |||
There was a problem hiding this comment.
Can we just add a shell script in indexer_rules_engine rather than adding an entire new project?
The same thing can be achieved with a simple curl command:
curl -o ./blocks/92476362.json https://70jshyr5cb.execute-api.eu-central-1.amazonaws.com/block/92476362`And if we want to make it generic:
#!/bin/bash
mkdir -p ./blocks
# Iterate over all script arguments
for block_id in "$@"
do
curl -o "./blocks/${block_id}.json" "https://70jshyr5cb.execute-api.eu-central-1.amazonaws.com/block/${block_id}"
doneWhich could then be used like so:
./download_blocks 92476362 92476363There was a problem hiding this comment.
Shell script is definitely simpler, which is a good goal. I try to write utilities like this in one of our programming languages since they tend to grow and then become something that needs to be called from an admin console, then by privileged users. Rust would make more sense here but I was trying to move fast.
I could put it in block_server, does that sound better?
There was a problem hiding this comment.
Personally, I still would prefer shell script because:
- It doesn't require bootstrapping and maintaining another project
- It can live close to where it's used
Placing it in block_server puts it even further from it's use 😅
I think it would be trivial to spin this out in to another project when the requirement comes, I'm not a fan of optimising for requirements which aren't fully formed yet.
gabehamilton
changed the title
There was a problem hiding this comment.
@gabehamilton A few questions:
For wildcard matching would *.near , *.account.near work?
Why is it that we want wildcards in the middle of an account name such as app.nea*crowd.near to work?
Also what about just * ? What is the expected behavior here?
|
I think we should go with just wildcards for sub accounts to start in the UI. We can allow more later as people ask for more and as we test the historical processing with wildcards. |
Co-authored-by: Morgan McCauley <[email protected]>
…esult instead of action level.
| @@ -115,3 +115,88 @@ fn build_indexer_rule_match_payload( | |||
| } | |||
| } | |||
| } | |||
|
|
|||
| #[cfg(test)] | |||
| mod tests { | |||
There was a problem hiding this comment.
Can we also test for a wildcard which doesn't match anything to avoid false positives here?
| @@ -2,3 +2,4 @@ | |||
| .env* | |||
| redis/ | |||
| *.log | |||
| /indexer/blocks/ | |||
There was a problem hiding this comment.
The tests explicitly depend on this data, should we just commit it to the repo?
…ngs (via delimeter and common prefixes). MOCK queue_url logs instead of sending SQS messages.
DPLT-980 wildcard and csv
DPLT-980 s3 list operation
DPLT-980 Refactor S3 and SQS operations into their own modules.
This tests the local matching. Future changes and tests for wildcard matching with historical indexed data need #81
Block server option allows CamelCase (default) or snake_case data to be returned.