File based filter configuration

[jfberry]

Have you ever wished you could configure Poracle channels more easily?
This is an experiment that might change quite a bit before finally committed. But there is no database migration so let's keep playing until we get to the right solution.

1. !copy command. You can !copy to <id> <id> to copy the current channel config to another channel(s). You can use names, hook names, or ids. Probably should do #channel mentions but I haven't done that yet. You can !copy from <id> to overwrite this channel with another channel's config.
2. You can define filters (see below), and add them to a channel with !addfilter name.
3. You can define a config file with filters and clones (copies), and execute the whole config file using !channelsync

Filter configuration

The filters folder contains filters. It'd be nice to have a few in the defaults folder.

Here is my hundos.json test filter:

{
  "pokemon":
  {
    "pokemon": [0],
    "min_iv": 100,
    "max_iv": 100
  }
}

I'll document more fully how to use it later (and will make some things easier) - for now it's mostly database fields - you can see src\lib\filters.js - only pokemon currently supported.
One thing you an do is have a filter based on another and override values:

{
  "based_on": "communityday",
  "pokemon":
  {
    "template": "2"
  }
}

channels.json

This file, in the config folder, defines a whole set of channel configs. Clones work like copy to - the source id is first and must be a proper id (?perhaps it should be able to be a webhook name too?). The array of destinations can use the % wildcards like apply and the copy to command.

The filters are the matching channels and then filters that will be applied directly. There is no way to override the areas in here or a template file, I figure those would be preset when you create the channel? Comments please

{
  "clones": {
 //   "id": ["id", "id"]
  },
  "filters": {
    "%iv100": ["hundos"],
    "825767787346657300": ["community2"]
  }
}

[mermao1]

Great ideas.
The covered area (geofence) can be defined in the channel creation (row in humans table).

Would be nice to have a file dedicated for multiple filters (filters.json) which could support rules for each filter.
ie:

{
    "pokemons":{
        "filters":{

            "IV0":{
                "min_iv": 0,
                "max_iv": 0
            },

            "IV90":{
                "min_iv": 90.0,
                "max_iv": 94.9,
                "ignored_pokemons": [1,4,7,10,13,16,19,21]
            },

            "IV95":{
                "min_iv": 95.0,
                "max_iv": 99.9,
                "ignored_pokemons": [1,4,7,10,13,16,19,21]
            },

            "IV100":{
                "min_iv": 100,
                "max_iv": 100
            },

            "rares":{
                "min_iv": 0,
                "pokemons":[130,131,149,176,237,247,289,373,375,376,409,444,445,559,566,567]
            },

            "ultrarare":{
                "min_iv": 0,
                "pokemons":[201,248,480,481,482,610,611,612,635]
            },

            "pvpgreat":{
                "max_rank": 3,
                "min_rank": 10,
                "max_cp": 1500
            },

            "pvpgreatperfect":{
                "max_rank": 1,
                "min_rank": 1,
                "max_cp": 1500
            }
        }
    },
    "raids":{
        "filters":{

            "RaidType1":{
                "min_level": 5, "max_level": 5,
                "min_time_left": 1800
            }
        }
    }
}

So we could have hundreds of different predefined filters in a single file.

channels.json could work as you described, defining which filter contained in filters.json would be used for each channel and a new option: specific template

{
    "pokemons":{
        "%rares":{
            "filter_from_filters_file":["rares"],
            "template_from_dts_file":["raresiv"]
        },

        "newyorkrsubarea":{
            "filter_from_filters_file":["rares"],
            "template_from_dts_file":["raresivnewyork"]
        },

        "825767787346657300":{
            "filter_from_filters_file":["rares"],
            "template_from_dts_file":["raresiv"]
        },

        "%pvpgreat":{
            "filter_from_filters_file":["pvpgreat"],
            "template_from_dts_file":["pvp"]
        },

        "newyorkpvpperfect":{
            "filter_from_filters_file":["pvpgreatperfect"],
            "template_from_dts_file":["perfectpvp"]
        },


    "raids":{

        "raidnewyork":{
            "filter_from_filters_file":["RaidType1"],
            "template_from_dts_file":["raidinnewyork"]
        },

        "%raids":{
            "filter_from_filters_file":["RaidType1"],
            "template_from_dts_file":["raiddts"]
        },

        "49845151654956":{
            "filter_from_filters_file":["RaidType1"],
            "template_from_dts_file":["raiddts"]
        }

    }
}

idk if possible but will be nice if accept uses %multiplechannels / specific-channel-name or channelID

Just to be clear to future participants in this conversation: a lot of my ideas I saw in PA.
I'm not bringing it here as something new, but good functions that would make Poracle even better.

[jfberry]

Thanks - that is very close to what I'm trying to do. Rather than have one big file though, I prefer the idea of individual filter files which can be shared. I had thought of the 'based_on' capability to change - eg templates or other details but perhaps template optionally being specified somewhere else, eg in the channels file means that the 'based_on' isn't so important.

Your examples above are all possible; only thing that I hadn't considered is pokemon excludes rather than positive inclusion. Underlying all this the filters are translated to poracle tracking (which is inclusive only). I'm not sure if hiding this is useful as if the pokemon in monsters.json expands for whatever reason without a resync they would be missed. Thoughts about whether exclusions are helpful very welcome!

It already accepts %multiplechannels, specific-channels and channel-ids so sounds like I'm quite close.

[mermao1]

Question: the filters are still being set in the database and the files only have to create these records in the db or will they be filtered directly in the json files?
I believe that json files are just for introducing the rules directly into the database, right? If yes, what will happen when we make changes to the json files? Will Poracle truncate monsters table and redo creation based on file changes?

[jfberry]

Yes, absolutely the intention is that the file based configuration feeds into existing tracking database, so there are no changes to the alerting part of Poracle.
On reloading the config files, poracle clears channels that are impacted and replays the new configuration into the monsters/raids/eggs/etc tables.
As per the PR description at the moment, the current implementation triggers the reload on !channelsync - it would be possible to do this on, eg. a restart also but didn't want to do it on a file change as this could leave partial unfinished configuration loaded.