Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Email and phone scrub lists #1786

Open
jace opened this issue Jul 5, 2023 · 1 comment
Open

Email and phone scrub lists #1786

jace opened this issue Jul 5, 2023 · 1 comment

Comments

@jace
Copy link
Member

jace commented Jul 5, 2023

To audit for spam and dead contacts, we will need external data sources. Two examples:

  • StopForumSpam provides a list of known abusive email addresses. When a user attempts sign-up with an email address in this list, they should be gated (eg: emailing support to request an unblock). For existing accounts, we'll need manual validation before classifying the account as spam.

  • TRAI publishes a Mobile Number Revocation List (MNRL) for expired Indian mobile phone numbers. These should be forgotten from our database, unlinking them from user accounts.

Both databases are significantly larger than our own, so it doesn't make sense to overload the existing EmailAddress and PhoneNumber models to hold this data. Instead, we should follow the model adopted with Geoname data, hosting this in a separate database with periodic updates.

This will entail:

  1. New bind_key alongside geoname for hosting contact data, or maybe rename geoname itself to be an extdata for external data.
  2. New CLI commands for downloading these databases, loading them, and scrubbing existing data.
  3. For the spam lists, a bloom filter for rapid lookup before doing a full index scan.
@jace
Copy link
Member Author

jace commented Jul 31, 2023

MNRL scrub support was added in #1810 but is pending a notification to users before it goes into production use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant