Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lots of various improvements.. #283

Open
wants to merge 36 commits into
base: master
Choose a base branch
from

Commits on Jan 13, 2022

  1. minor configuration updates

    add:
     * LICENSE
     * crx and xpi build scripts
     * Firefox ID in manifest
       - for installing an unsigned xpi in FF Developer edition
    
    fix:
     * webpack config to support building on Windows
    
    change:
     * name of output directory from 'dist' to 'PrivacyPass'
     * npm 'clean' script to include:
       - typescript output 'lib' directory
       - crx and xpi files generated by new build scripts
     * npm 'sjcl' script to explicitly call perl,
       rather than depending upon the shell to parse the shebang line
    warren-bank committed Jan 13, 2022
    Configuration menu
    Copy the full SHA
    1931272 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2022

  1. work in progress..

    warren-bank committed Jan 14, 2022
    Configuration menu
    Copy the full SHA
    a623dee View commit details
    Browse the repository at this point in the history
  2. work in progress..

    warren-bank committed Jan 14, 2022
    Configuration menu
    Copy the full SHA
    59e01e2 View commit details
    Browse the repository at this point in the history
  3. work in progress..

    use chrome.i18n to internationalize strings in UI
    warren-bank committed Jan 14, 2022
    Configuration menu
    Copy the full SHA
    c9b0c40 View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2022

  1. work in progress..

    * update message passing between userscript and background page
    * update redux store used by popup
    
    summary:
    ========
    
    * messages are currently only passed in one direction:
      - from userscript to background page (w/ response to callback)
      - messages:
        1. {clear: true}
           - removes all passes for all providers from local storage
           - no response
        2. {tokensCount: true, providerID: number}
           - providerID is:
             * optional;
               if not included, then the response includes all providers
             * the same integer that identifies each unique provider
                 1 == Cloudflare
                 2 == hCaptcha
                 etc..
           - response: {[key: string]: number}
             * each key is a `${providerID}`
             * each value is a count of the number of passes in wallet
           - for example:
               {"1": 30, "2": 5}
    
    * the data structure used by the redux store to hold state
      is identical to the response from the background page
      to messages that request 'tokensCount'
    warren-bank committed Jan 15, 2022
    Configuration menu
    Copy the full SHA
    6af07f7 View commit details
    Browse the repository at this point in the history
  2. work in progress..

    * refactor the 'voprf' crypto module into a reusable class,
      so each provider can own a distinct instance
    
    * add the dependency: 'keccak'
      - v2 of the extension included a snapshot of this library
      - v3 removed it, but there is still code that wants to use it
        * presumably, this branch of code will get called
          when the "hCaptcha" provider is reimplemented (..coming soon)
    warren-bank committed Jan 15, 2022
    Configuration menu
    Copy the full SHA
    4b7beb1 View commit details
    Browse the repository at this point in the history
  3. work in progress..

    * somewhere between a bug fix and a performance optimization:
      - pass an instance of the 'voprf' crypto class
        from provider(s) to the Token constructor
      - otherwise, each Token initializes its own instance
        of the 'voprf' crypto class using default parameters
    warren-bank committed Jan 15, 2022
    Configuration menu
    Copy the full SHA
    70ec3f1 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2022

  1. work in progress..

    * hCaptcha provider is added, but not yet functional
    
    status:
    =======
    * "handleBeforeRequest()" detects when
      a captcha is solved on an issuing domain
    * "issue()" makes a subsequent request
      for tokens to be signed
    * the response data does include "signatures",
      which can be properly parsed:
        {
          sigs:    string[];
          proof:   string;
          version: string = "1.0";
        }
    
    to do:
    ======
    * the code currently expects that the commitment
      for the version to be formatted the same as Cloudflare:
        {H: string; expiry: string; sig: string;}
    * however, the JSON data file:
        https://raw.githubusercontent.com/privacypass/ec-commitments/master/commitments-p256.json
      shapes the data for version HC["1.0"] differently:
        {H: string; G: string;}
    * annecdotally, the part that hurts most..
      is that this data file includes a version HC["1.01"]
      that is shaped in a way that is consistent and would work
    warren-bank committed Jan 16, 2022
    Configuration menu
    Copy the full SHA
    06222cc View commit details
    Browse the repository at this point in the history
  2. work in progress..

    * hCaptcha provider is added, but not yet functional
    
    status:
    =======
    * "handleBeforeRequest()" detects when
      a captcha is solved on an issuing domain
    * "issue()" makes a subsequent request
      for tokens to be signed
    * the response data does include "signatures",
      which can be properly parsed:
        {
          sigs:    string[];
          proof:   string;
          version: string = "1.0";
          prng?:   string = undefined;
        }
    * the version "1.0" commitment is shaped:
        {H: string; G: string;}
      which, unlike Cloudflare, does not require verification
    * since "prng" is not defined,
      it should default to "shake",
      as was the methodology used by the v2 extension
    
    to do:
    ======
    * the "shake" prng has a missing dependency: Buffer
    warren-bank committed Jan 16, 2022
    Configuration menu
    Copy the full SHA
    a4e23fb View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    00cba76 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2022

  1. Configuration menu
    Copy the full SHA
    6fe4c6c View commit details
    Browse the repository at this point in the history
  2. v3.3.1 w/ minified ES5

    warren-bank committed Jan 17, 2022
    Configuration menu
    Copy the full SHA
    578a97b View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2022

  1. v3.3.2 w/ minified ES5

    fix: transpile ES6 node modules: 'buffer' and 'keccak'
    warren-bank committed Jan 18, 2022
    Configuration menu
    Copy the full SHA
    929cabd View commit details
    Browse the repository at this point in the history
  2. minor configuration updates

    change:
     * name of output directory from 'PrivacyPass' to 'dist/PrivacyPass'
     * location of crx and xpi build scripts from '.bin' to 'dist/.bin'
     * location of pem input and crx/xpi output files from '/' to 'dist'
    warren-bank committed Jan 18, 2022
    Configuration menu
    Copy the full SHA
    b9df538 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2022

  1. Configuration menu
    Copy the full SHA
    cf7144c View commit details
    Browse the repository at this point in the history
  2. work in progress..

    * the methodology used in this commit does NOT work
      - I fully expected that it would
      - it does NOT
      - this commit is being tagged so I can point to it later,
        if asked: "why didn't you just...?"
    * the methodology is to:
      - detect requests that solve catchas made to issuing domains
        * as was done previously
      - rather than make the issuing request then and there,
        wait until the corresponding call to "handleHeadersReceived"
    * problem:
      - there (apparently) is no corresponding call to "handleHeadersReceived"
    * to do:
      - add new hook for call to "chrome.webRequest.onCompleted"
      - add plumbing for all providers to receive this new hook function
      - ...hope it receives a corresponding call
    
    - - - -
    
    hCaptcha should send issuing requests from a different hook function
    
    similar to the way redemption works (but in reverse),
    detect solved captcha on issuing domains before requests are sent,
    but delay the processing of hits until after the response is received
    
    previously, a timeout was used to delay processing;
    but this methodology was inexact and could lead to failed attempts.
    
    note that this is in stark contrast to the Cloudflare provider,
    which cancels the requests that it detects,
    and processes those hits immediately without any delay.
    warren-bank committed Jan 20, 2022
    Configuration menu
    Copy the full SHA
    2f3fad0 View commit details
    Browse the repository at this point in the history
  3. hCaptcha sends issuing requests from 'onCompleted' hook function

    similar to the way redemption works (but in reverse),
    detect solved captcha on issuing domains before requests are sent,
    but delay the processing of hits until after the response is received
    
    previously, a timeout was used to delay processing;
    but this methodology was inexact and could lead to failed attempts.
    
    note that this is in stark contrast to the Cloudflare provider,
    which cancels the requests that it detects,
    and processes those hits immediately without any delay.
    warren-bank committed Jan 20, 2022
    Configuration menu
    Copy the full SHA
    e101fc7 View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2022

  1. minor css tweak to popup window

    * each row of providers contains: name, token count
    * on very old browsers running the CRX2 extension,
      the count of tokens wasn't correctly horizontally aligned
    * now it is
      - the CRX2 extension on Chrome 30 now looks identical to
        the CRX3 extension on Chrome 90
        ..and all versions inbetween
        ..and more
    warren-bank committed Jan 21, 2022
    Configuration menu
    Copy the full SHA
    615f189 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2022

  1. Configuration menu
    Copy the full SHA
    37a5271 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2022

  1. Configuration menu
    Copy the full SHA
    706a9e7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    baa5e80 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b889333 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2022

  1. fix hCaptcha redemption

    warren-bank committed Jan 27, 2022
    Configuration menu
    Copy the full SHA
    8d57b84 View commit details
    Browse the repository at this point in the history
  2. minor refactoring

    warren-bank committed Jan 27, 2022
    Configuration menu
    Copy the full SHA
    1f84b01 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2022

  1. address race-condition: delay blocking async code by next-tick timer

    context:
    ========
    * "handleBeforeRequest" is a hook function implemented by each provider
    * a global "chrome.webRequest.onBeforeRequest" listener delegates
      to the hook function implemented by the active provider
    * the listener blocks the sending of all outbound requests,
      so it needs to perform its work quickly
    
    previous methodology:
    =====================
    * any code in the hook function
      that is required to produce a return value
      is performed synchronously
    * all other code is performed asynchronously:
        (async () => {
          // work that is time consuming
          // and does not contribute to a return value
        })()
    
    new methodology:
    ================
    * all such asynchronous code has been delayed by a timer
      to begin execution in the next tick of the event loop:
        setTimeout(
          async () => {},
          0
        )
    
    considerations:
    ===============
    * the previous methodology created a race-condition
      by delaying outbound requests
    * the new methodology creates a different race-condition
      by delaying the processing of code,
      which may not be finished at the time its result is needed
    
    observations:
    =============
    * the previous methodology prevented the Cloudflare provider
      from being able to issue tokens about 90% of the time
    * the new methodology prevents the Cloudflare provider
      from being able to issue tokens about 10% of the time
    
    additional observations:
    ========================
    * issuance and redemption by the hCaptcha provider
      is not impacted by either methodology
    * redemption by the Cloudflare provider
      is not impacted by either methodology
    
    conclusions:
    ============
    * while this isn't a perfect solution,
      it is a significant improvement to the user experience
    warren-bank committed Jan 28, 2022
    Configuration menu
    Copy the full SHA
    bf6d1ae View commit details
    Browse the repository at this point in the history
  2. revert hCaptcha and refactor Cloudflare

    the issue of a race-condition persists
    * the previous commit didn't improve the situation
      as much as it initially seemed
    * this commit rolls back some changes that I dislike,
      and makes a few minor improvements to the code,
      but doesn't solve the issue
    warren-bank committed Jan 28, 2022
    Configuration menu
    Copy the full SHA
    48f5e14 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2022

  1. detect and fix incorrect requests on Cloudflare issuing domain

    This methodology is aggressive.
    
    It doesn't aim to fix the race-condition
    that exists on the Cloudflare website;
    only Cloudflare can do that directly.
    
    It provides a workaround to deal with both possible outcomes.
    One outcome is desirable, and everything works as it should.
    The other outcome is unexpected, and this commit:
     * detects the incorrect request
     * cancels the incorrect request
     * cherry picks data from its headers
     * constructs the correct request to issue tokens
    
    This methodology is very similar to how a correct request is processed.
    The difference is where the data originates,
    which is then used to construct the request to issue tokens.
    
    a "correct" request:
    ====================
      curl 'https://captcha.website/?__cf_chl_captcha_tk__=xxx' \
        -H 'Content-Type: application/x-www-form-urlencoded' \
        -H 'Referer: https://captcha.website/' \
        --data-raw 'md=xxx&r=xxx&cf_captcha_kind=h&vc=xxx&captcha_vc=xxx&captcha_answer=xxx&cf_ch_verify=plat&h-captcha-response=captchka'
    
    an "incorrect" request:
    =======================
      curl 'https://captcha.website/' \
        -H 'content-type: application/x-www-form-urlencoded' \
        -H 'referer: https://captcha.website/?__cf_chl_tk=xxx' \
        --data-raw 'md=xxx&r=xxx&cf_captcha_kind=h&vc=xxx&captcha_vc=xxx&captcha_answer=xxx&cf_ch_verify=plat&h-captcha-response=captchka'
    
    In a "correct" request:
     * querystring of url contains value for key: '__cf_chl_captcha_tk__'
    
    In an "incorrect" request,
     * querystring of 'referer' header contains value for key: '__cf_chl_tk'
     * the "correct" url can be constructed by adding this value to the querystring key: '__cf_chl_captcha_tk__'
     * apparently, they are interchangeable!
       - the resulting request to issue tokens works 100% of the time
    warren-bank committed Jan 29, 2022
    Configuration menu
    Copy the full SHA
    71e4b54 View commit details
    Browse the repository at this point in the history
  2. fix: use feature detection so browser compatability isn't reduced

    notes:
    ======
     * only Chrome 72+ recognize the "extraHeaders" option value
       - both Firefox and older versions of Chrome
         will throw an error if this value is included
         when adding a listener to a "chrome.webRequest" event
    
    annoyances:
    ===========
     * the typescript definitions got in the way
     * "@types/chrome" doesn't define types for any of the
       "On---Options" classes in "chrome.webRequest"
     * for this reason, any time they need to be accessed..
       "chrome.webRequest" needs to be cast to "any"
     * a future version of "@types/chrome" will hopefully
       allow this casting to be removed
    
    observations regarding status of browser compatability:
    =======================================================
     * tested in Chrome 85
       - Cloudflare
         * working:
           - issuing of tokens
           - redeeming of tokens
       - hCaptcha
         * working:
           - issuing of tokens
           - redeeming of tokens
     * tested in Chrome 30
       - Cloudflare
         * not working:
           - issuing of tokens
       - hCaptcha
         * working:
           - issuing of tokens
           - redeeming of tokens
             * note: uses 2x tokens per bypassed captcha
     * tested in Firefox 97
       - Cloudflare
         * working:
           - issuing of tokens
           - redeeming of tokens
       - hCaptcha
         * not working:
           - issuing of tokens
    
    to do:
    ======
     * investigate why hCaptcha isn't issuing tokens in Firefox
       - it's behaving as though the original request had been cancelled
       - the request to issue tokens is being correctly sent to the provider,
         but it's being returned with a 403 status code:
             {"success":false,"error-codes":["invalid-data"]}
    warren-bank committed Jan 29, 2022
    Configuration menu
    Copy the full SHA
    3e39527 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2022

  1. Configuration menu
    Copy the full SHA
    84ea20b View commit details
    Browse the repository at this point in the history
  2. minor refactoring

    warren-bank committed Jan 30, 2022
    Configuration menu
    Copy the full SHA
    29febe8 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2022

  1. add static helper method: "getNormalizedFormData"

    During some real-world testing,
    I noticed that Chrome doesn't always parse
    'application/x-www-form-urlencoded' data in the POST body.
    
    This new helper method is available to providers to normalize
    the format of this data into a key/value hash object.
    
    Furthermore, it will also parse 'application/json' data.
    warren-bank committed Jan 31, 2022
    Configuration menu
    Copy the full SHA
    8283f46 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2022

  1. fix the ability to restore tokens from a JSON text file

    previous methodology:
    =====================
    * popup window:
      - dynamically creates an input file element
      - click event triggers the element to open the file chooser dialog
      - processes the list of selected files by:
        * reading the content of each file
        * parsing its JSON
        * passing the resulting object in a message to the background page
    * background page:
      - merges backup data with the tokens already saved in local storage
      - updates local storage with this new aggregate value
      - updates the popup window
    
    problem with previous methodology:
    ==================================
    * only works if the browser doesn't close the popup window
      before the data is passed to the background page
    * many browsers do close the popup window
      when the file input dialog is triggered for selection of input files
    
    new methodology:
    ================
    * popup window:
      - sends a message to the background page
    * background page:
      - opens a new tab and loads a static html page
    * static html page:
      - dynamically creates an input file element
      - adds a click event handler to the element,
        which requires user interaction to trigger
      - the click event handler processes the list of selected files by:
        * reading the content of each file
        * parsing its JSON
        * passing the resulting object in a message to the background page
      - the click event handler also tracks the count of files pending
        * after the processing of all files is complete,
          sends a final message to the background page
    * background page:
      - merges backup data with the tokens already saved in local storage
      - updates local storage with this new aggregate value
      - closes the tab containing the static html page
    
    comparison between methodologies:
    =================================
    * previous methodology:
      - pros:
        * simpler implementation
        * more elegant user experience
      - cons:
        * doesn't work in many browsers
    * new methodology:
      - pros:
        * works in all supported browsers
      - cons:
        * much more complicated implementation
        * less elegant user experience,
          which requires interaction with a standalone page in a new tab
    warren-bank committed Feb 1, 2022
    Configuration menu
    Copy the full SHA
    0ab56ce View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2022

  1. add i18n translations using IBM Watson Language Translator service

    probably not perfect, but hopefully a good starting point;
    PRs are welcome.
    warren-bank committed Feb 24, 2022
    Configuration menu
    Copy the full SHA
    7057859 View commit details
    Browse the repository at this point in the history
  2. minor html/css tweak to popup window

    * each row of providers contains: name, token count
      - previously, 2x floating divs were used for layout
        * there were no css rules to properly display text overflow
        * when only english strings were displayed,
          this was acceptable
        * when other language translations are displayed,
          this goes wrong very quickly
      - now, a simple 1x row by 2x column table element is used for layout
        * though a flexbox layout would be perfectly suited,
          it's only supported by newer browsers
        * table elements have always existed,
          and it works equally well
    
    example:
    ========
    * run Chrome using strings translated for German locale
      - bash:
          LANGUAGE='de' && chrome --lang=de
      - cmd in Windows:
          set "LANGUAGE=de" && chrome --lang=de
    warren-bank committed Feb 24, 2022
    Configuration menu
    Copy the full SHA
    6ee36f8 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2022

  1. accomodate for changes to Cloudflare provider backend

    previously, request to issue tokens required querystring parameter:
      __cf_chl_captcha_tk__
    
    now, the name of this parameter has been changed to:
      __cf_chl_f_tk
    warren-bank committed Mar 20, 2022
    Configuration menu
    Copy the full SHA
    333f32b View commit details
    Browse the repository at this point in the history
  2. CF: normalize name of querystring parameter in request to issue tokens

    the detection criteria allows for any of several parameter names,
    but the backend endpoint to issue Cloudflare tokens
    only accepts a single parameter name.
    warren-bank committed Mar 20, 2022
    Configuration menu
    Copy the full SHA
    ad517dd View commit details
    Browse the repository at this point in the history