should calculate stats about who spoke after who - מדד הוכחנים

[OriHoch]

pipeline 1: who-spoke-after

• input:
  • committee meeting protocol parts, can be loaded from this datapackage - get only the parts documents
  • each parts file contains a column of title
• processing:
  • keep last title and yield a row containing this title and the last title
  • this title is spoke field and last title is spoke_after field
• output:
  • csv, each row contains the following fields:
    • meeting_id
    • spoke - name of someone who spoke in a meeting
    • spoke_after - name of someone who spoke after

pipeline 2: who-spoke-after-mks

join the spoke after data with mk names

• input:
  • who-spoke-after output
  • mk_individual resource which contains list of mk names and alternative names
• processing:
  • go over the spoke names and look for names that match mk names / alt names
  • please copy the matching algorithm from this processor - it does the same matching, but for an entire meeting rather then each part
• output:
  • meeting_id
  • spoke_mk_id - id of mk who spoke
  • spoke_after_mk_id - id of mk who spoke after

pipeline 3: statistics - מדד הוכחנים

load the previous data and aggregate statistics

• input:
  • who-apoke-after-mks
  • mk_individual resource
• processing:
  • for each mk X mk combination - get the number of times they spoke after one another
  • get list of mks per (knesset number / faction / committee) which argue the most / least