Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues left by cohort processor refactoring #83

Open
Zrealshadow opened this issue Aug 8, 2022 · 1 comment
Open

Issues left by cohort processor refactoring #83

Zrealshadow opened this issue Aug 8, 2022 · 1 comment
Labels
structure-adjust Adjust project structure or internal data structure

Comments

@Zrealshadow
Copy link
Collaborator

Create an issue and submit a corresponding PR for each one.

  1. need to add the logic to check the meta chunk to speed up the query processing.
  2. remove aggregation logic in ValueSelection.
    • ValueSelection only handles filtering (can be merged with EventSelection)
    • RetUnit will encapsulate the logic of updating statistics (discussed below)
  3. Factory class to handle FieldRS creation (better encapsulation)
  4. rework the ProjectedTuple
    • existing impl aims to mimic the old logic which introduces additional indirections
    • keep it simple, let its producer decides on which indexed value to retrieve from it.
    • make it immutable. avoid loadattr that mutates internal data.
    • separate handling for special field: user id and action time
  5. MetaFieldRS to create value converter/translator (extensibility and polymorphism)
    • currently we only have two types, we can expect having more field types.
    • The MetaFieldRS translates the token (gid for string now) stored in data chunk to actual values. (no-op for range)
  6. Augment RetUnit
    • perhaps we should rename this variable
    • it will contain additional variables for max, min, etc. (now only counts)
    • A solution is to have a list of variable and keep a list functions (aggregators) that take in new value and update its corresponding variable.
  7. add documentations: DataHashFieldRS Need to add descriptions to the assumptions: all input vector ever used there have efficient implementation of getting by index (Zint, BitVector, ZintBitInput,)
@Zrealshadow
Copy link
Collaborator Author

#72

@hugy718 hugy718 changed the title aftercare for refactoring of cohort processing engine Issues left by cohort processor refactoring Aug 20, 2022
@KimballCai KimballCai added the structure-adjust Adjust project structure or internal data structure label Aug 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
structure-adjust Adjust project structure or internal data structure
Projects
None yet
Development

No branches or pull requests

2 participants