Job fields difference

[manycoding]

As in #157, write a rule which returns field values difference between 2 jobs.

difference(source_df, target_df, ["name", "url", "id"], dropna=false)
>>>Difference    FAILED

details part

>>>500 new names, 500 same names
450 names missing y, u, i: 1,  50, 30
150 new urls, 850 same urls
100 urls missing x, y, z: 5, 10..
20 new ids, 980 same ids
10 ids missing a, b, c: 77, 11, 55

• Fail condition - assuming we use this rule if we know that certain threshold of values should be the same. E.g. if more than 25% of values are different, it is a failure.
• dropna option, meaning if we exclude nan values from comparison. What's default value? Check
• Any data is allowed (numeric, nan, string, lists, dicts).
• Lists and dicts are converted to str. Strings are normalized (lower, strip).
• Do we need to output values? See example