[BugFix] Fix statistics agg functions to return NULL incorrectly (backport #47904)

[mergify]

Why I'm doing:

Return NULL incorrectly

For now, the result nullable property of aggregation functions contains two cases:

1. Always non-nullable
   • FE pass is_output_nullable with false value to BE, such as such as count, count distinct, and bitmap_union_int.
2. Nullable the same as input
   • FE pass is_output_nullable with true value to BE. (NOTE that BE need determine whether the result is nullable by itself!)
   • The result is NULl only when all the input rows are NULL.

This is OK, until the statistics aggregations occur. The reason is that the output of statistics aggregations is always nullable no matter what the input is, because they will return NULL when the number of input non-null rows is less than 2.

COVARIANCE

According to Wikipedia, when merging two COVARIANCE states $C_A$ and $C_B$, the formula should be as follows:

$$ C_X = C_A + C_B + (\overline{x}_A - \overline{x}_B)(\overline{y}_A - \overline{y}_B) \cdot \frac{n_A n_B}{n_X}, $$

but we use a wrong one as follows:

$$ C_X = C_A + C_B + (\overline{x}_A - \overline{x}_B)(\overline{y}_A - \overline{y}_B) \cdot \frac{n_A}{n_X}, $$

where $C_n$ is defined as follows:
$$C_n = \sum_{i=1}^{n} (x_i - \overline{x}_n)(y_i - \overline{y}_n)$$

What I'm doing:

1. Pass AggNullPred(State) to NullableAggregate.
   For now, NullableAggregate only returns null when the input is null, but the statistics function will return NULL when the number of input non-null rows is less than 2.
   Therefore, we need to add a new parameter AggNullPred to determine whether the output is nullable.
   And this is zero overhead when it is passed as a constexpr functor AggNonNullPred by default, since the compiler will eliminate this always false predicate.
2. Jusge is_result_nullable and use_nullable_fn according to use_intermediate_as_output, is_input_nullable, is_output_nullable and is_always_nullable_result.

Fixes #47762.

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 3.3
  • 3.2
  • 3.1
  • 3.0
  • 2.5

---

This is an automatic backport of pull request #47904 done by [Mergify](https://mergify.com). ## Why I'm doing:

Return NULL incorrectly

For now, the result nullable property of aggregation functions contains two cases:

1. Always non-nullable
   • FE pass is_output_nullable with false value to BE, such as such as count, count distinct, and bitmap_union_int.
2. Nullable the same as input
   • FE pass is_output_nullable with true value to BE. (NOTE that BE need determine whether the result is nullable by itself!)
   • The result is NULl only when all the input rows are NULL.

This is OK, until the statistics aggregations occur. The reason is that the output of statistics aggregations is always nullable no matter what the input is, because they will return NULL when the number of input non-null rows is less than 2.

COVARIANCE

According to Wikipedia, when merging two COVARIANCE states $C_A$ and $C_B$, the formula should be as follows:

$$ C_X = C_A + C_B + (\overline{x}_A - \overline{x}_B)(\overline{y}_A - \overline{y}_B) \cdot \frac{n_A n_B}{n_X}, $$

but we use a wrong one as follows:

$$ C_X = C_A + C_B + (\overline{x}_A - \overline{x}_B)(\overline{y}_A - \overline{y}_B) \cdot \frac{n_A}{n_X}, $$

where $C_n$ is defined as follows:
$$C_n = \sum_{i=1}^{n} (x_i - \overline{x}_n)(y_i - \overline{y}_n)$$

What I'm doing:

1. Pass AggNullPred(State) to NullableAggregate.
   For now, NullableAggregate only returns null when the input is null, but the statistics function will return NULL when the number of input non-null rows is less than 2.
   Therefore, we need to add a new parameter AggNullPred to determine whether the output is nullable.
   And this is zero overhead when it is passed as a constexpr functor AggNonNullPred by default, since the compiler will eliminate this always false predicate.
2. Jusge is_result_nullable and use_nullable_fn according to use_intermediate_as_output, is_input_nullable, is_output_nullable and is_always_nullable_result.

Fixes #47762.

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
• This is a backport pr

[mergify]

Cherry-pick of 023e50b has failed:

On branch mergify/bp/branch-2.5/pr-47904
Your branch is up to date with 'origin/branch-2.5'.

You are currently cherry-picking commit 023e50ba5e.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   be/src/exprs/agg/covariance.h
	modified:   be/src/exprs/agg/variance.h
	new file:   test/sql/test_agg_function/R/test_always_null_statistic_funcs
	new file:   test/sql/test_agg_function/T/test_always_null_statistic_funcs

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   be/src/exec/pipeline/aggregate/spillable_aggregate_blocking_sink_operator.cpp
	both modified:   be/src/exec/vectorized/aggregator.cpp
	both modified:   be/src/exec/vectorized/aggregator.h
	both modified:   be/src/exprs/agg/factory/aggregate_factory.hpp
	both modified:   be/src/exprs/agg/factory/aggregate_resolver.hpp
	both modified:   be/src/exprs/agg/factory/aggregate_resolver_variance.cpp
	both modified:   be/src/exprs/agg/nullable_aggregate.h

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[mergify]

@mergify[bot]: Backport conflict, please reslove the conflict and resubmit the pr