Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify Appleyard chopping #809

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Modify Appleyard chopping #809

wants to merge 1 commit into from

Conversation

totto82
Copy link
Member

@totto82 totto82 commented Jun 2, 2023 •
edited
Loading

Restrict update of rs/rv, rsw/rvw and zfraction in the extended blackoil model by the saturation scaling factor from the Appleyard-chopping.

This is part 1 of #803

Modify Appleyard chopping
c420959 
Restrict update of rs/rv, rsw/rvw and zfraction in the extended blackoil model by the saturation scaling factor from the Appleyard-chopping.
@totto82
Copy link
Member Author

totto82 commented Jun 2, 2023

jenkins build this please

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023

benchmark please

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023

Black is PR. Green is reference results.

image

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

FWIR: Summary file
	Fails for 3 entries
	Largest absolute error: 2.9583301e+02
	Largest relative error: 1.7041231e-01

Black is PR. Green is reference results.

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

GGPI:INJE: Summary file
	Fails for 2 entries
	Largest absolute error: 1.9626675e+08
	Largest relative error: 2.3400146e-01

image

WWPR:PROD3: Summary file
	Fails for 1 entries
	Largest absolute error: 2.2281599e-01
	Largest relative error: 7.5347273e-02

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWPGR:PROD1: Summary file
	Fails for 1 entries
	Largest absolute error: 1.2884613e+02
	Largest relative error: 1.1923457e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWPR:PROD3: Summary file
	Fails for 6 entries
	Largest absolute error: 5.3414059e-01
	Largest relative error: 1.0045919e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 •
edited by bska
Loading 

WLPR:OPU02: Summary file
	Fails for 4 entries
	Largest absolute error: 1.0356105e+02
	Largest relative error: 4.8567018e-01

image

WWIR:WIL01: Summary file
	Fails for 3 entries
	Largest absolute error: 3.1060254e+02
	Largest relative error: 7.2784564e-02

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWPR:PROD3: Summary file
	Fails for 4 entries
	Largest absolute error: 2.0443020e+00
	Largest relative error: 1.0837778e-01

image

WGIP:INJ1: Summary file
	Fails for 3 entries
	Largest absolute error: 1.3314957e+08
	Largest relative error: 1.8527693e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWPR:PROD3: Summary file
	Fails for 1 entries
	Largest absolute error: 1.1698884e-01
	Largest relative error: 2.0989576e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWPR:PROD3: Summary file
	Fails for 1 entries
	Largest absolute error: 1.1698884e-01
	Largest relative error: 2.0989576e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWPR:PROD1: Summary file
	Fails for 1 entries
	Largest absolute error: 1.2119231e+00
	Largest relative error: 1.9043798e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWPR:PROD1: Summary file
	Fails for 2 entries
	Largest absolute error: 1.6109295e+00
	Largest relative error: 1.0339842e-01

image

WBHP:INJ1: Summary file
	Fails for 1 entries
	Largest absolute error: 2.6048584e+01
	Largest relative error: 7.7440490e-02

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WBHP:INJ1: Summary file
	Fails for 1 entries
	Largest absolute error: 4.0556641e+01
	Largest relative error: 9.2056089e-02

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WTHP:PROD1: Summary file
	Fails for 1 entries
	Largest absolute error: 6.5482483e+00
	Largest relative error: 6.0313764e-02

image

WWPR:PROD1: Summary file
	Fails for 2 entries
	Largest absolute error: 8.0917358e+01
	Largest relative error: 2.1558314e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WOPR:PROD3: Summary file
	Fails for 1 entries
	Largest absolute error: 4.6544556e+01
	Largest relative error: 1.1191940e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WOPR:OP_1: Summary file
	Fails for 1 entries
	Largest absolute error: 3.7582776e+00
	Largest relative error: 9.9941726e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WWIR:WI_1: Summary file
	Fails for 2 entries
	Largest absolute error: 1.3175140e+02
	Largest relative error: 7.6847942e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WOPR:OP_1: Summary file
	Fails for 2 entries
	Largest absolute error: 1.8420239e+02
	Largest relative error: 1.9601552e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 •
edited
Loading 

WOPR:OP_1: Summary file
	Fails for 4 entries
	Largest absolute error: 2.2667194e+02
	Largest relative error: 3.0097662e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 

WTHP:PROD3: Summary file
	Fails for 9 entries
	Largest absolute error: 2.4072021e+01
	Largest relative error: 5.8558714e-01

image

WOPR:PROD3: Summary file
	Fails for 1 entries
	Largest absolute error: 1.9500918e+02
	Largest relative error: 7.5801425e-01

image

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023

I have manually gone through the test failures and plotted significant deviations. Cases not shown have only minor changes. The significant differences points to changes in time-stepping that again affects the results. Some more testing on field models is needed before concluding if these changes improves the stability of the newton update, but the test models are ok IMO.

@totto82
Copy link
Member Author

totto82 commented Jun 5, 2023 •
edited by bska
Loading

I think it would be good to automate the process to evaluate the test failures. Currently this involves significant
manual work. What I have done this time is to go through the test failures and plot the worst offending vectors using qsummary. Like

~/workspace/opm/qsummary/build/qsummary \
    flow+udq_wconprod/UDQ_WCONPROD. \
    ~/workspace/opm/opm-tests/udq_actionx/opm-simulation-reference/flow/UDQ_WCONPROD. \
    -v WLPR:OPU02

@akva2 What do you think? Could the current test infrastructure be extended with such an workflow?

@ytelses
Copy link

ytelses commented Jun 5, 2023

Benchmark result overview:

Test Configuration Relative
opm-git OPM Benchmark: drogon - Threads: 1 0.979
opm-git OPM Benchmark: drogon - Threads: 8 0.617
opm-git OPM Benchmark: smeaheia - Threads: 1 1
opm-git OPM Benchmark: smeaheia - Threads: 8 1
opm-git OPM Benchmark: spe10_model_1 - Threads: 1 0.991
opm-git OPM Benchmark: spe10_model_1 - Threads: 8 1.001
opm-git OPM Benchmark: flow_mpi_extra - Threads: 1 1.06
opm-git OPM Benchmark: flow_mpi_extra - Threads: 8 0.945
opm-git OPM Benchmark: flow_mpi_norne - Threads: 1 1.016
opm-git OPM Benchmark: flow_mpi_norne - Threads: 8 0.975
  • Speed-up = Total time master / Total time pull request. Above 1.0 is an improvement. *

View result details @ https://www.ytelses.com/opm/?page=result&id=2114

@GitPaean
Copy link
Member

GitPaean commented Jun 6, 2023 •
edited
Loading

I think it would be good to automate the process to evaluate the test failures. Currently this involves significant
manual work.

Yes, indeed. Manually going through all the test failures (very often, it is tens of them) is significant work. When time stepping changes, all the current jenkins comparison basically not sensible anymore. If the jenkins can help to plot all the relevant plots out, it will be a big step towards the right direction.

@totto82
Copy link
Member Author

totto82 commented Jun 6, 2023

I looked a bit more in details on the benchmark results for drogon. With CPRW the performance is similar for this PR and master.
image

@atgeirr
Copy link
Member

atgeirr commented Jun 7, 2023

It was suggested by @hnil to use fixed timesteps with no chopping for all feature tests, to avoid this quagmire. I think that is a good idea that could avoid a lot of extra work with tests that seemingly fail. An alternative that may be a bit weaker would be to only compare the solutions at the report steps.

For this concrete PR, I very much want to say "go ahead" but the changes are large enough to make me a little nervous. Maybe for one or a few of the "most failing" cases you could take a reference run, extract the timesteps, then run using the PR with the fixed steps, and see if the difference is then significant? (If not, then the difference was only caused by different timestepping and we are good to go.)

@GitPaean
Copy link
Member

GitPaean commented Jun 7, 2023

Maybe for one or a few of the "most failing" cases you could take a reference run,

The difficulty is that how to determine most failing is not easy to tell. We might oversee the real problems, for example, a bug. Then the purpose of jenkins regression test is defeated to a large extent.

@atgeirr
Copy link
Member

atgeirr commented Jun 7, 2023

The difficulty is that how to determine most failing is not easy to tell. We might oversee the real problems, for example, a bug. Then the purpose of jenkins regression test is defeated to a large extent.

I agree, and I did not intend this as a new general procedure, only as a way to assess the test failures here a bit better, at the same time seeing if the idea of fixing timesteps is workable.

@totto82
Copy link
Member Author

totto82 commented Aug 7, 2023

jenkins build this please

@totto82 totto82 mentioned this pull request Aug 14, 2023
Closed
@totto82
Copy link
Member Author

totto82 commented Jan 29, 2024

jenkins build this please

2 similar comments
@totto82
Copy link
Member Author

totto82 commented Jan 30, 2024

jenkins build this please

@totto82
Copy link
Member Author

totto82 commented Aug 26, 2024

jenkins build this please

@totto82
Copy link
Member Author

totto82 commented Aug 28, 2024

benchmark please

1 similar comment
@totto82
Copy link
Member Author

totto82 commented Sep 2, 2024

benchmark please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@totto82 @ytelses @GitPaean @atgeirr