Help with choosing compute instance for adjoint optimization (Meep Parallel ) on AWS/GCP platform

[satadrudas]

Hi, I am rather new to AWS and GCP and have just started learning. I wish to run adjoint optimization of a Silicon Photonic device on it. For example let's take the dimension of the device is 10um x 10um and the resolution is, say 50 pixels/um.

1. Is there any way to figure out how much memory bandwidth, cores, network latency etc would be optimal for this particular adjoint optimization? It would be really helpful if anyone could suggest the instances from AWS EC2 (such as c5n, z1d etc) or GCP (c2, m2 etc) which could be used for this purpose.

2. Also just to make things clear for me, in issue #2453, @smartalecH mentions that when it comes to adjoint optimizatin the adjoint solver automatically parallelizes and nothing extra needs to be added in the code. So basically all we have to do is add a mp.divide_parallel_processes to the code and launch the program with mpirun and specify the number of core and the parallelization taken care of by meep itself?

Thanks

[smartalecH]

So basically all we have to do is add a mp.divide_parallel_processes to the code and launch the program with mpirun and specify the number of core and the parallelization taken care of by meep itself?

No, not quite. Do not use divide_parallel_processes, unless you want groups of processors doing different things (like entirely different simulations).

If you just want to distribute a single simulation, then just launch it with mpirun (along with the number of cores etc) and meep will automatically distribute things under the hood.

[satadrudas]

Got it. Thanks for the info.