Deterministic Event Based calculation (hazard part) about oq-engine

[monellid]

"The computation will also take into account if the sites are correlated or not, reading the GROUND_MOTION_CORRELATION parameter specified in the configuration file."
I think it's better to say "The computation will also take into account if the user wants to compute correlated or uncorrelated ground motion fields, by reading the GROUND_MOTION_CORRELATION parameter specified in the configuration file"

[al-maisan]

Andrea: quick note re. parallelization: with the utils.tasks.distribute() component no extra effort is needed to do that.
You basically just need to pay attention to a parameter as shown in the branch you are currently reviewing (https://github.com/gem/openquake/pull/111)

[acerisara]

In this case we must pass to the calculator all the sites defined inside the region. Is it possible to use utils.tasks.distribute() to spawn N tasks but passing always the same input parameters?

[al-maisan]

I see, that's a (much) simpler use case (which is a good thing!). I will probably write a separate function (utils.tasks.parallelize() ?) that caters to it.

[al-maisan]

Andrea: when you spawn N tasks passing the same parameters to all of them: how is the calculation by these tasks carried out differently? Do they read additional data from the KVS and vary based on that?

[acerisara]

The component that we have to call (wrapper through jpype) is org.gem.calc.GroundMotionFieldCalculator. The object is created only once, passing the whole region, an attenuation relationship and an earthquake rupture. What we call N times, and we would like to spawn N tasks for that is the getUncorrelatedGroundMotionField (or getCorrelatedGroundMotionField_JB2009, depending on the configuration file) method. That method takes a random number generator as input, that provides the variability between the N calls.

[al-maisan]

On 04/07/2011 11:11 AM, acerisara wrote:

The component that we have to call (wrapper through jpype) is
org.gem.calc.GroundMotionFieldCalculator.
The object is created only once, passing the whole region, an
attenuation relationship and an earthquake rupture. What we call N
times, and we would like to spawn N tasks for that is the
getUncorrelatedGroundMotionField (or
getCorrelatedGroundMotionField_JB2009, depending on the configuration
file) method.
That method takes a random number generator as input,
Hello Andrea,

I had an interesting conversation with Damiano on this topic and I will
try to summarise it here.

Your statements above are all good as long as we are in a single celeryd
situation since that celeryd will load its jvm shared library via jpype
and all the celery tasks that run on it and share the same
GroundMotionFieldCalculator instance.

Things become more interesting when we have 2+ celery daemons :-) since
each of the latter has its own jvm and GroundMotionFieldCalculator instance.
Furthermore, all calculator instances have their own random number
generator (RNG) that's seeded with the same value.

If celeryd1 runs celery-task-1 and celeryd2 runs celery-task-2 then
things will break since both tasks will be executed with the same random
values (due to 2 calculators using their own random number generators
seeded with the same value).

One solution suggested by Damiano is for each celery task to take the
RNG seed as well as a sequence number

@task
def the_task(rng_seed, task_sequence_number, ..):

and then seed the RNG with (rng_seed * task_sequence_number).

That would preclude overlapping random value sequences in case of tasks
running on 2+ celery daemons but still give us some predictability.

Please let mw know what you think.

Best regards/Mit freundlichen Grüßen

Muharem Hrnjadovic [email protected]