easteriq / last_messaging

Messaging framework for the LAST project

LAST_Messaging

Messaging framework for the LAST project, implementing:

• ability of spawning programmatically new matlab sessions, controlled loosely by their caller (class obs.util.SpawnedMatlab) (LINUX only)
• messenger infrastructure allowing different matlab session to send matlab commands and queries among one another (classes obs.util.Messenger, obs.util.Listener, obs.util.Message). The infrastructure is based on udp datagrams allowing an N:1 communication between sessions.

OLD Reference ramblings in this document (also duplicated loosely here)

• I have assumed for the time being that the classes sit under +obs/+util and are thus to be prefixed with obs.util. If this is changed, remember to change accordingly all the references within the code.

Dependencies

• certainly depends from LAST_Handle, I doubt that by chance it depends on other components of the LAST project too.

• the matlab Instrument Control Toolbox is required

last_messaging's Issues

superunit.connect to a nonexisting host connects reports success, with data of the preceding one

E.g.

S = 

  superunit with properties:

         UnitHosts: {'last01e'  'last02e'  'last03e'  'last04e'  'last05e'  'last06e'  'last07e'  'last08e'  'last09e'  'last10e'}
      UnitTerminal: 'silentx'
    SlaveTerminals: 'silentx'
       RemoteUnits: [1x10 obs.util.SpawnedMatlab]
           Logging: 1
        LoggingDir: '~/log'

>> S.connect
{08:41:50.101|obs.util.SpawnedMatlab[master08]} connection with session master08 not estabilished, aborting

ans =

  1x10 logical array

   1   1   1   1   1   1   1   0   1   1

>> [S.RemoteUnits([6,7,9]).PID]

ans =

     1359362     1359362     1359362
>> S.RemoteUnits(7).Messenger  

ans = 

  Messenger with properties:

      DestinationHost: ''
      DestinationPort: 11000
            LocalPort: 41735
    EnablePortSharing: 'on'
                 Name: 'localhost:->:11000'
         MessagesSent: 8

>> S.RemoteUnits(7).Messenger.whoIsThere

ans =

    'localhost:11000->10.23.1.25:50001'

NB the units were previously connected to a superunit on last0, this explains the name with 1.25, and possibly creates a side effect.

Do `SpawnedMatlab`s need to be disowned?

Probably so, in order to guarantee that the remote units, if launched silently and not in a terminal, survive if the launching machine crashes or just disconnect. Since they are launched with ssh, one must probably do ssh ....; disown -ar && exit or something the like. Or perhaps use nohup matlab ..... Check.
Thinking in particular at superunit run on last0, spawning remote units on the various LAST machines.

allow commands on vectors of SpawnedMatlabs

>> Unit.Slave.disconnect
Error using isa
Too many input arguments.
Error in obs.util.SpawnedMatlab/disconnect (line 9)
    if isa(S.Messenger,'obs.util.MessengerCommon') 

it's work, I don't know if worth

spawning slaves with `.RemoteTerminal='none'` doesn't work anymore

I suspect I have messed up something when merging with multimessenger.

One way of testing:

Unit=obs.unitCS('03');
Unit.Mount=inst.XerxesSimulated('03_1');
for i=1:2;Unit.Slave{i}.Logging=true; end
for i=1:2;Unit.Slave{i}.LoggingDir='/tmp'; end
Unit.Slave{1}.RemoteTerminal='none'; % or the other, or both
Unit.connectSlave(1:2)

I get misformed MasterMessenger or messages, e.g.

{Error using icinterface/fwrite (line 187)
Invalid network address.

Error in obs.util.MessengerCommon/reply (line 13)
    fwrite(Msng.StreamResource,flat);

Error in obs.util.Messenger/datagramParser (line 106)
           Msng.reply(jsonencode(ME.message)); % double quotes for json

Error in obs.util.Messenger>@(varargin)Msng.datagramParser(varargin{:}) (line 17)
                [email protected];

Error in instrcb (line 42)
        feval(val{1}, obj, eventStruct, val{2:end});
}

>> MasterMessenger.StreamResource.RemoteHost

ans =

    'AstroPack'

iostream errors

obs.superunit.send reported a lot of errors like

{obs.util.Messenger} master02.Messenger receiver reports: Error writing to output stream. : iostream error

talking with this or that unit, and dispatched unitCS commands were not executed. Find explanation and eventual treatment or more precise report.

syntax errors in `classcommand()` confuse the messengers and prevent further dialogue

(this is after having brought multimessenger to production)

E.g. in a unit master

Unit.Camera{1}.classCommand('(')

The slave says

>> Unrecognized method, property, or field 'From' for class 'obs.util.Message'.

Error in obs.util.Messenger/datagramParser (line 75)
            M.Command,M.From.Host,M.From.Port,ME.message);

Error in obs.util.Messenger>@(varargin)Msng.datagramParser(varargin{:}) (line
17)
                [email protected];

Error in instrcb (line 42)
        feval(val{1}, obj, eventStruct, val{2:end});

After timeout the master says

{obs.util.Messenger} 03_slave_1.Messenger timed out waiting for a reply to "Unit.Camera{1}.("

but now

>> Unit.Slave{1}.Messenger.areYouThere
{obs.util.Messenger} 03_slave_1.Messenger timed out waiting for a reply to "true"

ans =

  logical

   0

and in the slave

>> MasterMessenger.whoIsThere
{obs.util.Messenger} localhost:8501->10.23.3.5:50001 timed out waiting for a reply to "Msng.Name"

ans =

     []

The Responders are still capable of communicating

Messenger query output printed in the receiving session, instead of being squelched, if processed too late

Associated to PolishookDavid/LAST_OCS#26.

Not reproducing it with:

p=obs.util.SpawnedMatlab;
p.spawn
p.connect
p.Messenger.send('a=3;pause(10)'); p.Responder.query('a;'), p.Responder.query('a;')

or variations

queries from the spawned session take 10 sec to complete

In the master:

>> S=obs.util.SpawnedMatlab;
>> S.spawn
>> S.connect;
>> tic; S.Messenger.areYouThere, toc

ans =

  logical

   1

Elapsed time is 0.142787 seconds.
>> tic; S.Responder.areYouThere, toc

ans =

  logical

   1

Elapsed time is 0.061294 seconds.

In the slave instead

>> tic; MasterMessenger.areYouThere, toc

ans =

  logical

   1

Elapsed time is 10.016357 seconds.
>> tic; MasterResponder.areYouThere, toc

ans =

  logical

   1

Elapsed time is 10.015797 seconds.

what did I do to cause this?