Environment Variable not seen by brenda, but there when I ssh into the instance about brenda

[aliasguru]

Tested it right now, works perfectly!

Open your .brenda.conf, and add a section like this at the end:

# CUSTOM VAR section
# add commands to be executed before rendering here
STARTUP_PRERUN=export BLENDER_USER_CONFIG=/path/you/want/;export BLENDER_USER_SCRIPTS=/path/you/need/;export OCIO=/guess/what/config.ocio

Separate all vars and commands with a semicolon as indicated above. I guess things like this would work as well (but have not tested it):

# CUSTOM VAR section
# add commands to be executed before rendering here
STARTUP_PRERUN=sudo ln -s /mnt/brenda/log /home/ubuntu/log

This would for example re-enable the ability to simply execute brenda-tool ssh tail log again (which atm doesn't work)

[aliasguru]

The AMI is based on Ubuntu 16.04.2, Blender is version 2.79.

[tuttlen]

I think I may have run into this. I have to look at the script again. But I believe it starts the instance and then runs the script. There may be a sudo command inside the script that changes the system somehow. Can you insert a couple pipe out commands at the beginning and end of the blender script to see the if the variables change?

…

On Oct 8, 2017 1:46 PM, "aliasguru" ***@***.***> wrote: The AMI is based on Ubuntu 16.04.2, Blender is version 2.79. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#32 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABIPf9PPmK2erlSjkTYgV5Vsb0bPDdfYks5sqRhrgaJpZM4PxzkS> .

[aliasguru]

I've modified the brenda source code to inject a few echos and system commands. The script sent to the server now looks like this:

----------------------------
AMI ID: ami-xxxxxxxxx
Max bid price 0.06
Request type: one-time
Instance type: c4.xlarge
Instance count: 1
Instance store device: /dev/sdb
SSH key name: xxxxxxxxxxxxxxxx
Security groups: ('brenda-web',)
Startup Script:
   #!/bin/bash
   echo user
   echo ----
   whoami
   echo current location
   echo ----------------
   pwd
   echo environment variables:
   echo ----------------------
   printenv
   echo ----------------------
   # run Brenda on the EC2 instance store volume
   B="/mnt/brenda"
   if ! [ -d "$B" ]; then
     for f in brenda.pid log task_count task_last DONE ; do
       ln -s "$B/$f" "/home/ubuntu/$f"
     done
   fi
   export BRENDA_WORK_DIR="."
   # export BLENDER_USER_CONFIG=/CustomSoftware/xxx/
   # export BLENDER_USER_SCRIPTS=/CustomSoftware/yyy/
   # export OCIO=/CustomSoftware/filmic-blender/config.ocio
   mkdir -p "$B"
   cd "$B"
   echo user
   echo ----
   whoami
   echo current location
   echo ----------------
   pwd
   echo environment variables:
   echo ----------------------
   printenv
   echo ----------------------
   /usr/local/bin/brenda-node --daemon <<EOF
   AWS_ACCESS_KEY=[redacted]
   AWS_SECRET_KEY=[redacted]
   BLENDER_PROJECT=s3://bucketName/renderme.zip
   WORK_QUEUE=sqs://Kiska-Test
   RENDER_OUTPUT=s3://outBucketName
   DONE=shutdown
   echo user
   echo ----
   whoami
   echo current location
   echo ----------------
   pwd
   echo environment variables:
   echo ----------------------
   printenv
   echo ----------------------
   EOF

As you can see, I've commented out my own export commands, which allowed me to finally set those vars as I really need them. When I now retrieve the system log from that instance using the AWS console, I see that the commands are executed:

[   20.783797] cloud-init[1331]: user
[   20.786265] cloud-init[1331]: ----
[   20.796147] cloud-init[1331]: root
[   20.798499] cloud-init[1331]: current location
[   20.801062] cloud-init[1331]: ----------------
[   20.803646] cloud-init[1331]: /
[   20.805912] cloud-init[1331]: environment variables:
[   20.808564] cloud-init[1331]: ----------------------
[   20.811132] cloud-init[1331]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[   20.814566] cloud-init[1331]: PWD=/
[   20.816847] cloud-init[1331]: LANG=en_US.UTF-8
[   20.819416] cloud-init[1331]: SHLVL=1
[   20.821770] cloud-init[1331]: _=/usr/bin/printenv
[   20.824306] cloud-init[1331]: ----------------------
[   20.827020] cloud-init[1331]: user
[   20.829361] cloud-init[1331]: ----
[   20.831664] cloud-init[1331]: root
[   20.833954] cloud-init[1331]: current location
[   20.836465] cloud-init[1331]: ----------------
[   20.838914] cloud-init[1331]: /
[   20.841150] cloud-init[1331]: environment variables:
[   20.843732] cloud-init[1331]: ----------------------
[   20.846318] cloud-init[1331]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[   20.849641] cloud-init[1331]: PWD=/
[   20.851882] cloud-init[1331]: LANG=en_US.UTF-8
[   20.854362] cloud-init[1331]: SHLVL=1
[   20.856656] cloud-init[1331]: _=/usr/bin/printenv
[   20.859302] cloud-init[1331]: ----------------------
[   20.861906] cloud-init[1331]: ln: failed to create symbolic link '/home/ubuntu/log': File exists
[   20.865175] cloud-init[1331]: user
[   20.867372] cloud-init[1331]: ----
[   20.869637] cloud-init[1331]: root
[   20.871885] cloud-init[1331]: current location
[   20.874330] cloud-init[1331]: ----------------
[   20.876976] cloud-init[1331]: /mnt/brenda
[   20.879422] cloud-init[1331]: environment variables:
[   20.882121] cloud-init[1331]: ----------------------
[   20.884818] cloud-init[1331]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[   20.888208] cloud-init[1331]: PWD=/mnt/brenda
[   20.890816] cloud-init[1331]: LANG=en_US.UTF-8
[   20.893339] cloud-init[1331]: SHLVL=1
[   20.895659] cloud-init[1331]: BRENDA_WORK_DIR=.
[   20.898248] cloud-init[1331]: _=/usr/bin/printenv
[   20.900814] cloud-init[1331]: OLDPWD=/
[   20.903237] cloud-init[1331]: ----------------------

So, at all times, the user which runs the brenda commands is root, and the current location is / (which is odd, I'd have expected it to be /root). There is a symlink error, but that one is caused by myself, I had already created that one on the AMI. The error is harmless. The environment vars however are clearly the minimalistic ones, not the ones ssh gives me.

One hope I can see: While digging around, I noticed that the startup script actually looks for a key called STARTUP_PRERUN. I interpret this as someone (presumably James Yonan) giving us a possibility to inject own commands to the script, by placing them into .brenda.conf. That's what I'll try next. If it works, users are not obliged to change source code for defining own AMI vars.