slac-lcls / ezpsana Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 2.0 218 KB

Easy PSANA everywhere!

Shell 12.66% Dockerfile 8.85% Python 78.49%

ezpsana's Introduction

Easy Photon System ANAlysis

How to:

bring psana where you need it
build a psana tailored to your needs

Usage

There are 3 main ways to work with psana: through a JupyterHub, or using a scheduler (LSF or SLURM) in either batch or interactive mode. We describe below how to go about each of those modalities at various facilities: TBD.

Find or build your container

While most of the approach we follow is based on building conda environment, we ultimately build and run containers. Here is how to find or build your own psana-based container!

Check if your container already exists!

All psana-based containers that we build are stored at the following URL Have a look there! If you don't find what you are looking for, proceed to the next step...

Create your container

In order to run the following script, make sure your python has the docker module. If not, pip install docker. If you want, you can also run it in the ezpsana environment that you could create with conda env create -f ezpsana.yml.

(env)[terminal]$ python create-container.py -h
usage: create-container.py [-h] -b BASE_ENV_FILENAME
                           [-a ADDITIONAL_DEPS_FILENAME] -o OUTPUT_FILENAME
                           [-d] [-c] -r REPOSITORY -t TAG -v VERSION

optional arguments:
  -h, --help            show this help message and exit
  -b BASE_ENV_FILENAME, --base-env-yaml-file BASE_ENV_FILENAME
                        YAML file for base environment
  -a ADDITIONAL_DEPS_FILENAME, --additional-deps-yaml-file ADDITIONAL_DEPS_FILENAME
                        YAML file for additional dependencies
  -o OUTPUT_FILENAME, --output-yaml-file OUTPUT_FILENAME
                        output YAML file name (will be written in the 'docker'
                        directory)
  -d, --build-docker-container
                        Build docker container
  -c, --no-cache        Do not use cache when building the docker container
  -r REPOSITORY, --repository REPOSITORY
                        Repository for the docker image
  -t TAG, --tag TAG     Tag for the docker image
  -v VERSION, --version VERSION
                        Version for the docker image

Below is an example where the container is built with GitHub Actions:

python create-container.py -b yaml/ana-4.0.35.yml -o ana-4.0.35.yml -e -c -r fpoitevi -t ana -v 4.0.35
git add .github/workflows/main.yml
git tag -a build-ana-4.0.35 -m "attempt at building psana and pushing it to fpoitevi's DockerHub"
git push origin build-ana-4.0.35

Run your container

on SDF

If the Singularity image you need is not already on SDF, you can pull it from DockerHub like so:

singularity pull docker://slaclcls/ana-py3:latest

Then, request the resources that you need and run the container once the resources have been allocated:

srun -A LCLS -n 1 --gpus 1 --pty /bin/bash

singularity exec --nv -B /sdf,/gpfs,/scratch,/lscratch /scratch/fpoitevi/singularity_images/ana-py3_latest.sif /bin/bash

ezpsana's People

Contributors

Stargazers

Watchers

Forkers

irischang020 lcls-users

(ezpsana) [fpoitevi@sdf-login03 ezpsana]$ ./create-container.py -b yaml/ana-4.0.28-py3.yml -a yaml/skopi.yml -o skopi-ana-none-4.0.28-py3.yml -r skopi-ana -t none-4.0.28-py3 -u fpoitevi -d
>> Reading base environment YAML file: yaml/ana-4.0.28-py3.yml
>> Reading additional environment YAML file: yaml/skopi.yml
>> Adding custom dependencies to base environment
>> Writing output YAML file: docker/skopi-ana-none-4.0.28-py3.yml
>> Building docker image with tag: fpoitevi/skopi-ana:none-4.0.28-py3. Please wait (it can take several minutes)
Error accessing the docker service from the system: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

OpenMPI problem on SDF

With the latest environment, the following error is met on SDF compute nodes:

[fpoitevi@rome0023 singularity_images]$ conda activate skopi-ana-none-4.0.28-py3
(skopi-ana-none-4.0.28-py3) [fpoitevi@rome0023 singularity_images]$ python
Python 3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 16:07:37)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mpi4py
>>> from mpi4py import MPI
[rome0023:122763] pmix_mca_base_component_repository_open: unable to open mca_gds_ds12: /scratch/fpoitevi/conda/envs/skopi-ana-none-4.0.28-py3/bin/../lib/libmca_common_dstore.so.1: undefined symbol: pmix_output_check_verbosity (ignored)
[rome0023:122763] pmix_mca_base_component_repository_open: unable to open mca_gds_ds21: /scratch/fpoitevi/conda/envs/skopi-ana-none-4.0.28-py3/bin/../lib/libmca_common_dstore.so.1: undefined symbol: pmix_output_check_verbosity (ignored)
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[rome0023:122763] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

I will try the container approach soon. We might have to compile openmpi directly in the image...

AttributeError: module 'yaml' has no attribute 'safe_load'

Weird bug... When I do the following and the output .yml already exists under docker/ I get this error. Removing the .yml solves the issue but I'm not sure why...

(ezpsana) [fpoitevi@sdf-login03 ezpsana]$ ./create-container.py -b yaml/ana-4.0.28-py3.yml -a yaml/skopi.yml -o skopi-ana-none-4.0.28-py3.yml -r skopi-ana -t none-4.0.28-py3 -u fpoitevi
>> Reading base environment YAML file: yaml/ana-4.0.28-py3.yml
Traceback (most recent call last):
  File "./create-container.py", line 108, in <module>
    base_env = yaml.safe_load(fh)
AttributeError: module 'yaml' has no attribute 'safe_load'```