pgierz / esm_analysis Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
It turns out that there is a permission error when you try to mark an experiment that isn't yours (duh).
Enter the top-level directory of your experiment: /work/ba0989/a270077/CMIP6_PMIP4/a270073/CMIP6/CMIP_PMIP/dynveg_true/1percCO2/
Making marker file .top_of_exp_tree in /work/ba0989/a270077/CMIP6_PMIP4/a270073/CMIP6/CMIP_PMIP/dynveg_true/1percCO2
Press Enter to continue, Ctrl-C to canel...
Traceback (most recent call last):
File "/pf/a/a270077/.local/bin/esm_analysis", line 11, in <module>
load_entry_point('esm-analysis', 'console_scripts', 'esm_analysis')()
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/mnt/lustre01/pf/a/a270077/Code/esm_analysis/esm_analysis/cli.py", line 44, in fldmean
analyzer = EsmAnalysis(preferred_analysis_dir=preferred_analysis_dir)
File "/mnt/lustre01/pf/a/a270077/Code/esm_analysis/esm_analysis/esm_analysis.py", line 109, in __init__
with open(self.EXP_BASE + "/.top_of_exp_tree", "w") as f:
PermissionError: [Errno 13] Permission denied: '/work/ba0989/a270077/CMIP6_PMIP4/a270073/CMIP6/CMIP_PMIP/dynveg_true/1percCO2//.top_of_exp_tree'
0.4.2
Lars was following the example for fldmean temp2 on Mistral. There seems to be a problem with the echam fldmean function call, it still wants file lists.
This will generate a fldmean for: temp2
You passed in preferred_analysis_dir: None
WARNING:root:Oops: Trouble initializing or no analysis class available for: oasis3mct
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: hdmodel
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: esm_analysis.dkrz.log.02
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: jsbach
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: pism
ERROR:root:Error was: <class 'ModuleNotFoundError'>
Load 2d part of the grid in 2 second(s)
Traceback (most recent call last):
File "/pf/a/a270124/.local/bin/esm_analysis", line 8, in <module>
sys.exit(main())
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/cli.py", line 71, in fldmean
analyzer.fldmean(varname)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/esm_analysis.py", line 486, in fldmean
return component.fldmean(varname)
TypeError: fldmean() missing 1 required positional argument: 'file_list'
The best way to get around this is to make sure file_list isn't used anywhere
in any of the calls from the EsmAnalysis instance. Everything should just work
with variable names; or ask questions if it gets stuck.
Docs aren't showing up where they should be?
This will generate a fldmean for: temp2
You passed in preferred_analysis_dir: None
Traceback (most recent call last):
File "/home/ollie/lackerma/.local/bin/esm_analysis", line 8, in <module>
sys.exit(main())
File "/global/AWIsoft/intel/2019/intelpython3/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/global/AWIsoft/intel/2019/intelpython3/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/global/AWIsoft/intel/2019/intelpython3/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/global/AWIsoft/intel/2019/intelpython3/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/global/AWIsoft/intel/2019/intelpython3/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/ollie/lackerma/.local/lib/python3.6/site-packages/esm_analysis/cli.py", line 67, in fldmean
analyzer = EsmAnalysis(preferred_analysis_dir=preferred_analysis_dir)
File "/home/ollie/lackerma/.local/lib/python3.6/site-packages/esm_analysis/esm_analysis.py", line 151, in __init__
self.CDO = cdo.Cdo()
File "/home/ollie/lackerma/.local/lib/python3.6/site-packages/cdo.py", line 144, in __init__
self.operators = self.__getOperators()
File "/home/ollie/lackerma/.local/lib/python3.6/site-packages/cdo.py", line 190, in __getOperators
version = parse_version(getCdoVersion(self.CDO))
File "/home/ollie/lackerma/.local/lib/python3.6/site-packages/cdo.py", line 73, in getCdoVersion
return match.group(1)
AttributeError: 'NoneType' object has no attribute 'group'
0.4.2
3.6
Looks like there is a problem when on;y one particular component has a
variable, in this case so
. Probably I need to do a check if the list is empty
or not and check if it only has one entry. This should ideally throw a more
useful error message...
This will generate a fldmean for: so
You passed in preferred_analysis_dir: None
WARNING:root:Oops: Trouble initializing or no analysis class available for: oasis3mct
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: hdmodel
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: esm_analysis.dkrz.log.02
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: jsbach
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: pism
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: esm_analysis.dkrz.log.03
ERROR:root:Error was: <class 'ModuleNotFoundError'>
Load 2d part of the grid in 2 second(s)
Traceback (most recent call last):
File "/pf/a/a270124/.local/bin/esm_analysis", line 8, in <module>
sys.exit(main())
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/cli.py", line 71, in fldmean
analyzer.fldmean(varname)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/esm_analysis.py", line 486, in fldmean
return component.fldmean(varname)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/esm_analysis.py", line 485, in fldmean
component = self.get_component_for_variable_short_name(varname)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/esm_analysis.py", line 476, in get_component_for_variable_short_name
return multi_comps[0]
IndexError: list index out of range
Currently, if a requested variable is found in more than one file tag (e.g. temp2
in both echam6_echam
and echam6_ma
), the program asks each time which files should be taken. This preference could be stored (and, if needed, ignored)
This will generate a fldmean for: temp2
You passed in preferred_analysis_dir: None
WARNING:root:Oops: Trouble initializing or no analysis class available for: oasis3mct
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: hdmodel
ERROR:root:Error was: <class 'ModuleNotFoundError'>
WARNING:root:Oops: Trouble initializing or no analysis class available for: fesom
ERROR:root:Error was: <class 'AttributeError'>
Traceback (most recent call last):
File "/pf/a/a270124/.local/bin/esm_analysis", line 8, in <module>
sys.exit(main())
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/cli.py", line 69, in fldmean
preferred_analysis_dir=preferred_analysis_dir
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/esm_analysis.py", line 202, in initialize_analysis_components
)(exp_base=self.EXP_BASE, preferred_analysis_dir=preferred_analysis_dir)
File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_analysis/components/fesom.py", line 49, in __init__
self._config = self._config.get("fesom", {})
AttributeError: 'str' object has no attribute 'get'
Before I give esm-analysis
to the group, I'd like FESOM
support.
Currently, I can easily load data into an xarray dataframe. However, there are
currently MemoryError
s occuring as soon as I try to use the multi-file
dataset to do anything. An example will follow.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.