enable configurable logging level via configuration about lithops

[JosepSampe]

Instead of configuration, wich is fixed for all pywren executions, IBM-PyWren already contains a parameter to set internal log level:

pw = pywren.ibm_cf_executor(log_level='DEBUG')

However, code executed in CF does not get this configuration. It should be propagated.

[gilv]

The default if log_level is not provided should be INFO (to reduce number of log prints )

[gilv]

Adding more to this thread, logging level should get to cos.py, that currently contains

logging.getLogger('ibm_boto3').setLevel(logging.CRITICAL)
logging.getLogger('ibm_botocore').setLevel(logging.CRITICAL)
logging.getLogger('urllib3').setLevel(logging.CRITICAL) 
logger = logging.getLogger(__name__)

[gilv]

@omerb01 can you please handle this?

[gilv]

@JosepSampe we can provide log level as you suggested

 pw = pywren.ibm_cf_executor(log_level='DEBUG')

but this one also works, what is the difference, if any?

 import logging
 logging.basicConfig(level=logging.DEBUG)
 pw = pywren.ibm_cf_executor()

[JosepSampe]

@gilv They are two different ways to set logging level.
The one integrated in PyWren (pw = pywren.ibm_cf_executor(log_level='DEBUG')) uses the wrenlogging.py module, that uses a personalized log format:

2019-03-20 08:51:08,083 [INFO] pywren_ibm_cloud.cf_connector: IBM Cloud Functions init for namespace: [email protected]_us_east
2019-03-20 08:51:08,083 [INFO] pywren_ibm_cloud.cf_connector: IBM Cloud Functions init for host: https://us-east.functions.cloud.ibm.com
2019-03-20 08:51:08,083 [INFO] pywren_ibm_cloud.invokers: IBM Cloud Functions init for Runtime: pywren_3.6
2019-03-20 08:51:08,626 [INFO] pywren_ibm_cloud.executor: IBM Cloud Functions executor created with ID 8425f7a3-fed7

In contrast, the other case you need to import the logging module and manually set the logging.basicConfig, which has another log format:

INFO:pywren_ibm_cloud.cf_connector:IBM Cloud Functions init for namespace: [email protected]_us_east
INFO:pywren_ibm_cloud.cf_connector:IBM Cloud Functions init for host: https://us-east.functions.cloud.ibm.com
INFO:pywren_ibm_cloud.invokers:IBM Cloud Functions init for Runtime: pywren_3.6
INFO:pywren_ibm_cloud.executor:IBM Cloud Functions executor created with ID 484b1049-4eb3

Maybe from a user perspective it is easier to set the log level parameter in the executor rather than have to import the logging package and manually create the logging config in each file.

[omerb01]

@gilv logging level gets already to cos.py
do you want to get information from packages ibm_boto3, ibm_botocore and urllib3 also? cause currently for these packages, logging level is set to CRITICAL only.

[gilv]

@omerb01 this is cos.py looks like

logging.getLogger('ibm_boto3').setLevel(logging.DEBUG)
logging.getLogger('ibm_botocore').setLevel(logging.CRITICAL)
logging.getLogger('urllib3').setLevel(logging.CRITICAL)
logger = logging.getLogger(name)

logging doesn't get there.

@omerb01 let's folllow suggestion from @JosepSampe above.

[omerb01]

@gilv as we discussed, added an option to get boto3 logs on pull request #100

usage:

pw = pywren.ibm_cf_executor(log_level='DEBUG_BOTO3')

[JosepSampe]

This issue is finally fixed in pull #102
Now functions have same log_level as the local PyWren