Flair or one of its imports resets warning settings about flair

[hcl14]

Hi @tabergma , please excuse me for the late response. I did further investigation and found that the line which actually resets warnings is

tagger = flair.models.SequenceTagger.load('ner') 

You can check that when this line is commented, everything is Ok and warning settings from __init__ are applied. I investigated it further:

in models/sequence_tagger_model.py in class method:

    @classmethod
    def load_from_file(cls, model_file):

        warnings.filterwarnings("ignore")
        state = torch.load(model_file, map_location={'cuda:0': 'cpu'})
        warnings.filterwarnings("default")

Which seems to be made because of a lot of warnings issued by torch.

I propose to change this function according to this Stackoverflow answer:

    @classmethod
    def load_from_file(cls, model_file):

        with warnings.catch_warnings():
            warnings.filterwarnings("ignore")
            state = torch.load(model_file, map_location={'cuda:0': 'cpu'})
        
        model = SequenceTagger(
            hidden_size=state['hidden_size'],
            embeddings=state['embeddings'],
            tag_dictionary=state['tag_dictionary'],
            tag_type=state['tag_type'],
            use_crf=state['use_crf'],
            use_rnn=state['use_rnn'],
            rnn_layers=state['rnn_layers'])

        model.load_state_dict(state['state_dict'])
        model.eval()
        if torch.cuda.is_available():
            model = model.cuda()
        return model

The all warning issued by torch will be cought, but global settings not changed. Setting warning settings in __init__ is redundant then.

Perhaps there might exist another places in the code, where such settings occur, they might be changed as well.

[tabergma]

Hi @hcl14! I could not reproduce the issue you described. Is it still relevant? What kind of system are you using?

[hcl14]

Hi @tabergma ,

My current system is Debian 9.5 with python 3.6 manually installed and all packages installed via pip, but the same behavior was observed on Ubuntu 18.04 with out-of-the-box python distribution and tensorflow built from source (output above).

All packages installed via pip:

$ pip3.6 freeze
absl-py==0.5.0
astor==0.7.1
awscli==1.14.32
boto==2.49.0
boto3==1.9.17
botocore==1.8.36
bz2file==0.98
certifi==2018.8.24
chardet==3.0.4
colorama==0.3.7
docutils==0.14
flair==0.2.1
gast==0.2.0
gensim==3.4.0
grpcio==1.15.0
h5py==2.8.0
idna==2.7
jmespath==0.9.3
Keras==2.2.4
Keras-Applications==1.0.6
Keras-Preprocessing==1.0.5
Markdown==2.6.11
mock==2.0.0
numpy==1.14.5
pbr==4.2.0
protobuf==3.6.1
pyasn1==0.4.4
python-dateutil==2.7.3
PyYAML==3.12
regex==2018.8.29
requests==2.19.1
rsa==3.4.2
s3transfer==0.1.13
scipy==1.1.0
segtok==1.5.6
six==1.11.0
smart-open==1.7.1
tensorboard==1.11.0
tensorflow==1.11.0
termcolor==1.1.0
torch==0.4.0
tqdm==4.23.4
typing==3.6.4
urllib3==1.23
Werkzeug==0.14.1

1. Please consider running the following script:
   test.py:

print("Loading flair NER")
from flair.models import SequenceTagger
from flair.data import Sentence

tagger = SequenceTagger.load('ner') 
print("NER Loaded")

import numpy as np
import tensorflow as tf 

The output is:

$ python3.6 test.py
Loading flair NER
NER Loaded
/usr/local/lib/python3.6/importlib/_bootstrap_external.py:426: ImportWarning: Not importing directory /home/hcl/.local/lib/python3.6/site-packages/google: missing __init__
  _warnings.warn(msg.format(portions[0]), ImportWarning)
/usr/local/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/home/hcl/.local/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:75: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  return _inspect.getargspec(target)
/home/hcl/.local/lib/python3.6/site-packages/tensorflow/python/keras/backend.py:4888: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/hcl/.keras/keras.json' mode='r' encoding='UTF-8'>
  _config = json.load(open(_config_path))

2. Consider then the script without flair:

'''
print("Loading flair NER")
from flair.models import SequenceTagger
from flair.data import Sentence

tagger = SequenceTagger.load('ner') 
print("NER Loaded")
'''
import numpy as np
import tensorflow as tf 

No garbage output created:

$ python3.6 test.py
$ 

3. Now consider the script with full silencing of warnings:

print("Loading flair NER")
from flair.models import SequenceTagger
from flair.data import Sentence

tagger = SequenceTagger.load('ner') 
print("NER Loaded")

import warnings
warnings.filterwarnings("ignore")

import numpy as np
import tensorflow as tf

The output is clear:

$ python3.6 test.py
Loading flair NER
NER Loaded

[hcl14]

The problem is that, after loading flair, I can't bring warnings back to the default level, only silence them completely ("ignore"). If I do

import warnings
warnings.resetwarnings()

it does not help.

[hcl14]

I created a script which shows which warnings are enabled by default, and which are activated by flair (all):

test.py:

print("Loading flair NER")
from flair.models import SequenceTagger
from flair.data import Sentence

tagger = SequenceTagger.load('ner') 
print("NER Loaded")


import warnings
warnings.warn("1.DeprecationWarning", DeprecationWarning)
warnings.warn("2.ImportWarning", ImportWarning)
warnings.warn("3.UserWarning", UserWarning)
warnings.warn("4.SyntaxWarning", SyntaxWarning)
warnings.warn("5.RuntimeWarning", RuntimeWarning)
warnings.warn("6.FutureWarning", FutureWarning)
warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
warnings.warn("8.UnicodeWarning", UnicodeWarning)

Running with flair:

$ python3.6 test.py
Loading flair NER
NER Loaded
test.py:10: DeprecationWarning: 1.DeprecationWarning
  warnings.warn("1.DeprecationWarning", DeprecationWarning)
test.py:11: ImportWarning: 2.ImportWarning
  warnings.warn("2.ImportWarning", ImportWarning)
test.py:12: UserWarning: 3.UserWarning
  warnings.warn("3.UserWarning", UserWarning)
test.py:13: SyntaxWarning: 4.SyntaxWarning
  warnings.warn("4.SyntaxWarning", SyntaxWarning)
test.py:14: RuntimeWarning: 5.RuntimeWarning
  warnings.warn("5.RuntimeWarning", RuntimeWarning)
test.py:15: FutureWarning: 6.FutureWarning
  warnings.warn("6.FutureWarning", FutureWarning)
test.py:16: PendingDeprecationWarning: 7.PendingDeprecationWarning
  warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
test.py:17: UnicodeWarning: 8.UnicodeWarning
  warnings.warn("8.UnicodeWarning", UnicodeWarning)

Running with flair commented out:

$ python3.6 test.py
test.py:13: UserWarning: 3.UserWarning
  warnings.warn("3.UserWarning", UserWarning)
test.py:14: SyntaxWarning: 4.SyntaxWarning
  warnings.warn("4.SyntaxWarning", SyntaxWarning)
test.py:15: RuntimeWarning: 5.RuntimeWarning
  warnings.warn("5.RuntimeWarning", RuntimeWarning)
test.py:16: FutureWarning: 6.FutureWarning
  warnings.warn("6.FutureWarning", FutureWarning)
test.py:18: UnicodeWarning: 8.UnicodeWarning
  warnings.warn("8.UnicodeWarning", UnicodeWarning)

[tabergma]

With the next release (coming today, latest tomorrow morning) we have the following warning settings set in flair:

import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
warnings.filterwarnings("ignore", category=ImportWarning)
warnings.filterwarnings("ignore", category=PendingDeprecationWarning)
warnings.filterwarnings("ignore", category=ResourceWarning) 

Can you please try again after the release and let us know, if the warnings still appear for you? Thank you.

[hcl14]

Thanks for answering, @tabergma . I'm not sure I understood you clearly: will flair still reset warning settings? I thought it's best to follow existing warning settings which are set in parent program.

[tabergma]

According to https://docs.python.org/3/library/warnings.html#describing-warning-filters only a few warnings are set by default. As some dependency apparently reset all warnings, I set the warning settings to the default warning settings again.
If you have any other suggestion, please let me know.

[hcl14]

Ok, I understood, so that's not flair resets warnings.
Well, googling says that one can actually get active warning filters, so it's theoretically possible to capture them before all other imports:

print("Loading flair NER")

import warnings
print(warnings.filters)

from flair.models import SequenceTagger
from flair.data import Sentence

tagger = SequenceTagger.load('ner') 
print("NER Loaded")

print('\n')
print(warnings.filters)
print('\n')

#import warnings
warnings.warn("1.DeprecationWarning", DeprecationWarning)
warnings.warn("2.ImportWarning", ImportWarning)
warnings.warn("3.UserWarning", UserWarning)
warnings.warn("4.SyntaxWarning", SyntaxWarning)
warnings.warn("5.RuntimeWarning", RuntimeWarning)
warnings.warn("6.FutureWarning", FutureWarning)
warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
warnings.warn("8.UnicodeWarning", UnicodeWarning)

The strange thing for me is that those filters do not seem to change after loading flair, and are still 'ignore', while warnings are issued:

$ python3.6 test.py
Loading flair NER
[('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'BytesWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)]
NER Loaded

[('default', re.compile('', re.IGNORECASE), <class 'Warning'>, re.compile(''), 0), ('ignore', re.compile('', re.IGNORECASE), <class 'Warning'>, re.compile(''), 0), ('always', None, <class 'scipy.special.sf_error.SpecialFunctionWarning'>, None, 0), ('ignore', re.compile('A true SSLContext object is not available.*', re.IGNORECASE), <class 'botocore.vendored.requests.packages.urllib3.exceptions.InsecurePlatformWarning'>, re.compile('.*urllib3\.util\.ssl_'), 0), ('ignore', None, <class 'urllib3.exceptions.DependencyWarning'>, None, 0), ('ignore', re.compile('numpy.ndarray size changed', re.IGNORECASE), <class 'Warning'>, re.compile(''), 0), ('ignore', re.compile('numpy.ufunc size changed', re.IGNORECASE), <class 'Warning'>, re.compile(''), 0), ('ignore', re.compile('numpy.dtype size changed', re.IGNORECASE), <class 'Warning'>, re.compile(''), 0), ('always', None, <class 'numpy.lib.polynomial.RankWarning'>, None, 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'BytesWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0), ('always', None, <class 'urllib3.exceptions.SecurityWarning'>, None, 0), ('default', None, <class 'urllib3.exceptions.SubjectAltNameWarning'>, None, 0), ('default', None, <class 'urllib3.exceptions.InsecurePlatformWarning'>, None, 0), ('default', None, <class 'urllib3.exceptions.SNIMissingWarning'>, None, 0), ('default', None, <class 'requests.exceptions.FileModeWarning'>, None, 0), ('always', None, <class 'botocore.vendored.requests.packages.urllib3.exceptions.SecurityWarning'>, None, 0), ('default', None, <class 'botocore.vendored.requests.packages.urllib3.exceptions.InsecurePlatformWarning'>, None, 0)]

test.py:20: DeprecationWarning: 1.DeprecationWarning
warnings.warn("1.DeprecationWarning", DeprecationWarning)
test.py:21: ImportWarning: 2.ImportWarning
warnings.warn("2.ImportWarning", ImportWarning)
test.py:22: UserWarning: 3.UserWarning
warnings.warn("3.UserWarning", UserWarning)
test.py:23: SyntaxWarning: 4.SyntaxWarning
warnings.warn("4.SyntaxWarning", SyntaxWarning)
test.py:24: RuntimeWarning: 5.RuntimeWarning
warnings.warn("5.RuntimeWarning", RuntimeWarning)
test.py:25: FutureWarning: 6.FutureWarning
warnings.warn("6.FutureWarning", FutureWarning)
test.py:26: PendingDeprecationWarning: 7.PendingDeprecationWarning
warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
test.py:27: UnicodeWarning: 8.UnicodeWarning
warnings.warn("8.UnicodeWarning", UnicodeWarning)

So I have no idea what might cause that.
Will wait for release then to confirm that warnings are set to default level.

[hcl14]

Hi @tabergma , I still don't see any new release neither here on github (latest is 2.0), nor in pip repository. Am I looking in the wrong places or it's not out yet?

[tabergma]

@hcl14 Sorry for the delay. We just published the new release.

[hcl14]

Hi @tabergma , please excuse me for the late response. I upgraded to flair 0.3.1 from pip:

$ pip3.6 freeze | grep flair
flair==0.3.1

But there seems no change neither when I import only from flair.models and flair.data:

print("Loading flair NER")

import warnings
print(warnings.filters)

from flair.models import SequenceTagger
from flair.data import Sentence

tagger = SequenceTagger.load('ner') 
print("NER Loaded")

'''
print('\n')
print(warnings.filters)
'''


#import warnings
warnings.warn("1.DeprecationWarning", DeprecationWarning)
warnings.warn("2.ImportWarning", ImportWarning)
warnings.warn("3.UserWarning", UserWarning)
warnings.warn("4.SyntaxWarning", SyntaxWarning)
warnings.warn("5.RuntimeWarning", RuntimeWarning)
warnings.warn("6.FutureWarning", FutureWarning)
warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
warnings.warn("8.UnicodeWarning", UnicodeWarning)

Output:

$ python3.6 -i test.py
Loading flair NER
[('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'BytesWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)]
NER Loaded
test.py:19: DeprecationWarning: 1.DeprecationWarning
  warnings.warn("1.DeprecationWarning", DeprecationWarning)
test.py:20: ImportWarning: 2.ImportWarning
  warnings.warn("2.ImportWarning", ImportWarning)
test.py:21: UserWarning: 3.UserWarning
  warnings.warn("3.UserWarning", UserWarning)
test.py:22: SyntaxWarning: 4.SyntaxWarning
  warnings.warn("4.SyntaxWarning", SyntaxWarning)
test.py:23: RuntimeWarning: 5.RuntimeWarning
  warnings.warn("5.RuntimeWarning", RuntimeWarning)
test.py:24: FutureWarning: 6.FutureWarning
  warnings.warn("6.FutureWarning", FutureWarning)
test.py:25: PendingDeprecationWarning: 7.PendingDeprecationWarning
  warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
test.py:26: UnicodeWarning: 8.UnicodeWarning
  warnings.warn("8.UnicodeWarning", UnicodeWarning)

Nor when I import flair package directly hoping to reset warning settings:

print("Loading flair NER")

import warnings
print(warnings.filters)

import flair
#from flair.models import SequenceTagger
#from flair.data import Sentence

tagger = flair.models.SequenceTagger.load('ner') 
print("NER Loaded")

'''
print('\n')
print(warnings.filters)
'''


#import warnings
warnings.warn("1.DeprecationWarning", DeprecationWarning)
warnings.warn("2.ImportWarning", ImportWarning)
warnings.warn("3.UserWarning", UserWarning)
warnings.warn("4.SyntaxWarning", SyntaxWarning)
warnings.warn("5.RuntimeWarning", RuntimeWarning)
warnings.warn("6.FutureWarning", FutureWarning)
warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
warnings.warn("8.UnicodeWarning", UnicodeWarning)

Result:

$ python3.6 -i test.py
Loading flair NER
[('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'BytesWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)]
NER Loaded
test.py:20: DeprecationWarning: 1.DeprecationWarning
  warnings.warn("1.DeprecationWarning", DeprecationWarning)
test.py:21: ImportWarning: 2.ImportWarning
  warnings.warn("2.ImportWarning", ImportWarning)
test.py:22: UserWarning: 3.UserWarning
  warnings.warn("3.UserWarning", UserWarning)
test.py:23: SyntaxWarning: 4.SyntaxWarning
  warnings.warn("4.SyntaxWarning", SyntaxWarning)
test.py:24: RuntimeWarning: 5.RuntimeWarning
  warnings.warn("5.RuntimeWarning", RuntimeWarning)
test.py:25: FutureWarning: 6.FutureWarning
  warnings.warn("6.FutureWarning", FutureWarning)
test.py:26: PendingDeprecationWarning: 7.PendingDeprecationWarning
  warnings.warn("7.PendingDeprecationWarning", PendingDeprecationWarning)
test.py:27: UnicodeWarning: 8.UnicodeWarning
  warnings.warn("8.UnicodeWarning", UnicodeWarning)

[tabergma]

I'm sorry, but I'm not sure how to help you further.

Do you have any suggestion how to solve the problem?

[alanakbik]

@hcl14 hey thanks this looks like it could address the issue!

would you like to do a pull request to add this?

[tabergma]

@hcl14 Thanks for finding this! The same code is used in our text classification model (see https://github.com/zalandoresearch/flair/blob/master/flair/models/text_classification_model.py#L89).

[hcl14]

@tabergma @alanakbik Created pull request: #186

[hcl14]

@tabergma @alanakbik Tested flair from newly merged master branch, issue seems resolved.

[alanakbik]

Awesome! Thanks again for your help!