turicas / salarios-magistrados

Baixa as planilhas de salários de magistrados, extrai os contracheques, limpa e exporta pra CSV

Salários Magistrados - CNJ

Script que baixa todas as planilhas de salários de magistrados do site do CNJ, extrai a aba "Contracheque", faz algumas limpezas e exporta tudo para CSV.

Licença

A licença do código é LGPL3 e dos dados convertidos Creative Commons Attribution ShareAlike. Caso utilize os dados, cite a fonte original e quem tratou os dados, como: Fonte: Conselho Nacional de Justiça, dados tratados por Álvaro Justen/Brasil.IO. Caso compartilhe os dados, utilize a mesma licença.

Dados

Caso você não queira/possa rodar o script, acesse diretamente os dados convertidos no Brasil.IO.

Se esse programa e/ou os dados resultantes foram úteis a você ou à sua empresa, considere fazer uma doação ao projeto Brasil.IO, que é mantido voluntariamente.

Erros nos Dados

Nem todas as planilhas puderam ser convertidas. Verifique o arquivo erros.csv para entender quais erros existem nos dados originais e como isso se propaga para os dados gerados pelo script.

Encontrou algum erro na conversão que o script faz? Crie uma issue nesse repositório.

Rodando

Esse script depende de Python 3.7+ e de algumas bibliotecas. Instale-as executando:

pip install -r requirements.txt

Para rodar:

./run.sh

Esse script irá rodar dois scripts, um que baixa as planilhas e outro que as extrai e gera o resultado. Você pode rodá-los independentemente também:

# Baixa planilhas e gera `data/output/planilha.csv`:
scrapy runspider --loglevel=INFO -o data/output/planilha.csv download_files.py
gzip data/output/planilha.csv

# Lê `data/output/planilha.csv.gz` e gera outros arquivos em `data/output`:
python parse_files.py

Um diretório data será criado, onde:

• data/download: planilhas baixadas;
• data/output: arquivos de saída (CSVs compactados).

salarios-magistrados's Issues

Inconsistência no SQLite?

Com o arquivo salarios-magistrados-2018-03-05 parece ter um registro com erro de importação.

Encontrei com esta query:
select nome, sum(total_de_rendimentos) as v from table1 group by nome order by v desc;

E todo este valor vem deste registro:
select * from table1 where nome = '0';

O arquivo original parece normal

http://www.cnj.jus.br/files/conteudo/arquivo/2018/02/42ed31339fa078281424080592f96e49.xls

Alterar lógica para pegar mês atual

A página inicial do dataset no CNJ mostra sempre o mês mais atual, porém ele não está listado no menu e, por isso, o script não está fazendo a captura.

Github Page

Olá @turicas gostei muito da iniciativa.
Que tal criar uma gh page com os dados estruturados? Não tenho conhecimento em python, então a ideia é gerar um json para consumir.

Corrigir e uniformizar coluna "mesano_de_referencia"

Quase todos os dados da coluna mesano_de_referencia possuem o formato YYYY-MM-DD (que facilita importar esse campo como DATE em sistemas de bancos de dados para fazer filtros), porém alguns registros estão fora desse padrão e precisamos convertê-los antes de salvar na planilha final:

• Valor 01/2018, arquivo 05b56035f8725b28c6c53dee91074a5d.xls
• Valor #NAME? (fórmula: ='dados cadastrais'!C3), arquivo 9f1500172b6e26c1663abb37d7b951b2.xlsx

List index out of range

O script está criando um csv vazio na pasta output

Traceback (most recent call last):
File "salarios_magistrados.py", line 196, in
main()
File "salarios_magistrados.py", line 192, in main
export_csv(result, output)
File "salarios_magistrados.py", line 138, in export_csv
writer = csv.DictWriter(fobj, fieldnames=data[0].keys())
IndexError: list index out of range

Script não roda: IndexError: list index out of range

Traceback (most recent call last):
File "salarios_magistrados.py", line 196, in
main()
File "salarios_magistrados.py", line 192, in main
export_csv(result, output)
File "salarios_magistrados.py", line 138, in export_csv
writer = csv.DictWriter(fobj, fieldnames=data[0].keys())
IndexError: list index out of range

Extrair dados históricos

Apesar de o script de captura rodar de tempos em tempos, podemos rodá-lo uma vez só e capturar os dados históricos, dado que agora o CNJ disponibiliza links (menu à esquerda) para meses anteriores.
http://www.cnj.jus.br/transparencia/remuneracao-dos-magistrados/

Erro na execução: module 'rows' has no attribute 'import_from_xpath'

Após ter instalado o python 3.6 e demais dependências, tentei executar o script, mas ocorreu o seguinte erro:

Traceback (most recent call last):
File "salarios_magistrados.py", line 197, in
main()
File "salarios_magistrados.py", line 162, in main
links = get_links(date=today)
File "salarios_magistrados.py", line 42, in get_links
table = rows.import_from_xpath(
AttributeError: module 'rows' has no attribute 'import_from_xpath'

Extrair dados de outras abas

Atualmente somente a aba "Contracheque" está sendo extraída, o ideal seria extrair todas as abas disponíveis.

Problemas com encoding no Mac OSX

O seguinte erro acontece quando executo no mac:

$ python salarios_magistrados.py
File "salarios_magistrados.py", line 51
SyntaxError: Non-ASCII character '\xc3' in file salarios_magistrados.py on line 51, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Separar data de captura com mês/data de referência

Atualmente a planilha gerada contém o mês de referência do contracheque, mas dependendo da atualização dos links no site do CNJ podemos ter uma planilha final com meses diferentes para cada tribunal. Idealmente deveríamos ter uma planilha por mês de referência como saída. Um possível caminho é guardar em uma planilha intermediária (como a output/links-YYYY-MM-DD.csv) o link para cada mês/ano de referência para cada tribunal e, a partir dessa planilha, gerar a planilha final para o mês/ano de referência desejado (ou o mais atual).

erro ao executar script Ubuntu 16.04

Olá.
Esta é a mensagem de erro ao executar o script no Ubuntu 16.04 (python 3.5).

ramon@ramon:~/git/salarios-magistrados$ python3 salarios_magistrados.py
File "salarios_magistrados.py", line 162
rows.export_to_csv(links, output_path / f'links-{today}.csv')
^
SyntaxError: invalid syntax

Muito obrigado por seu trabalho!

Não executa os downloads

Bom dia!

Ao executar o 'scrapy runspider --loglevel=INFO -o data/output/planilha.csv download_files.py
gzip data/output/planilha.csv', aparece a seguinte mensagem:

(base) C:\Users\henri\Downloads\PythonScripts\Lerningwithcodes\Rows\salarios-magistrados-develop>scrapy runspider --loglevel=INFO -o data/output/planilha.csv download_files.py
2020-04-01 11:40:42 [scrapy.utils.log] INFO: Scrapy 2.0.1 started (bot: scrapybot)
2020-04-01 11:40:42 [scrapy.utils.log] INFO: Versions: lxml 4.5.0.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1.5.2, w3lib 1.21.0, Twisted 20.3.0, Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1d 10 Sep 2019), cryptography 2.7, Platform Windows-10-10.0.18362-SP0
Traceback (most recent call last):
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\henri\AppData\Local\Continuum\anaconda3\Scripts\scrapy.exe_main.py", line 9, in
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\cmdline.py", line 145, in execute
_run_print_help(parser, _run_command, cmd, args, opts)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\cmdline.py", line 99, in _run_print_help
func(*a, **kw)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\cmdline.py", line 153, in _run_command
cmd.run(args, opts)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\commands\runspider.py", line 79, in run
module = _import_file(filename)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\commands\runspider.py", line 21, in import_file
module = import_module(fname)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\importlib_init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "C:\Users\henri\Downloads\PythonScripts\Lerningwithcodes\Rows\salarios-magistrados-develop\download_files.py", line 17, in
settings.DOWNLOAD_PATH.mkdir()
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\pathlib.py", line 1258, in mkdir
self._accessor.mkdir(self, mode)
FileNotFoundError: [WinError 3] O sistema não pode encontrar o caminho especificado: 'C:\Users\henri\Downloads\PythonScripts\Lerningwithcodes\Rows\salarios-magistrados-develop\data\download'

E se eu crio o diretório \data\download manualmente ele aponta o seguinte erro ao executar o mesmo comando ou ao executar o 'python download_files.py':

(base) C:\Users\henri\Downloads\PythonScripts\Lerningwithcodes\Rows\salarios-magistrados-develop>scrapy runspider --loglevel=INFO -o data/output/planilha.csv download_files.py
2020-04-01 11:42:42 [scrapy.utils.log] INFO: Scrapy 2.0.1 started (bot: scrapybot)
2020-04-01 11:42:42 [scrapy.utils.log] INFO: Versions: lxml 4.5.0.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1.5.2, w3lib 1.21.0, Twisted 20.3.0, Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1d 10 Sep 2019), cryptography 2.7, Platform Windows-10-10.0.18362-SP0
2020-04-01 11:42:43 [scrapy.crawler] INFO: Overridden settings:
{'FEED_FORMAT': 'csv',
'FEED_URI': 'data/output/planilha.csv',
'LOG_LEVEL': 'INFO',
'SPIDER_LOADER_WARN_ONLY': True}
2020-04-01 11:42:43 [scrapy.extensions.telnet] INFO: Telnet Password: f08d1305d799891e
2020-04-01 11:42:43 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats']
2020-04-01 11:42:43 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2020-04-01 11:42:43 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2020-04-01 11:42:43 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2020-04-01 11:42:43 [scrapy.core.engine] INFO: Spider opened
2020-04-01 11:42:43 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2020-04-01 11:42:43 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2020-04-01 11:42:44 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.cnj.jus.br/transparencia-cnj/remuneracao-dos-magistrados/> (referer: None)
Traceback (most recent call last):
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\utils\defer.py", line 117, in iter_errback
yield next(it)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\utils\python.py", line 345, in next
return next(self.data)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\utils\python.py", line 345, in next
return next(self.data)
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
for r in iterable:
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
for x in result:
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
for r in iterable:
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 338, in
return (_set_referer(r) for r in result or ())
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
for r in iterable:
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in
return (r for r in result or () if _filter(r))
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
for r in iterable:
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in
return (r for r in result or () if _filter(r))
File "c:\users\henri\appdata\local\continuum\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
for r in iterable:
File "C:\Users\henri\Downloads\PythonScripts\Lerningwithcodes\Rows\salarios-magistrados-develop\download_files.py", line 47, in parse
update_message.split("enviaram os dados de")[1]
IndexError: list index out of range
2020-04-01 11:42:44 [scrapy.core.engine] INFO: Closing spider (finished)
2020-04-01 11:42:44 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 842,
'downloader/request_count': 3,
'downloader/request_method_count/GET': 3,
'downloader/response_bytes': 25646,
'downloader/response_count': 3,
'downloader/response_status_count/200': 1,
'downloader/response_status_count/301': 1,
'downloader/response_status_count/302': 1,
'elapsed_time_seconds': 1.519021,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2020, 4, 1, 14, 42, 44, 596899),
'log_count/ERROR': 1,
'log_count/INFO': 10,
'response_received_count': 1,
'scheduler/dequeued': 3,
'scheduler/dequeued/memory': 3,
'scheduler/enqueued': 3,
'scheduler/enqueued/memory': 3,
'spider_exceptions/IndexError': 1,
'start_time': datetime.datetime(2020, 4, 1, 14, 42, 43, 77878)}
2020-04-01 11:42:44 [scrapy.core.engine] INFO: Spider closed (finished)

Alguém pode me ajudar?

Corrigir problemas com algumas extrações

ERROR extracting data/download/09bdb09f5321da699d04d0732725b6f0.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/113a32634fd9da880773db8d1691e6b7.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/17c4baad557fbf9c02f2126570e2d1e7.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/32308d036350e35b2183f79a60fb7bb4.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/4384b3e3b46e5ec53bf5229c8926abac.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/4384b3e3b46e5ec53bf5229c8926abac.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/4540c1c2be549915a60146f768b65fe9.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/466e6758472f262bb60ce04910a0de7a.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/53a0e3750cc307fb5597d50862f7d78e.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/76d75ed27dda34c0bf98b703316aa890.xls - xlrd.biffh.XLRDError: Workbook is encrypted
ERROR extracting data/download/807b3d6c5ff5fd82f6764185c9d70a05.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/880b09da849f4a0c1cb129db6fa7d4d4.xls - xlrd.biffh.XLRDError: Workbook is encrypted
ERROR extracting data/download/a59df917977b82f4f1a3e088874b3bb7.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/a6518deb6527cdb1305a8c6a393e7d6c.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/b6162b4e67e30f022a79ff76164b93b7.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/bab068542a761ac2c3496bf9d724388a.xls - xlrd.biffh.XLRDError: No sheet named <'Contracheque'>
ERROR extracting data/download/be0c705a49f8097edc22c95ddda1a6da.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/d1761412a7e60085a4231e67c063b9ba.xls - xlrd.biffh.XLRDError: No sheet named <'Contracheque'>
ERROR extracting data/download/ec0553c1014237418ec9779fac4b9771.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
ERROR extracting data/download/f785c052aa0c50cec0ca9218a76fff6b.xlsx - AttributeError: 'MergedCell' object has no attribute 'value'
WARNING *** file size (551546) not 512 + multiple of sector size (512)
WARNING *** file size (551546) not 512 + multiple of sector size (512)

ERROR: Spider error processing <GET https://www.cnj.jus.br/transparencia-cnj/remuneracao-dos-magistrados/> (referer: None)

Houve alguma mudança na url desde a última vez que esse repositório foi atualizado?

Eu tentei usar agora, mas aparece o erro:

[scrapy.core.scraper] ERROR: Spider error processing <GET https://www.cnj.jus.br/transparencia-cnj/remuneracao-dos-magistrados/> (referer: None)

A url https://www.cnj.jus.br/transparencia-cnj/remuneracao-dos-magistrados ainda funciona, mas para acessar os dados é preciso entrar em outra página. Fiquei na dúvida se era assim quando o repositório foi criado.

Da forma como está, o código não está construindo as bases, apenas os templates. Ao menos no meu caso.

Remover linhas cujo nome é em branco

Se for a última linha da planilha, pode significar o total, como neste arquivo - Tribunal de Justiça do Acre.

Erro na execução do script.

python salarios_magistrados.py
File "salarios_magistrados.py", line 163
rows.export_to_csv(links, output_path / f'links-{today}.csv')
^
SyntaxError: invalid syntax

Executando o script no Ubuntu 16.04 x64 com o Python 3.5.2.

ERROR! locale.Error: unsupported locale setting

Tentei executar o script, porém todos os arquivos ficaram com o erro unsupported locale setting e o CSV do output ficou vazio.

Executei no Python 3.6 do Windows 10 e no Ubuntu 17.04 (no WSL) e o resultado foi o mesmo.

Extracting 29dbd9d3ec082188394449dbc564aa12.xls... ERROR! locale.Error: unsupported locale setting
Extracting d9657da5c5c430b044079bb2cb16eae2.xls... ERROR! locale.Error: unsupported locale setting
Extracting f2501340b64611e66224aeb45767eaf3.xls... ERROR! locale.Error: unsupported locale setting
Extracting bc506d094092730aee64ccb701212763.xls... ERROR! locale.Error: unsupported locale setting
Extracting f2d41244514643191ff6d45629b3855a.xls... ERROR! locale.Error: unsupported locale setting
Extracting d018d05e8bbcfe7e028207196b73ef89.xls... ERROR! locale.Error: unsupported locale setting
Extracting d55ef67a7c20e436fa1d6167698d80ab.xlsx... ERROR! AttributeError: 'PosixPath' object has no attribute 'seek'
Extracting 46ed9711a0fcd5f70292fc754855e65f.xls... ERROR! locale.Error: unsupported locale setting
Extracting 731778983576abbbdf37a2bf3cd4778d.xls... ERROR! locale.Error: unsupported locale setting
Extracting 858736c9622609245b1d0804f12dcca6.xls... ERROR! locale.Error: unsupported locale setting
Extracting 62d78b54747c7784ad0c69733dd1840f.xls... ERROR! locale.Error: unsupported locale setting
Extracting bec48e6e752fab72acaff674b658b782.xls... ERROR! locale.Error: unsupported locale setting
Extracting ce76b51ebf98eed08f71be75f4821fac.xls... ERROR! locale.Error: unsupported locale setting
Extracting e14eeae1fca5ca5bd18bf9b5f916fd87.xls... ERROR! locale.Error: unsupported locale setting
Extracting 34d32ee7a5856ed495229eff7598cdc8.xls... ERROR! locale.Error: unsupported locale setting
Extracting a87206d2e30bea3b98b5069637e23345.xls... ERROR! locale.Error: unsupported locale setting
Extracting 880b09da849f4a0c1cb129db6fa7d4d4.xls... ERROR! xlrd.biffh.XLRDError: Workbook is encrypted
Extracting 1e822641403083c6eea6aa6f0de8926c.xls... ERROR! locale.Error: unsupported locale setting
Extracting 7331383ae787372d4516f9a57e225b29.xls... ERROR! locale.Error: unsupported locale setting
Extracting 4bd6abae6161f3cff8ef22a6dfc75eb2.xls... ERROR! locale.Error: unsupported locale setting
Extracting e9c5203e1754365d32d0d54694b52f06.xls... ERROR! locale.Error: unsupported locale setting
Extracting 1430a6cc5e45e8f8aaf2005cffafe0e2.xls... ERROR! locale.Error: unsupported locale setting
Extracting 80a61ce4eabf38fbe244487b26cab618.xls... ERROR! locale.Error: unsupported locale setting
Extracting a1a46203a1e70463dff818be0d36dc42.xls... ERROR! locale.Error: unsupported locale setting
Extracting 2ef84d431d5458e277a808c24f97e131.xls... ERROR! locale.Error: unsupported locale setting
Extracting 807b3d6c5ff5fd82f6764185c9d70a05.xlsx... ERROR! AttributeError: 'PosixPath' object has no attribute 'seek'
Extracting 9b005d01801aeb8229fd2441f3fa0266.xls... ERROR! locale.Error: unsupported locale setting
Extracting cbbdf731cd71ba3cff033ccb11a40f53.xls... ERROR! locale.Error: unsupported locale setting
Extracting 6b51f50a8db991b2423f744c20241e58.xls... ERROR! locale.Error: unsupported locale setting
Extracting a210b7d9ad4cb93505c641fdec72eedf.xls... ERROR! locale.Error: unsupported locale setting
Extracting b3ba528d0792694f0ce73fedb2ac5a5b.xls... ERROR! locale.Error: unsupported locale setting
Extracting 76b0f4b6aed4c78f8d725c1d64547d27.xls... ERROR! locale.Error: unsupported locale setting
Extracting a5aa9d016d1c0d22f42c15a77d3d1167.xls... ERROR! locale.Error: unsupported locale setting
Extracting 537bc6a598497a28f1bba19bbe0be32c.xls... ERROR! locale.Error: unsupported locale setting
Extracting 1ca33e571e5797806b060e01bd5abb9f.xls... ERROR! locale.Error: unsupported locale setting
Extracting 5e822f30d79fe11007547ad9ca6d80e4.xls... ERROR! locale.Error: unsupported locale setting
Extracting 10fd1425f1f3fb34ebdfdff28abff802.xls... ERROR! locale.Error: unsupported locale setting
Extracting 1a16cf850bc7f356fc75b2610e6881da.xls... ERROR! locale.Error: unsupported locale setting
Extracting c5d6b706a418cfa2645820b9439f8df9.xls... ERROR! locale.Error: unsupported locale setting
Extracting 678d0c376d63872e42ceb30f25e436dc.xls... ERROR! locale.Error: unsupported locale setting
Extracting 555e727f51f9f4e168355093ea0ea63f.xls... ERROR! locale.Error: unsupported locale setting
Extracting 255932e6ae9eced3a25435aaf164caa0.xls... ERROR! locale.Error: unsupported locale setting
Extracting 6ea7db9193f0f499a772d1211ab8ee8d.xls... ERROR! locale.Error: unsupported locale setting
Extracting 9e1775c2a5033c3e1e94b3332b3e0309.xls... ERROR! locale.Error: unsupported locale setting
Extracting 55a9b27f8437ab27b71923ecfad52321.xls... ERROR! locale.Error: unsupported locale setting
Extracting 01ab246f5bf697d41dbb5db9678cca85.xls... ERROR! locale.Error: unsupported locale setting
Extracting e7bff9330e4034a33d15648d250a8ab4.xls... ERROR! locale.Error: unsupported locale setting
Extracting 17c4baad557fbf9c02f2126570e2d1e7.xlsx... ERROR! AttributeError: 'PosixPath' object has no attribute 'seek'
Extracting 8a16186e1ae5b46cd2279db1a784114f.xls... ERROR! locale.Error: unsupported locale setting
Extracting f1e38beec1e142e975f349b3b327e175.xls... ERROR! locale.Error: unsupported locale setting
Extracting 926598d64fb1a68718a346c7ecfda4bf.xls... ERROR! locale.Error: unsupported locale setting
Extracting 4a4b574f159cd7c3599bbe88defc5f93.xls... ERROR! locale.Error: unsupported locale setting
Extracting 0da5135dfa080e9753fb29b1adb224e6.xls... ERROR! locale.Error: unsupported locale setting
Extracting ea0e8c7a2c5950ff2d43a6c5b904db1b.xls... ERROR! locale.Error: unsupported locale setting
Extracting 5be125a2505f0a786e0d0ee2c761235a.xls... ERROR! locale.Error: unsupported locale setting
Extracting ccc9af81a772f9677664ba8930f6d793.xls... ERROR! locale.Error: unsupported locale setting
Extracting 76d75ed27dda34c0bf98b703316aa890.xls... ERROR! xlrd.biffh.XLRDError: Workbook is encrypted
Extracting 7e7e0f4cf7c02020b6ba87af4d72b7df.xls... ERROR! locale.Error: unsupported locale setting
Extracting 0f0106832fd80792b1f5d0e530deca71.xls... ERROR! locale.Error: unsupported locale setting
Extracting 584b48b8aa7557ee4715f9af11702b1f.xls... ERROR! locale.Error: unsupported locale setting
Extracting 405307b91e25735d1c904c28c7fe9278.xls... ERROR! locale.Error: unsupported locale setting
Extracting ea0b60b31ec7fdaf910bfc1ebbe67391.xls... ERROR! locale.Error: unsupported locale setting
Extracting 5919e4911f4b308a6828521384d77099.xls... ERROR! locale.Error: unsupported locale setting
Extracting c29feefebab8aa9b61cd1aa489299ecc.xls... ERROR! locale.Error: unsupported locale setting
Extracting 0c57450e62e30c13ddf67ce04a98009d.xls... ERROR! locale.Error: unsupported locale setting
Extracting 6693d4df7eca0afc01fbb3f4a4128b33.xls... ERROR! locale.Error: unsupported locale setting
Extracting 771fdf0733554ec4970802cd7167deb6.xls... ERROR! locale.Error: unsupported locale setting
Extracting 32dc6570569ce0b878874e322d206916.xls... ERROR! locale.Error: unsupported locale setting
Extracting e2770aee8bb3780f8b35e8254156967e.xls... ERROR! locale.Error: unsupported locale setting
Extracting e861cd842ca6b1d75aca6b8caa3ce039.xls... ERROR! locale.Error: unsupported locale setting
Extracting cfb6f3358660bd02aa89c815fe367069.xls... ERROR! locale.Error: unsupported locale setting
Extracting 7a0d51d24163565a87fba497958c9bf5.xls... ERROR! locale.Error: unsupported locale setting
Extracting 9fd2d3b23ef1b8042e895129f4029822.xls... ERROR! locale.Error: unsupported locale setting
Extracting 44d37e5d2efb352d343c20c9f6cb82b3.xls... ERROR! locale.Error: unsupported locale setting
Extracting b66895928f6b9bc054da290dd11c223c.xls... ERROR! locale.Error: unsupported locale setting
Extracting d87e4e887a7e5e20c8e82c4b0d21adfb.xls... ERROR! locale.Error: unsupported locale setting
Extracting 1f1ada0d9fdd4ed009ef65077ede7667.xls... ERROR! locale.Error: unsupported locale setting
Extracting bf2344e8ba4d261689b0bc8ca995769e.xls... ERROR! locale.Error: unsupported locale setting
Extracting a2484d14aa2c49b51a849fbb70e25723.xls... ERROR! locale.Error: unsupported locale setting
Extracting 95c42d31111ba11f6ae0db207c71b142.xls... ERROR! locale.Error: unsupported locale setting
Extracting 205177541f31dafc9516e7bb641dcb8d.xls... ERROR! locale.Error: unsupported locale setting
Extracting 10224bfa85ec6645b9bf2b7a34c15e6a.xls... ERROR! locale.Error: unsupported locale setting
Extracting 8e3419d6347bc3d61e7a877483edacd2.xls... ERROR! locale.Error: unsupported locale setting
Extracting 0b4d4aab97233da33e803a5ba889c5d5.xls... ERROR! locale.Error: unsupported locale setting
Extracting 1453eed57df8d257e5265a88be4f31b0.xls... ERROR! locale.Error: unsupported locale setting
Extracting result to output/salarios-magistrados-2017-12-20.csv...
Traceback (most recent call last):
  File "salarios_magistrados.py", line 196, in <module>
    main()
  File "salarios_magistrados.py", line 192, in main
    export_csv(result, output)
  File "salarios_magistrados.py", line 138, in export_csv
    writer = csv.DictWriter(fobj, fieldnames=data[0].keys())
IndexError: list index out of range
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 24, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File "salarios_magistrados.py", line 196, in <module>
    main()
  File "salarios_magistrados.py", line 192, in main
    export_csv(result, output)
  File "salarios_magistrados.py", line 138, in export_csv
    writer = csv.DictWriter(fobj, fieldnames=data[0].keys())
IndexError: list index out of range