robinandeer / chanjo-report

Report generator for Chanjo output

Hi there 👋

I work as a Staff Engineer at Hedvig where we're building a new kind of insurance company.

My team is responsible for the online purchase journey 🛍️. We use GraphQL, React, TypeScript, and Next.js to build a modern, scalable and e-commerse inspired shopping experience.

Personally, I'm interested in all things Web Development - from the frontend to the backend. I find it the most exciting app platform to work with today.

You can find my blog at robinandeer.com where I write about my experiences with Web Development and other things I find interesting.

Latest blog post

Progressive enhancement as productivity booster

Progressive enhancement boosts productivity when building web apps. It keep me focused and helps me break down complex features.

chanjo-report's Issues

new coverage report stuff

New reference set of exons: ref seq based

• start with NM+NR + 1. XM per gene missing other transcripts
• HGNC refseq ids (1 per gene) => to generate coverage report (based on TX not GENES)
• add new field for HGNC/canonical transcript to make query fast

OperationalError even though the table exists when exporting to PDF / HTML

Divided into multiline for simplicity:

sqlalchemy.exc.OperationalError: (OperationalError) no such table:
sample 'SELECT sample.id AS sample_id, sample.group_id AS sample_group_id, sample.cutoff AS 
sample_cutoff, sample.extension AS sample_extension, sample.coverage_source AS sample_coverage_source, sample.element_source AS sample_element_source, sample.created_at AS sample_created_at, sample.updated_at AS sample_updated_at
FROM sample
WHERE sample.id = ? ORDER BY sample.id' ('mysample',)

But:

sqlite> .tables
block            interval         interval_data    superblock     
block_data       interval__block  sample           superblock_data
sqlite> 

TypeError: argument of type 'NoneType' is not iterable

Installed chanjo report as instructed. Tried to generate a report based on the sqlite db generated by running the demo at http://chanjo.readthedocs.org/en/latest/introduction.html . Ran the command:

chanjo report --render pdf --samples "sample2" > ./out.pdf

but was returned the error

[2016-02-10 22:26:46,830] INFO    : chanjo.cli.root          : version 3.2.1
Traceback (most recent call last):
  File "/usr/local/bin/chanjo", line 9, in <module>
    load_entry_point('chanjo==3.2.1', 'console_scripts', 'chanjo')()
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/chanjo_report-2.4.0-py2.7.egg/chanjo_report/cli/core.py", line 45, in report
    click.echo(render_method(api, options=context.obj))
  File "/usr/local/lib/python2.7/dist-packages/chanjo_report-2.4.0-py2.7.egg/chanjo_report/interfaces/pdf/core.py", line 22, in render_pdf
    app = create_app(config=config)
  File "/usr/local/lib/python2.7/dist-packages/chanjo_report-2.4.0-py2.7.egg/chanjo_report/server/app.py", line 18, in create_app
    configure_extensions(app)
  File "/usr/local/lib/python2.7/dist-packages/chanjo_report-2.4.0-py2.7.egg/chanjo_report/server/app.py", line 34, in configure_extensions
    api.init_app(app)
  File "/usr/local/lib/python2.7/dist-packages/chanjo/store/api.py", line 37, in init_app
    self.connect(db_uri=uri)
  File "/usr/local/lib/python2.7/dist-packages/chanjo/store/core.py", line 72, in connect
    if 'mysql' in db_uri:
TypeError: argument of type 'NoneType' is not iterable

Not exactly sure what is wrong, any help much appreciated.

Improve explanations

Go over and see if I can explain/rename metrics to be more intuitive

Add form to customize report

Should include:

• cutoff for completeness
• name of gene panel
• language? (if possible)
• genes for panel
• which completeness levels to include

Entry point not found for report

Upon installing chanjo_report, I get

ImportError: Entry point ('chanjo.subcommands', 'report') not found

As far as I can tell, both chanjo and chanjo-report have been installed correctly.
Removing chanjo-report gets rid of the error, but I actually need the functionality. ;)

Error with cairo

OSError: dlopen() failed to load a library: cairo / cairo-2

Make flask optional

It should be entirely possible to separate out Jinja2+WeasyPrint for PDF generation and Flask as an optional dependency for the server.

Does it really matter though? Probably not unless it significantly complicates the install. Given that bioinformatics already requires so many dependencies, what's one more really? 😉

Error in sqlite table column reading by chanjo report

Following the upgrade suggested in #22 I still am running into problems when I run the command:

chanjo report --render pdf --samples "sample2" > ./out.pdf

This time the error report is:

[2016-03-09 17:17:11,092] INFO    : chanjo.cli.root          : version 3.4.1
--------------------------------------------------------------------------------
DEBUG in api [/usr/local/lib/python2.7/dist-packages/chanjo/store/api.py:45]:
set up new session
--------------------------------------------------------------------------------
[2016-03-09 17:17:11,450] ERROR   : chanjo_report.server.blueprints.report.views: (pysqlite2.dbapi2.OperationalError) no such column: sample.source [SQL: u'SELECT sample.id AS sample_id, sample.group_id AS sample_group_id, sample.source AS sample_source, sample.created_at AS sample_created_at \nFROM sample \nWHERE sample.group_id = ?'] [parameters: (u'coverage',)]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/chanjo_report/server/blueprints/report/views.py", line 45, in map_samples
    samples = {sample_obj.id: sample_obj for sample_obj in query}
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2736, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2751, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (pysqlite2.dbapi2.OperationalError) no such column: sample.source [SQL: u'SELECT sample.id AS sample_id, sample.group_id AS sample_group_id, sample.source AS sample_source, sample.created_at AS sample_created_at \nFROM sample \nWHERE sample.group_id = ?'] [parameters: (u'coverage',)]
Traceback (most recent call last):
  File "/usr/local/bin/chanjo", line 11, in <module>
    sys.exit(root_command())
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/chanjo_report/cli/core.py", line 46, in report
    click.echo(render_method(api, options=context.obj))
  File "/usr/local/lib/python2.7/dist-packages/chanjo_report/interfaces/pdf/core.py", line 26, in render_pdf
    return HTML(url).write_pdf()
  File "/usr/local/lib/python2.7/dist-packages/flask_weasyprint/__init__.py", line 182, in HTML
    return _wrapper(weasyprint.HTML, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flask_weasyprint/__init__.py", line 165, in _wrapper
    return class_(guess, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/weasyprint/__init__.py", line 82, in __init__
    with result as (source_type, source, base_url, protocol_encoding):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/weasyprint/__init__.py", line 290, in _select_source
    with result as result:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/weasyprint/__init__.py", line 298, in _select_source
    with fetch(url_fetcher, url) as result:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/weasyprint/urls.py", line 315, in fetch
    raise URLFetchingError('%s: %s' % (name, value) if value else name)
weasyprint.urls.URLFetchingError: ValueError: Flask-WeasyPrint got HTTP status 500 INTERNAL SERVER ERROR for http://localhost/group/coverage

I am just trying this out on the demo data from chanjo to begin. Many thanks @robinandeer for your help with the previous issue.

Write version with vX.X

Docker + Fig

https://registry.hub.docker.com/u/danishabdullah/phusion-anaconda/dockerfile/

internationalization

Tutorial on Babel with Flask:
http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xiv-i18n-and-l10n

Example project implmentation:
https://github.com/imwilsonxu/fbone/blob/eba498955d59d44de4a01796ad2a4773b310f13e/fbone/app.py

Consolidate tables into fewer to look better in the report

End of report indicator

There should be an indicator for the end of the report <hr> or something

Link to PDF generated report from report "template"

"Please enter path to database" writes to stdout

This makes writing PDFs impossible (bogus data written along with the real data).

Allow gene search by hgnc_symbol and possibly alias

See also scout Scout #499.

Narrower column view for html export

We are having some issues merging coverage and general reports in scout. It would be helpful if chanjo-report could export a narrower report so that columns dont get cut on merger. When doing so, why not add an option to deliver the html for the core body part only to avoid having to redact it later in Scout.

coverage report gives different numbers when page is refreshed

Hi @robinandeer ! Hope everything is fine with you :)
I'm trying to solve this bug in scout (Clinical-Genomics/scout#778) . The problem is after creating a report, if the page is refreshed (or the url is copied to another browser page) the hidden list of gene ids that is passed from scout is not passed any more so the coverage is not done on the genes of the panel.

To fix the problem I was going to save the initial gene list into a session object that I'm going to use to calculate the coverage if the request.args are null or if the method=='GET'.

Do you want me to create a pull request on your repo here or you prefer me to fix everything on the clinical genomics fork and that's it?

Pull import date from database to report

Change from "generated at" to something like "uploaded at"

report "undetermined sex" if y-coverage is too low

this has come up as a false determination before

Picking PDF generator

Latex
No. Barely readable syntax.

Report Lab
No. Way to low level. I don't care about pixel alignment.

xhtml2pdf
Promising. Simple. Works with flask.

WeasyPrint
Very promising. First choice candidate. Has better CSS support than other options. Flask support.

Other points:

• It's perfectly possible to render Jinja2 to HTML without a server.

Date of creation in report

The report should indicate with a date when it was generated

List of non-completely covered genes/transcripts/exons

This should be possible to include in the report.

When doing this it would be nice to think a little extra of the abstractions in the package to make it easily extendable.

I'm thinking about the levels: Interface (PDF, tabular) > component (completeness, gender). This should be possible to extract into multiple layers of abstractions.

New version on pypi

The version of pypi does not work with chanjo 4

How to setup and start up chanjo report server

Hi @robinandeer
I know you are very busy with many work, so I write a bit documentation about this when you have time.

I ran chanjo report command without know how to setup and start the chanjo report server. The command did not produce any report. Another thing, I use sqlite database for chanjo. But I guess I have to setup and use MySQL database in order to use chanjo report. Is this the most important thing that causes chanjo report to fail?

Avoid template naming clashes

Avoid using general names like "index.html" for blueprint templates since they are easily overwritten by other global templates accidentally.