Coder Social home page Coder Social logo

metabolicatlas / metabolicatlas Goto Github PK

View Code? Open in Web Editor NEW
18.0 10.0 2.0 67.61 MB

Open source genome-scale metabolic models of human and model animals

Home Page: https://metabolicatlas.org

License: GNU General Public License v3.0

Python 0.17% Shell 1.10% JavaScript 38.16% HTML 0.63% Vue 59.49% Dockerfile 0.17% CSS 0.13% SCSS 0.14%
metabolism metabolic-models metabolic-network atlas metabolic-atlas nbisweden

metabolicatlas's Introduction

DOI CII Best Practices Code analysis

About Metabolic Atlas

Metabolic Atlas is a web platform integrating open-source genome scale metabolic models (GEMs) for easy browsing and analysis. The goal is to collect many curated GEMs, and to bring these models closer to FAIR principles. The website provides visualisations and comparisons of the GEMs, and links to resources, algorithms, other databases, and more general software applications. In short, the vision is to create a one-stop-shop for everything metabolism related. Read more about the project on the About page.

Cite us

If you use Metabolic Atlas in your scientific work, please cite:

Version 3.0: Li, F., et al, 2022. GotEnzymes: an extensive database of enzyme parameter predictions. NAR, gkac831 doi:10.1093/nar/gkac831

To cite previous versions or other adjacent work, see the Citation page on the website.

Contributing

If you would like to contribute to the project, for example if you have ideas for new features, discovered a bug or if you would like to improve the code base, please have a look at CONTRIBUTING.md. All ideas and contributions are highly appreciated.

If you discover any potential vulnerabilities associated with the project, please reach out to us at [email protected].

Get started

The front-end uses Vue.js, with help of Vue CLI. The backend uses ExpressJS and Neo4j as the database.

Docker and docker-compose are used to manage the dependencies of this project. Start by installing these if they are not present on the system.

If you want to try out the latest features of MetabolicAtlas, change the branch to main.

Apart from the current repository, two additional repositories are required in order to deploy Metabolic Atlas locally, they are

  • data-generation: for generating neo4j database
  • data-files: contains all the data necessary data (integrated models, maps, FTP repository) using Git LFS

Clone the three required repositories by

git clone https://github.com/MetabolicAtlas/MetabolicAtlas
git clone https://github.com/MetabolicAtlas/data-generation
git clone https://github.com/MetabolicAtlas/data-files && pushd data-files; git lfs pull; popd

Go to the repository data-generation and follow the instructions on how to generate the data files required by Metabolic Atlas.

In the folder MetabolicAtlas that has been cloned, add a env-local.env file based on the env-local.env.sample file:

cp env-local.env.sample env-local.env

and modify this env-local.env file.

Make sure the paths for DATA_FILES_PATH and DATA_GENERATOR_PATH are correct for your setup, eg. the paths to where you have downloaded the repositories data-files and data-generation.

To load the list of helper commands run:

source proj.sh

Run the helper command that creates the database and builds the Docker images of the project:

build-stack

Finally, start the Docker containers with

start-stack

Given successful deployment, the frontend should be accessible at: http://localhost/. If you encounter any problems try looking at the logs logs api / logs frontend.

To deploy the stack to a remote server, create another ENV file, e.g. env-dev.env, and modify it accordingly:

cp env-local.env.sample env-dev.env

Use the tag of the file (dev in the previous example) as an argument to the deploy-stack command to pass the ENV file to the Docker command and the Docker compose file:

deploy-stack dev

GotEnzymes

To reconstruct the database for GotEnzymes on the local (development) machine. Run the following. This should take ~10 minutes.

 ma-exec pg psql -f scripts/init.sql -U postgres

For remote servers, the init script is configured to run automatically if the database has not been initialized. To reconstruct the database, delete the mounted volume for the database on the remote server (located at /var/lib/docker-volumes/pg/postgres-data) and deploy again.

Description of helper commands

  • To bootstrap the project: build-stack

  • To run the project: start-stack

  • To stop the project: stop-stack

  • To run a command inside a docker container: ma-exec <CONTAINER_NAME> <COMMAND>. Below is a list of useful container specific commands.

    • Run API data validation tests: ma-exec api yarn test
    • To run compartment related tests only: ma-exec api npx jest --testNamePattern=compartments
    • Use prettier to format frontend (also works for api) files: ma-exec frontend yarn format
    • Run eslint for frontend (also works for api): ma-exec frontend yarn lint
    • Create production build and stats.html : ma-exec frontend yarn build
  • To clean the project (delete containers and volumes): clean-stack

  • To display real-time logs: logs [container-name: frontend/api/nginx/neo4j/ftp]

  • To deploy the project: deploy-stack

  • To (re-)import the Neo4j database: import-db

A note to Unix/Linux users

When rebuilding the stack, you might have to change the ownership of the directory neo4j:

sudo chown -R <user> neo4j

Replace <user> with your user name. The ownership will be automatically reset when running the project, so you will have to repeat this step for every rebuild.

Licenses

FOSSA Status

metabolicatlas's People

Contributors

dependabot[bot] avatar e0 avatar glormph avatar grokkaine avatar haowang-bioinfo avatar henrikew avatar inghylt avatar jessica-sandler avatar jonandernovella avatar kattislej avatar kusalananda avatar malinahlberg avatar mihai-sysbio avatar nanjiangshu avatar pdmnyberg avatar pecholleyc avatar perjo avatar shan-chalmers avatar virinchi22 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

fossabot e0

metabolicatlas's Issues

Null in title for genes in Interaction Partners

When looking at a gene in the Interaction Partners, the title shows "Interaction Partners for null" if you are using model mouse, fruit fly, zebrafish or rat. Seems to be related to neo4j data as the data returned from MetabolicAtlas/frontend/src/api/interactionPartners.js has null value for component.name when using the models mentioned above.

Sort results alphabetically from API

As a user I would like lists of results, from the API, to be sorted alphabetically, so that I do not have to think about the order of the items in the list.

All api queries, except gotEnzymes routes, are to be sorted (there might be exceptions) Only endpoints which return list are relevant and only the top level list is to be sorted.

Related discussion: #574 (review)

Acceptance criteria

  • if there is an front-end sorting, that should be removed
  • sort by model if applicable
  • sort only top-level structure
  • some requests do not need sorting (such as the graph layout or when already sorting score)
  • create separate issue(s) for the items where sorting would take longer to fix
  • update tests if necessary

Improve asset handling in frontend

We want to avoid external dependencies in the code, so we need to bundle the JS dependencies with the application. We also want to add minification and all those fancy things.

Merge data overlay changes

As a developer, I would like to merge in the changes from #571, so that no important work from the past is lost.

It's a good initiative to try to resolve legacy issues but the PR is not ready to be merged yet.

Related comment: #571 (comment)

broken FTP server

As a user, I want to download all model files easily, so that I can do my research. At the moment, the FTP server is unavailable - logging in just hangs.

Use template styling components in gem browser

As a developer, I would like to have a template for all the components in the gem browser, so that I can avoid repetitive code in the stack and expect similar behavior for the different components

new layout for maps-available card

As I user, it should be straightforward and simple to see which maps are available (2D, 3D).

Proposed layout:

Map-name 2D, 3D

  • remove the functionality related to getAvailableMaps in frontend/src/store/modules/maps.js as it is no longer used
  • the ... to expand is weird because it expands both lists regardless of which is clicked
  • if the list is long then the scrolling happens inside the card, perhaps we can reduce the times this is needed or allow the cards to grow in length by narrowing the reactions table
  • sort maps titles (including part x for Cytosol maps from Human-GEM)
  • from Pierre: the font size is bigger than the rest of the text in the GEM Browser
  • make it clear what's a button and what not (the IP button above is blue, but the Export to TSV button is green)

Consider upgrading Flask

A new version of Flask was just released[0]. I don't think there is a pressing need to upgrade right now, since 0.10 seems to work fine for our purposes and nothing in the new version would make further development for us significantly easier. However, we should keep an eye on Flask development, especially the upcoming 1.0 release, to make sure that HMA will be up to date and compatible.

[0] https://www.palletsprojects.com/blog/flask-011-released/

Add explanation for large number of missing reactions for cytosol maps

As a user, I would like to know that there is a large number of missing reactions due to the compartment being split into several maps, so that I don't think the cytosol maps are inadequate

To ask for feedback: Could be beneficial to have a function that summarize the reactions from each individual map so the user can view the actual number of missing reactions in the compartment.

Information on how to report bugs and vulnerabilities on the web-page

CII Best Practice badge:
The project MUST provide a process for users to submit bug reports (e.g., using an issue tracker or a mailing list). (URL required)
The project MUST publish the process for reporting vulnerabilities on the project site. (URL required)

As a user, I would like to be able to effortless report bugs and/or vulnerabilities, so that I don't have to waste time to find out how or ignore reporting out of laziness.

Note

  1. This issue is mainly for adding user report portal on the frontend, the issue #601 will take are of the bug reporting related to the GitHub.
  2. This issue is also related to the private issue #601

Investigate possibility of improving "Findability on the Web" of pages with multiple entities

As a developer, I would like to investigate if there is a good solution for improving the "Findability on the Web" for pages with multiple entities, so that I can decide if these pages could be adapted and if a new issue should be created.

This is a research task and the goal is to land on a conclusion and possible plan (new issue description).

The pages of interest are:

  • the Explore page, where all the integrated models are listed
  • papers under resources
  • the metabolic atlas citation
  • all other references on the about page

Related issue: #579

Comment about potential solution: #579 (comment)

Verify bioschemas in production environment

As a developer, I would like to verify the bioschemas format in a production environment, so that I can ensure the markup is implemented properly.

This is related to #579. As a verification, it would be good to use a tool (for example: https://validator.schema.org) to make sure that the markup generated is as expected. Currently only the gene pages in gem browser have implementation according to a bioschemas compatible profile. The following page can be used as an example to test: https://metabolicatlas.org/explore/Human-GEM/gem-browser/gene/ENSG00000136010.

Edit: It would also be good to test the title and description changes in link previews (for example sending over Slack), as well as in search engines (for example Google and DuckDuckGo).

logo and page body not aligned

As a user, I would like to see the site logo and the page body aligned to the left, so that I can experience a better look and feel when browsing the website.

Example:

image

This "bug" exists in most pages due to the fact that there is a padding, specifically padding-left, on the logo. It might be easiest to see if the padding-left can be removed to achieve this visual alignment consistently for all pages. If this is the approach, it's important to test on different screen widths.

implement bioschemas.org for existing content

Original issue description

Use bioschemas.org with the help of a library like vue-headful/vue-meta/vue-head for the following:

  • everything in the GEM Browser
  • papers under resources
  • the metabolic atlas citation
  • all other references on the about page

Focus on content that already exists on Met Atlas, and on bioschemas (don’t worry about schema.org). There are a bunch of ways to test the implementation.

Updated issue description that reflects change in scope

As a developer, I would like to improve the schema definition of the HTML document for gem browser component pages, so that users and systems (search engine, link previewer, etc.) receive with more meaningful previews that reflect the contents of the pages.

  • update the title for gem browser component pages
  • add a description for gem browser component pages
  • add a bioschemas compatible schema for gene pages

Review Map Viewer sidebar scrolling

As a user, I would (probably) like to have the 2D, 3D and Gem browser fixed, so that I don't have to scroll all the way to the top of the sidebar if I have chosen a subsystems map with a name at the end of the alphabet

Improve deployment workflow regarding environment variables for different environments

As a developer, I would like to improve the deployment workflow regarding environment variables, so that I can avoid editing the .env file before deploying to a different environment

This is to avoid potential problems with using wrong environment variables when deploying.

For example, different environment files could be used:

  • .env_local -> localhost
  • .env_dev -> dev server
  • .env_prod -> prod server

Consistent usage disabling or removing content when RNA levels via proteinAtlas.org is unavailable

In Map Viewer, if the model is not human we are currently removing the entire div with possibility to use gene expression data from proteinatlas . However, in Interaction Partners we are just disabling it and have the text "Unavailable for [model]".

Map Viewer example, Interaction Partners example

As a user, I would like related content to behave in the same manner, so that I know what to expect when using the web page

Add cache to nginx

Since the response from the backend is fairly static, adding a cache to nginx could improve performance and reduce database load (which currently doesn't really seem to be an issue, but still).

Improve documentation of GitHub repo

CII Best Practice badge: The information on how to contribute SHOULD include the requirements for acceptable contributions (e.g., a reference to any required coding standard). (URL required)

As a developer, I would like to know what problem MetabolicAtlas solves and how to contribute (including a reference to coding standard), so that it is straightforward for me to understand the project and contribute.

Checklist

  • describe in the readme what MetAtlas is about
  • Link to the contribution guideline
  • Update in the readme about how the users should report a bug or vulnerability
  • Update the CII badge

Map Viewer keyboard shortcuts

Jira issue MA-598

On Map Viewer 2D
esc → exit fullscreen ; part of the default browser behavior
arrows → pan
ctrl/cmd + scroll → scroll (noticed in Japan on Safari)

More details from discussion on Teams:

Users are looking at the 2D Map Viewer. They use ctrl + scroll or cmd + scroll. In most websites, this sets the zoom level for the page. However, on Google Maps, this zooms in on the map. I've seen this many times, all of a sudden the page is insanely zoomed, going from proper desktop to tiny mobile, and back, quite fast. So, if on the 2D Map Viewer, the behaviour should be like on Google Maps.

Resolved in: #570

make sure the gene page always shows related reactions

As a user, I would like to be presented with a list of reactions when visiting the gene page, so that I can explore reactions that are associated with this gene.

Bug: For some genes, for example https://metabolicatlas.org/explore/Human-GEM/gem-browser/gene/ENSG00000000419, it shows 0 related reactions. This should not be possible since a gene should always be associated with at least one reaction to be considered as a part of a model. (Thanks to @pecholleyc for pointing this out.)

Fix CodeQL warnings

As a developer, I would like all security warnings to be fixed, so that I can be confident that the code is not vulnerable

Make 2D map coverage comparison exportable

As a user, I would like to be able to export the comparison of reactions of the 2D maps from the popup, so that I don't have to save them manually

The export functionality should look similar to what we have in other places in the website

This issue is a continuation of PR #598

2D map coverage

For the 2D maps, in the Map Viewer sidebar, show the coverage, i.e. how many of that compartment/subsystem reactions are on that map. Normally this would not be needed - the Human-GEM maps have very good coverage. This seems not be the case for the Yeast-GEM maps. Therefore, users need to be informed when a map has low coverage, and the best way to do that is to show the coverage of each map.The coverage should be computed by comparing the contents of the map vs the contents of the model - since the map are hand-drawn, there is a difference between those and the “actual” model content in the database.

It will also be good to inform the user which reactions are missing in the 2D map compared to the database.

Note: the coverage should be limited to reactions and only for compartment and subsystem maps (not custom maps).

  • use a selector to extract the reactions from the svg
  • use (a new?) api endpoint to retrieve all the reaction ids for a compartment or a subsystem
  • compare the two sets of the reactions
  • show the results of the comparison in the frontend
  • have a Show more/Expand button for detailed diff
  • make sure the Documentation explains the cause of difference (the maps have been manually add using a previous instance of the model)
  • use an info icon similar to the Data overlay to link to the relevant section in the Documentation

image

disable data sources for non-Human-GEMs

Proteinatlas.org is a data source that should only be used for Human-GEM. Therefore, it should not be visible for the rest of the models.

At the moment, we have no data sources for the rest of the GEMs, so it’s okay to completely disable this section.

testing

CII Best Practice:

  • The project MUST use at least one automated test suite that is publicly released as FLOSS (this test suite may be maintained as a separate FLOSS project). The project MUST clearly show or document how to run the test suite(s) (e.g., via a continuous integration (CI) script or via documentation in files such as BUILD.md, README.md, or CONTRIBUTING.md)
  • A test suite SHOULD be invocable in a standard way for that language
  • It is SUGGESTED that the test suite cover most (or ideally all) the code branches, input fields, and functionality
  • It is SUGGESTED that the project implement continuous integration (where new or changed code is frequently integrated into a central code repository and automated tests are run on the result)
  • The project MUST have a general policy (formal or not) that as major new functionality is added to the software produced by the project, tests of that functionality should be added to an automated test suite.
  • The project MUST have evidence that the test_policy for adding tests has been adhered to in the most recent major changes to the software produced by the project.
  • It is SUGGESTED that this policy on adding tests (see test_policy) be documented in the instructions for change proposals.

As a developer, I would like to have test validating my code, so that I know that changes I make don't create bugs that are missed during PR reviews

As a user, I would like to have test validating the code, so that I know that the software I use is functional and that I use valid data

Create consistency regarding usage of external and identifier in the frontend

As a user, I would like to have the same "group name" for identifiers belonging to external databases, so that I get a cohesive experience.

Currently, external-db and identifier are used to reference to the same type of identifier, i.e. identifiers referencing to BiGG, HepatoNet1 etc.

It might be worth not using identifier as the group name, as we technically have identifiers in all of the data types.

Mismatch of reactions in model and SVG map

As a developer, I would like all reaction IDs in the SVG map to be found in the model, so that the IDs are coherent.

Verified issue is for the new TSV format Golgi apparatus there were 41 reaction IDs in the SVG map not found in the data-files/integrated-models/Human-GEM/Human-GEM.yml

Verification has not been performed for old TSV format

page for external ids

As a user, I would like to see all the entities on Met Atlas that correspond to an external identifier, so that I can see if it is used in any of the integrated models and link to it directly.

For more context, a guiding example is that identifiers.org/metabolicatlas.externalid:dbname+id would link to metabolicatlas.org/externalid/dbname/id . The setting up of this on identifiers.org is a different card.

Refactor backend app structure

The views module in hma_backend is now over 1000 lines. We should consider using Blueprints instead to make the code more modular. An example Flask application that implements this (as well as other sensible practices) is overholt.

Improve UX for first time users of Interaction Partners

When at https://metabolicatlas.org/ and pressing Interaction Partners the model and id are “hardcoded“ to be Human-GEM and m02007p. That leads to https://metabolicatlas.org/explore/Human-GEM/interaction-partners/m02007p, i.e. a page with no info on how to choose a different metabolite or information about the tool. For first time users, it would be beneficial with a “choose random metabolite” or short info regarding using the search bar + interaction icon and link to the documentation.

Preferably, the “don’t make me think“ mentality should be kept in mind when designing this.

Add lint integration for `frontend`

Locally: As a developer, I would like to have lint warnings fixed automatically every time I save a file, so that I don't have to fix them manually or run docker exec -it frontend yarn lint

In a PR: As a developer, I would like to have lint checks run when pushing to GitHub, so that lint warnings are discovered and fixed asap.

The same lint config should be applied to all repositories. this is reported as different issues

We want this to work cross-platform (macOS, Linux), and cross-editor (vim, VSCode not required anymore) in the team.

As developers, the linter we use should do as much work as possible. The current airbnb configuration is hard to set up with auto-correct. An easier solution would be to use prettier. This should also address the long lines, so that we get closer to following the NBIS dev guidelines.

A lot of changes in the code are expected.

To be linted:

  • .md (might be hard, can be reported as a separate issue)
  • .js
  • .vue
  • .scss
  • python, bash? (might be hard, can be reported as a separate issue)

only one record is shown in the external database table when there are multiple records

For the external database table in the Gem-browser, there are sometimes more than one records in one item, e.g. for the item MetaNetX of the metabolite m00016m, there are two records

m00016m	MetaNetX	MNXM11430	https://identifiers.org/metanetx.chemical:MNXM11430
m00016m	MetaNetX	MNXM162947	https://identifiers.org/metanetx.chemical:MNXM162947

However, on the GEM-browser m00016m only one record, the first one, is shown. This will also affect GEM-browser for genes and reactions in the new TSV format since there are quite a few items contains multiple records.

I'm not sure if it is an expected feature but I guess we can show then or as we did for the Gene rule on the reaction page.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.