Skip to content

aametwally/WEVOTE-web

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

What is WEVOTE-web?

Build Status

WEVOTE-web is cloud-based framework of the WEVOTE ensemble taxonomic identification method. The framework to improves the usability of WEVOTE algorithm. In addition, it provides an interactive visual analytics tool to ease the interpretation of the classification results. WEVOTE-web application can also be used by researchers as a repository to store their experimental history for further revisions. A complete setup for the project and its dependencies as a web application is available as an Amazon Machine Image (AMI) for a direct deployment on AWS EC2 machine. The latest AMI is ami-0dab1ab8b4111de4d

Publication

  1. Asem Alaa, Ahmed A. Metwally. "Cloud-based Solution for Improving Usability and Interactivity of Metagenomic Ensemble Taxonomic Identification Methods", IEEE Biomedical and Health Informatics (2018). [online]

  2. Ahmed A. Metwally, Yang Dai, Patricia W. Finn, and David L. Perkins. "WEVOTE: Weighted Voting Taxonomic Identification Method of Microbial Sequences." PloS one 11, no. 9 (2016): e0163527. [online]

Table of Contents

  1. Amazon AMI with a complete setup
  2. WEVOTE Computational Module: building from source
    1. Getting Started
    2. Prerequisites
    3. Installing, Testing, and Running
    4. Alternative method: building and running wevoteREST through Docker-Compose (Ubuntu Xenial 16.04)
      1. Prerequisites
  3. WEVOTE Web Module: building from source
    1. Overview
    2. Building and running the application
      1. Prerequisites
    3. Alternative method: building and running through Docker-Compose (Ubuntu Xenial 16.04)
      1. Prerequisites

Amazon AMI with a complete setup

A complete setup of the project including the five classification methods (i.e BLASTN, KRAKEN, CLARK, MetaPhlAn, TIPP), is available through Amazon Machine Image (AMI), where the associated large databases are downloadable through scripts located in the $HOME directory, i.e /home/ubuntu. The initial size of the image is 10 GB EBS storage, while you may need to reserve 400 GB EBS to account for downloading the associated databases. The memory budget of the instance is subject to the intended methods to use. For example, if the methods are used, but KRAKEN and CLARK, an instance with memory of 1 GiB. Whereas, incorporating KRAKEN and CLARK would require an instance of 80 GiB memory budget.

Usage

  1. Launch an instance with approporiate specifications using the public AMI ami-0dab1ab8b4111de4d.

  2. From the Amazon Web Console, edit the security group attached to your instance by adding new Inbound rule with the following parameters:

Type Protocol Port Range Source
Custom TCP Rule TCP 8080 ::/0, 0.0.0.0/0
  1. Connect to the launched machine with a terminal

  2. From the home directory, download the associated databases subject to your interest using the following scripts:

./addBlastDB.sh
./addCLARKDB.sh
./addKrakenDB.sh
./addWEVOTEDB.sh
  1. Write the following to the instance's terminal
screen
nohup ./WEVOTE_PACKAGE/WEVOTE/bin/wevoteREST -d ~/WEVOTE_PACKAGE/WEVOTE_DB & 
sudo service mongod start  
cd ~/wevote/web && npm run pipeline-amazon  
  1. Now, you can access the web interface from any web-browser on any machine:
http://<the-launched-instance-Public-DNS>:8080
(e.g., `http://ec2-54-157-9-86.compute-1.amazonaws.com:8080`)

WEVOTE Computational Module: building from source

Getting Started

This section details steps for installing and running WEVOTE. Current WEVOTE version only supports Linux. If you experience difficulty installing or running the software, please contact (Ahmed Metwally: [email protected]).

Prerequisites

  • g++.
  • CMake (minimum version 3.5).
  • Qt SDK: for command line argument processing beside other modules are expected to be used extensively through development.
  • cpprest: a restfull API c++ library.
  • OpenMP: for multithreading execution.

To install above dependencies:

sudo add-apt-repository universe
sudo apt-get update
sudo apt-get install build-essential cmake qt5-default libcpprest-dev
  • BLASTN, Kraken, TIPP, CLARK, and MetaPhlan installed on the machine.
  • A machine with RAM of at least 75 GB to run Kraken and Clark. You may ignore this prerequisite if you do not use kraken or clark.

Installing, Testing, and Running

Installing WEVOTE Core applications

Clone the project to your machine

git clone https://github.com/aametwally/WEVOTE-web.git

Building the core library and applications using CMake

Assuming taxonomic binning tools and the corresponding database are installed at ~/WEVOTE_PACKAGE as shown in the table:

Tool Path Database location
BLASTN ~/WEVOTE_PACKAGE/blast ~/WEVOTE_PACKAGE/blastDB/nt (prefix)
CLARK ~/WEVOTE_PACKAGE/clark ~/WEVOTE_PACKAGE/clarkDB (dir)
KRAKEN ~/WEVOTE_PACKAGE/kraken ~/WEVOTE_PACKAGE/krakenDB (dir)
MetaPhlAn ~/WEVOTE_PACKAGE/metaphlan -
TIPP ~/WEVOTE_PACKAGE/tipp -

In addition, the build type, installation prefix, and Qt root directory can be specified in the cmake command. In this build, we the following configuration is used:

Parameter Description Value
CMAKE_BUILD_TYPE The build type (e.g Release or Debug) Release
CMAKE_INSTALL_PREFIX The installation directory /projects/wevote
cd WEVOTE-web
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/projects/wevote \
-DBLASTN_PATH=/home/ubuntu/WEVOTE_PACKAGE/blast \
-DBLASTN_DB=/home/ubuntu/WEVOTE_PACKAGE/blastDB/nt \
-DKRAKEN_PATH=/home/ubuntu/WEVOTE_PACKAGE/kraken \
-DKRAKEN_DB=/home/ubuntu/WEVOTE_PACKAGE/krakenDB \
-DCLARK_PATH=/home/ubuntu/WEVOTE_PACKAGE/clark \
-DCLARK_DB=/home/ubuntu/WEVOTE_PACKAGE/clarkDB \
-DMETAPHLAN_PATH=/home/ubuntu/WEVOTE_PACKAGE/metaphlan ..

After installation three applications are installed at CMAKE_INSTALL_PREFIX/bin:

  • wevotePipeline: the full pipeline app from sequences file.
  • wevoteClassifier: wevote classification app accepts as an input an ensemble file including multiple votes (i.e taxonomic binning) per sequence.
  • abundanceAnnotator: generating the community profile from WEVOTE classification file.
  • wevoteREST: an Http Restful server with exposing the the pipeline with the three different use cases: full pipeline, classification, and community profile.

Running WEVOTE Rest computational server

cd <CMAKE_INSTALL_PREFIX>/bin
./wevoteREST -h
Usage: ./wevoteREST [options]
./wevoteREST help

Options:
  -h, --help                                 Displays this help.
  -H, --host <host>                          host where application is served.
  -P, --port <port>                          The port (i.e socket number)
                                             selected for the application.
  -d, --taxonomy-db-path <taxonomy-db-path>  The path of the taxonomy database
                                             file.
  -v, --verbose <verbose>                    Enable verbose mode.

Example

./wevoteREST -d path/to/taxonomy/dir

Running Wevote Classifier

Change directory to CMAKE_INSTALL_PREFIX/bin

cd /projects/wevote/bin
./wevoteClassifier -h
Usage: ./wevoteClassifier [options]
./wevoteClassifier help

Options:
  -h, --help                                 Displays this help.
  -i, --input-file <input-file>              Input ensemble file produced by
                                             the used algorithms.
  -d, --taxonomy-db-path <taxonomy-db-path>  The path of the taxonomy database
                                             file.
  -p, --output-prefix <output-prefix>        OutputFile Prefix
  -n, --threads <threads>                    Num of threads.
  -k, --penalty <penalty>                    Penalty.
  -a, --min-num-agreed <min-num-agreed>      Minimum number of tools agreed
                                             tools on WEVOTE decision.
  -s, --score <score>                        Score threshold.
  -v, --verbose <verbose>                    Enable verbose mode.

Running Abundance Annotator

cd /projects/wevote/bin
./abundanceAnnotator -h
Usage: ./abundanceAnnotator [options]
./abundanceAnnotator help

Options:
  -h, --help                                 Displays this help.
  -i, --input-file <input-file>              Input file produced by wevote
                                             algorithm.
  -d, --taxonomy-db-path <taxonomy-db-path>  The path of the taxonomy database
                                             file.
  -p, --output-prefix <output-prefix>        OutputFile Prefix

Running Wevote Pipeline

cd /projects/wevote/bin
./wevotePipeline -h
Usage: ./wevotePipeline [options]
./wevotePipeline help

Options:
  -h, --help                                 Displays this help.
  -i, --input-file <input-file>              Input ensemble file produced by
                                             the used algorithms.
  -d, --taxonomy-db-path <taxonomy-db-path>  The path of the taxonomy database
                                             file.
  -p, --output-prefix <output-prefix>        OutputFile Prefix
  -n, --threads <threads>                    Num of threads.
  -k, --penalty <penalty>                    Penalty.
  -a, --min-num-agreed <min-num-agreed>      Minimum number of tools agreed
                                             tools on WEVOTE decision.
  -s, --score <score>                        Score threshold.
  --clark                                    Run CLARK.
  --blastn                                   Run BLASTN
  --tipp                                     Run TIPP.
  --metaphlan                                Run MetaPhlAn.
  --kraken                                   Run KRAKEN
  -v, --verbose <verbose>                    Enable verbose mode.

Alternative method: building and running wevoteREST through Docker Container(Ubuntu Xenial 16.04)

Prerequisites: Docker

Install Docker Community Edition (CE): (follow instructions).

Change directory to web, then build and run wevoteREST as a docker container

cd WEVOTE-web
sudo docker build . -t computational
sudo docker run --rm -it computational

WEVOTE Web Module: building from source

Overview

This module provides the web application that backs the WEVOTE taxonomic classification system, More. The module includes a server, client, visualization applications. All applications are purely implemented in TypeScript. The server application is based on Express.js and mongooselibraries and handles all database operations, user sessions, and communication with the WEVOTE computational server See wevote computational server. While the client and visualization applications (the front-end side) is implemented using the AngularJS framework and d3.js library.

Building and running the application

Prerequisites

OR run the following commands:

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs

Change directory to web, then build wevote-web from scratch

cd WEVOTE-web/web
npm install
npm run build

Running the application

Before running make sure

e.g on linux:

sudo service mongod start
Launching Wevote Server
npm start

Alternative method: building and running through Docker-Compose (Ubuntu Xenial 16.04)

Prerequisites: Docker & Docker-Compose

  1. Installing Docker Community Edition (CE): (follow instructions).
  2. Installing Docker-Compose: (follow instructions).

Change directory to web, then build and run wevote-web as a dockerized application

cd WEVOTE-web/web
sudo docker-compose run --build

About

Cloud package of WEVOTE framework

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published