Bonus tips

🎱 Some extra features to maximize your isabl journey!"

🌈 Colored Logs with Batch Systems

Default: No colored logs

Colored unbuffered logs

When running jobs using batch systems like LSF or Slurm, many programs detect whether their output is being sent to a terminal (TTY) or a file and may disable features like color formatting or progress bars if they detect non-interactive output.

One workaround, using it carefully, is to use the unbuffer utility from the expect package. unbuffer simulates a terminal by running the program in a pseudo-terminal (PTY), tricking the program into thinking it is running interactively in a terminal, so it continues to produce colored output.

Requirements:

• Have unbuffer available. See tips on how to install on an HPC.

• unbuffer: True, in your isabl_clis SUBMIT_CONFIGURATION.

• >=0.3.28 version of @papaemmelab/isabl_web

• 8c8ad75 commit of isabl_cli

Unbuffered output will redirect both stdout/stderr to the samehead_job.logfile, causing the file head_job.err output to be empty from errors of the running application.

By simulating a pseudo-PTY, you can gain desired colored outputs, but some outputs like progress bars print undesired characters that can clutter your logs. Be aware to disable those if possible (i.e. toil ... --disableProgress),or for specific applications add a dummy envTERM=dumb to tell the program there's no effective terminal attached.

➕ Bonus tip: Install unbuffer in an HPC with miniconda.

conda create -n myenv
conda activate myenv
conda install -c conda-forge expect

Having $OPT_DIR where you install your packages and $BIN_DIR where you add your executable binaries, create a runnable script called unbuffer, where you add your miniconda env's unbuffer executable and add its shared libraries:

${BIN_DIR}/unbuffer

#!/bin/bash

OPT_DIR=<your-installation-dir>
export LD_LIBRARY_PATH="${OPT_DIR}/miniconda3/envs/myenv/lib:$LD_LIBRARY_PATH"
${OPT_DIR}/miniconda3/envs/myenv/bin/unbuffer "$@"

Test it works:

$ unbuffer echo "Hello World!" 
Hello World

🧬 Use IGV links to be 1-click away to visualize your mutation calls

The frontend allows the user to visualize alignment files (bam/ cram) through IGV, by using the API's igv endpoints in 2 cases:

• /api/v1/experiments/igv/{system_id}?assembly={assembly}&locus={locus}: To visualize an aligned file stored in theExperiment.bam_files

• /api/v1/analyses/igv/{pk}?result={result}&result_index={result_index}&locus={locus}: to visualize an analysis result whose frontend_type is igv_<result><result-index>. See Frontend Result Types

But you can also use these endpoints widely to dynamically create your own igv links, when doing data analyses or creating reports!

Demo time

This is a cool example where you can generate clickable links in jupyter to visualize fusions events. The idea is to be 1-click away to visualize the reads of a specific sample in the locus of interest to see if a mutation is a real event or not. Taking into account that the param variable locus supports multiple locus for example to visualize the 2 breakpoints of the fusion. Each locus, creates a new igv view in a tab. The locus can be defined by adding multiple ones separated by a space.

Create IGV links in a Jupyter Notebook

Here you have an IGV link for each event. For example, clicking in the GTF2I::GTF2IRD1fusion link, will open an IGV where you can visualize the 2 breakpoints: https://{my-instance}/api/v1/experiments/igv/{system_id}?assembly={assembly}&locus={locus1}%20{locus2}

%20 is the safe url encoding for a space character, needed to append multiple locus.

Visualize events pointing to the desired locus. Example showing aGTF2I::GTF2IRD1fusion in 2 locus at 7:74125440and 7:74015371

This feature is extremely powerful and useful, especially for generating reports with tens or thousands of variants. You can click on any variant to visualize it in IGV!