Registering Metadata

Isabl Data Model

Isabl models a data generation process where Experiments such as Whole Genome Sequencing are performed on Samples collected from different Individuals. This normalization approach reduces data redundancy and improves data integrity.

Data Generation Process

Unique Together Constraints

Database Diagram

Unique together constraints enable Isabl link new samples and experiments to existing records in the database. The following fields are enforced to be unique together across the entire system:

Database Schema

Unique Together Fields

Individual

• Center

• Species

• Identifier

Samples

• Individual

• Sample Class

• Identifier

Aliquots

• Sample

• Identifier

Experiments

• Sample

• Aliquot ID

• Technique

• Identifier

Applications

• Name

• Version

• Species

• Assembly

The concept of cohorts, where multiple Experiments are grouped and analyzed together, is fundamental and well supported. Furthermore, isabl also tracks and executes Assembly aware Bioinformatics Applications making sure that results are a function of the reference genome. Instances of these applications are also tracked and referred as Analyses.

Metadata Registration

Isabl offer different mechanisms for metadata registration.

Adding Extra Choices

If you need more choices for species, gender, sample category, and technique methods, please refer to the extra choices documentation.

Sync Diseases with Onco Tree

If you are working with Cancer, you can create sync Isabl Diseases with Onco Tree. Simply run:

# you can find more onco tree versions at http://oncotree.mskcc.org
python ./manage.py sync_oncotree --oncotree-version oncotree_2019_03_01

Register Samples with Excel

Through the web interface, is possible to import an Excel Submission to register new samples.

By clicking in Create New Submission button in the top right menu of the user, or by clicking in Add Batch Samples in the top right button of the Project view.

It will open a modal where you can download the latest Submission form by clicking GET FORM. By latest, it means it will be updated with the available custom fields, and available choices added to options like: center, diseases, platforms and techniques.

After the submission is created it can be uploaded through the web interface and a preliminary summary from the metadata submitted will be shown. This Information about the number of models that will be created (i.e. 1 Individual, 2 Samples, 4 Experiments) or errors in the submission form fields (i.e. Error: individual gender FMALE is not a valid choice) guides you in the submission process, before you can commit it.

After committing your Submission, your new available samples should've been created by now, and you can visualize in the Sample Tree the relationship between the new models you just registered.

Register Samples with RESTful API and CLI

Isabl comes with a comprehensive RESTful API reference, where you can learn how to use every available endpoint for each resource of the database. You can access it by browsing to http://<your-isabl-host>/api/v1/

Create endpoints are get or create, they try to retrieve existing objects using either the primary key field or unique together constraints, and if it doesn't find a match creates a new object. If the view supports nested objects, these will also be retrieved or created using the same criteria.

Let's say you want to create a new Individual. According to the API documentation, we need to provide at species, gender, an identifier, and the center associated with the individual. Note that center is a nested object, you may need to create a new one or query an existing one. Let's say you want to get an existing one. This is how you'd do it with isabl_cli:

import isabl_cli as ii

center = ii.get_instance(
    'centers',
    'MEMORIAL SLOAN KETTERING (MSK)',
)
individual = ii.create_instance(
    'individuals',
    identifier = 'EXTERNAL_ID_1',
    species = 'HUMAN',
    gender = 'FEMALE',
    center = center,
)

You can also make http requests directly to the API (you can create a new token from the admin site):

# get token for authentication
curl  -X POST  \
    -d '{ "username": <your-username>, "password": <your-password> }'
    -H "Content-Type: application/json"  \
    http://<your-isabl-host>/api/v1/rest-auth/login/

# create individual
curl \
    -X POST \
    -H 'Authorization: Token <your-token>' \
    -H "Content-Type: application/json" \
    -d '{"identifier": "EXTERNAL_ID_1", "species": "HUMAN", "gender": "FEMALE", "center": {"acronym": "MSK", "name": "MEMORIAL SLOAN KETTERING" } }' \
    http://<your-isabl-host>/api/v1/individuals

Manage User Groups and Permissions

Isabl offers an optional configuration of groups that you can adopt:

In order to create these groups run the following command:

python manage.py create_default_groups