Registering Metadata

🏷 Create Individuals, Samples, and Experiments before importing data.

Isabl Data Model
Isabl models a data generation process where Experiments such as Whole Genome Sequencing are performed on Samples collected from different Individuals. This normalization approach reduces data redundancy and improves data integrity.
Data Generation Process
Unique Together Constraints
Database Diagram
Unique together constraints enable Isabl link new samples and experiments to existing records in the database. The following fields are enforced to be unique together across the entire system:
Database Schema
Unique Together Fields
Individual
Center
Species
Identifier
Samples
Individual
Sample Class
Identifier
Aliquots
Sample
Identifier
Experiments
Sample
Aliquot ID
Technique
Identifier
Applications
Name
Version
Species
Assembly
The concept of cohorts, where multiple Experiments are grouped and analyzed together, is fundamental and well supported. Furthermore, isabl also tracks and executes Assembly aware Bioinformatics Applications making sure that results are a function of the reference genome. Instances of these applications are also tracked and referred as Analyses.
Metadata Registration
Isabl offer different mechanisms for metadata registration.
Only users with the proper permissions or superusers can create or modify models in the database, by using any of the methods for metadata registration.
When using the web interface, available buttons such as Create New Submission (+) may be hidden depending of your user role. If you're not seeing this feature, or your getting permission denied using the API, please contact your isabl administrators.
Adding Extra Choices
If you need more choices for species, gender, sample category, and technique methods, please refer to the extra choices documentation.
Sync Diseases with Onco Tree
If you are working with Cancer, you can create sync Isabl Diseases with Onco Tree. Simply run:
# you can find more onco tree versions at http://oncotree.mskcc.org
python ./manage.py sync_oncotree --oncotree-version oncotree_2019_03_01
Register Samples with Excel
Through the web interface, is possible to import an Excel Submission to register new samples.
Note that this feature is limited to create only new Individuals, Samples and Experiments. If you need to create Centers, Diseases, Techniques, Platforms, for your available choices you need to use the Admin interface at http://<your-isabl-host>/admin/or the API method.
By clicking in Create New Submission button in the top right menu of the user, or by clicking in Add Batch Samples in the top right button of the Project view.
It will open a modal where you can download the latest Submission form by clicking GET FORM. By latest, it means it will be updated with the available custom fields, and available choices added to options like: center, diseases, platforms and techniques.
When prompted to allow macros, say yes. This will enable you to toggle between optional and required columns.
Metadata can be registered using Excel Submission forms.
After the submission is created it can be uploaded through the web interface and a preliminary summary from the metadata submitted will be shown. This Information about the number of models that will be created (i.e. 1 Individual, 2 Samples, 4 Experiments) or errors in the submission form fields (i.e. Error: individual gender FMALE is not a valid choice) guides you in the submission process, before you can commit it.
After uploading the submission file, if you don't get any validation errors and your summary looks correct, hit the Commit Submission button to register the submission and make definitive changes in the database.
After committing your Submission, your new available samples should've been created by now, and you can visualize in the Sample Tree the relationship between the new models you just registered.
Sample Tree of the new registered samples.
Register Samples with RESTful API and CLI
Isabl comes with a comprehensive RESTful API reference, where you can learn how to use every available endpoint for each resource of the database. You can access it by browsing to http://<your-isabl-host>/api/v1/
Swagger API Documentation
Create endpoints are get or create, they try to retrieve existing objects using either the primary key field or unique together constraints, and if it doesn't find a match creates a new object. If the view supports nested objects, these will also be retrieved or created using the same criteria.
IMPORTANT: If an existing object is retrieved, its fields won't be updated with the posted data.
Let's say you want to create a new Individual. According to the API documentation, we need to provide at  species, gender, an identifier, and the center associated with the individual. Note that center is a nested  object, you may need to create a new one or query an existing one. Let's say you want to get an existing one. This is how you'd do it with isabl_cli:
Using Isabl SDK
import isabl_cli as ii
​
center = ii.get_instance(
    'centers',
    'MEMORIAL SLOAN KETTERING (MSK)',
)
individual = ii.create_instance(
    'individuals',
    identifier = 'EXTERNAL_ID_1',
    species = 'HUMAN',
    gender = 'FEMALE',
    center = center,
)
You can also make http requests directly to the API (you can create a new token from the admin site):
A request to the API
# get token for authentication
curl  -X POST  \
    -d '{ "username": <your-username>, "password": <your-password> }'
    -H "Content-Type: application/json"  \
    http://<your-isabl-host>/api/v1/rest-auth/login/
​
# create individual
curl \
    -X POST \
    -H 'Authorization: Token <your-token>' \
    -H "Content-Type: application/json" \
    -d '{"identifier": "EXTERNAL_ID_1", "species": "HUMAN", "gender": "FEMALE", "center": {"acronym": "MSK", "name": "MEMORIAL SLOAN KETTERING" } }' \
    http://<your-isabl-host>/api/v1/individuals
Manage User Groups and Permissions
Isabl offers an optional configuration of groups that you can adopt:
Group
Permissions
Managers
Can create, update, and delete Custom Fields, Individuals, Centers, Diseases, Experiments, Techniques, Platforms, Projects, and Submissions.
Analysts
Can create, update, and delete Custom Fields, Applications, Analyses, and Assemblies. They can also download analyses results.
Engineers
Engineers have the same permissions of both managers and analysts.
 In order to create these groups run the following command:
python manage.py create_default_groups
These groups are optional and you can create your own using the Django Admin.
Pro tip: use the Can Download Results permission to  configure what users can download analyses results in your Isabl instance.

Quick Start

Importing Data

Last modified 6mo ago

Samples	Individual Sample Class Identifier
Aliquots	Sample Identifier

Experiments	Sample Aliquot ID Technique Identifier
Applications	Name Version Species Assembly

Group	Permissions
Managers	Can create, update, and delete Custom Fields, Individuals, Centers, Diseases, Experiments, Techniques, Platforms, Projects, and Submissions.
Analysts	Can create, update, and delete Custom Fields, Applications, Analyses, and Assemblies. They can also download analyses results.
Engineers	Engineers have the same permissions of both managers and analysts.