# Registering Metadata
## Isabl Data Model
`Isabl` models a data generation process where *Experiments* such as Whole Genome Sequencing are performed on *Samples* collected from different *Individuals*. This normalization approach reduces data redundancy and improves data integrity.
{% tabs %}
{% tab title="Data Generation Process" %}
{% endtab %}
{% tab title="Unique Together Constraints" %}
Unique together constraints enable Isabl link new samples and experiments to existing records in the database. The following fields are enforced to be unique together across the entire system:
| Database Schema | Unique Together Fields |
| --------------- | ---------------------- |
| **Individual** | |
| -------------- | ----------------------------------------------------------- |
| **Samples** | - Individual
- Sample Class
- Identifier
|
| ------------ | -------------------------------------------------------------------- |
| **Aliquots** | |
| **Experiments** | - Sample
- Aliquot ID
- Technique
- Identifier
|
| ---------------- | -------------------------------------------------------------------------------- |
| **Applications** | - Name
- Version
- Species
- Assembly
|
| {% endtab %} | |
{% tab title="Database Diagram" %}
{% endtab %}
{% endtabs %}
The concept of cohorts, where multiple *Experiments* are grouped and analyzed together, is fundamental and well supported. Furthermore, `isabl` also tracks and executes *Assembly* aware *Bioinformatics Applications* making sure that results are a function of the reference genome. Instances of these applications are also tracked and referred as *Analyses*.
## Metadata Registration
Isabl offer different mechanisms for metadata registration.
{% hint style="warning" %}
Only users with the proper permissions or *superusers* can create or modify models in the database, by using any of the methods for metadata registration.
When using the web interface, available buttons such as **Create New Submission (+)** may be hidden depending of your user role. If you're not seeing this feature, or your getting *permission denied* using the API, please contact your `isabl` administrators.
{% endhint %}
### Adding Extra Choices
If you need more choices for `species`, `gender`, sample `category`, and technique `methods`, please refer to the [extra choices documentation](https://docs.isabl.io/isabl-settings#extra-choices-settings).
### Sync Diseases with Onco Tree
If you are working with Cancer, you can create sync Isabl Diseases with [Onco Tree](http://oncotree.mskcc.org/#/home). Simply run:
```bash
# you can find more onco tree versions at http://oncotree.mskcc.org
python ./manage.py sync_oncotree --oncotree-version oncotree_2019_03_01
```
## Register Samples with Excel
Through the web interface, is possible to import an *Excel Submission* to register new samples.
{% hint style="warning" %}
Note that this feature is limited to create only new *Individuals*, *Samples* and *Experiments*. If you need to create *Centers*, *Diseases*, *Techniques*, *Platforms*, for your available choices you need to use the **Admin** interface at `http:///admin/`or the [API method](#register-samples-with-restful-api-and-cli).
{% endhint %}
By clicking in **Create New Submission** button in the top right menu of the user, or by clicking in **Add Batch Samples** in the top right button of the Project view.
It will open a modal where you can download the latest *Submission* form by clicking **GET FORM.** By latest, it means it will be updated with the available custom fields, and available choices added to options like: center, diseases, platforms and techniques.
{% hint style="info" %}
When prompted to allow *macros*, say yes. This will enable you to toggle between optional and required columns.
{% endhint %}
After the submission is created it can be uploaded through the web interface and a preliminary summary from the metadata submitted will be shown. This Information about the number of models that will be created (i.e. *1 Individual, 2 Samples, 4 Experiments*) or errors in the submission form fields (i.e. *Error: individual gender FMALE is not a valid choice*) guides you in the submission process, before you can commit it.
{% hint style="success" %}
After uploading the submission file, if you don't get any validation errors and your summary looks correct, hit the **Commit Submission** button to register the submission and make definitive changes in the database.
{% endhint %}
After committing your *Submission,* your new available samples should've been created by now, and you can visualize in the *Sample Tree* the relationship between the new models you just registered.
## Register Samples with RESTful API and CLI
`Isabl` comes with a comprehensive RESTful API reference, where you can learn how to use every available endpoint for each resource of the database. You can access it by browsing to `http:///api/v1/`
Create endpoints are *get or create*, they try to retrieve existing objects using either the *primary key* field or *unique together constraints*, and if it doesn't find a match creates a new object. If the view supports nested objects, these will also be retrieved or created using the same criteria.
{% hint style="warning" %}
**IMPORTANT**: If an existing object is retrieved, its fields won't be updated with the posted data.
{% endhint %}
Let's say you want to create a new *Individual.* According to the API documentation, we need to provide at species, gender, an identifier, and the center associated with the individual. Note that `center` is a *nested* object, you may need to create a new one or query an existing one. Let's say you want to get an existing one. This is how you'd do it with `isabl_cli`:
{% code title="Using Isabl SDK" %}
```python
import isabl_cli as ii
center = ii.get_instance(
'centers',
'MEMORIAL SLOAN KETTERING (MSK)',
)
individual = ii.create_instance(
'individuals',
identifier = 'EXTERNAL_ID_1',
species = 'HUMAN',
gender = 'FEMALE',
center = center,
)
```
{% endcode %}
You can also make http requests directly to the API (you can create a new token from the admin site):
{% code title="A request to the API" %}
```bash
# get token for authentication
curl -X POST \
-d '{ "username": , "password": }'
-H "Content-Type: application/json" \
http:///api/v1/rest-auth/login/
# create individual
curl \
-X POST \
-H 'Authorization: Token ' \
-H "Content-Type: application/json" \
-d '{"identifier": "EXTERNAL_ID_1", "species": "HUMAN", "gender": "FEMALE", "center": {"acronym": "MSK", "name": "MEMORIAL SLOAN KETTERING" } }' \
http:///api/v1/individuals
```
{% endcode %}
## Manage User Groups and Permissions
Isabl offers an optional configuration of groups that you can adopt:
| Group | Permissions |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| **Managers** | Can create, update, and delete Custom Fields, Individuals, Centers, Diseases, Experiments, Techniques, Platforms, Projects, and Submissions. |
| **Analysts** | Can create, update, and delete Custom Fields, Applications, Analyses, and Assemblies. They can also download analyses results. |
| **Engineers** | Engineers have the same permissions of both managers and analysts. |
In order to create these groups run the following command:
```bash
python manage.py create_default_groups
```
{% hint style="info" %}
These groups are **optional** and you can create your own using the Django Admin.
{% endhint %}
{% hint style="success" %}
**Pro tip:** use the `Can Download Results` permission to configure what users can download analyses results in your Isabl instance.
{% endhint %}