ĺ

By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our for more information.
Manage PreferencesAccept All Cookies

NVIDIA TAO Toolkit: How to Build a Data-Centric Pipeline to Improve Model Performance - Part 2 of 3

Jose Gabriel Islas Montero
Jose Gabriel Islas Montero
February 22, 2024
6
 min read
NVIDIA
NVIDIA TAO Toolkit: How to Build a Data-Centric Pipeline to Improve Model Performance - Part 2 of 3
h2
Example H2
h3
Example H3
h4
Example H4
h5
Example H5
h6
Example H6

During this series, we will use ĺ to build a Data-Centric pipeline to debug and fix a model trained with the NVIDIA TAO Toolkit.

‍

. We demystify the NVIDIA ecosystem and define a Data-Centric pipeline based on a model trained with the NVIDIA TAO framework.

‍

Part 2 (current). Using the ĺ API, we show you how to upload a dataset and a model to the ĺ platform.

‍

Part 3. We identify failures in our model due to data issues, fix these failures and improve our model’s performance with the fixed dataset.

‍

Table of Contents

  1. Recap from Part 1
  2. Getting an access token
  3. Uploading our dataset
  4. Uploading our model
  5. Manually uploading a dataset/model
  6. What’s next

‍

1. Recap from Part 1

In the of this series we focused on three main things:

  1. We introduced a blueprint for building a Data-Centric pipeline.
  2. We broke down the NVIDIA TAO Toolkit and several of the moving parts of the NVIDIA ecosystem.
  3. We briefly introduced the .

‍

Before continuing, make sure that you have the following pre-requisites:

  • A trained model on the we introduced in Part 1. You can either adventure yourself with the NVIDIA TAO Toolkit using this or download a trained model .
  • Both the annotations of your dataset and the predictions of your model are expected to be in  — If you opt for (already in COCO format) instead of training one by yourself, you can disregard this step.
  • One provider to store your data with the appropriate configuration (click on the links to configure your data storage provider): , or .
  • A ĺ account.

‍

⚠️ ĺ’ advanced features, including the API, are available for premium users only. However, to make the most out of this series, Section 5 describes how you can upload a dataset/model on the freemium version of ĺ. Please, sign up for a sandbox account .

‍

2. Getting an access token

To obtain an access token, first generate your API keys (see Figure 1).

‍

Figure 1. Find and save your ĺ API keys

‍

The API keys will serve you to retrieve an , as shown on the code below.

‍

💡 Hint: contains examples in other programming languages different from Python (e.g., Node, Go, PHP, Ruby, etc).

‍


import requests

url = "https://dashboard.tenyks.ai/api/auth/apikey"

payload = {
    "api_key": "my_api_key",
    "api_secret": "my_api_secret"
}
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

‍

A successful response is shown below:

‍


{
  "access_token": "eyJraWQ...QH_HAbA",
  "id_token": "eyJraWQiOiJ...daHusA",
  "expires_in": 3600,
  "refresh_token": "eyJjd....gvqtm5",
  "token_type": "Bearer"
}

‍

Save the value of access_token , we’ll use it during the rest of the article.

‍

⚠️ Be aware that the access token has an expiration time of 3,600 seconds or 60 minutes — follow the same procedure to obtain a new one, if required.

‍

💡Hint: Our following code examples might appear verbose to the trained eye, and in fact they are: for this series, we aim to be as explicit as possible!

‍

3. Uploading our dataset to ĺ

⚠️️ For the code examples in Section 3 and 4:

  • We use as data storage provider during this post. For more information, follow this .
  • Replace credentials.value and azure_uri with your own values.
  • Use your access_token in the headers.

‍

3.1 Images & Annotations Ingestion

We assume you have configured your data storage provider as mentioned on Section 1. If you haven’t, detailed instructions of how to do the setup Azure, can be found .

‍

Let’s push our dataset to ĺ! 🚀

  • Uploading images:

# Instructions
# - In "payload", for {value} and {azure_uri} use your own values.
# - Use your {access_token} in the "authorization" key of the headers.
import requests

url = "https://dashboard.tenyks.ai/api/workspaces/tenyks/datasets"

payload = {
    "task_type": "object_detection",
    "images_location": {
        "type": "azure",
        "credentials": {
            "type": "connection_string",
            "value": "DefaultEndpointsProtocol=https;AccountName=tenyksapi;AccountKey=q1UXCLR5JKyG//hcZSztJu+l/cQhpIS2+ASt5FbjSw==;EndpointSuffix=core.windows.net"
        },
        "azure_uri": "https://azure_storage_account.blob.core.windows.net/tenyks-datasets/kitti/images/"
    },
    "key": "tshirts_nvidia_tao", # replace with your own value
    "display_name": "tshirts_nvidia_tao", # replace with your own value
}
headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "authorization": "Bearer eyJraWQiO...JeTh-RN8WPOULqr7NFP8_ETrJ4ECpg"
}

response = requests.post(url, json=payload, headers=headers)

‍

  • Uploading annotations:

# Instructions
# - In "payload", for {value} and {azure_uri} use your own values. 
# - Use your {access_token} in the "authorization" key of the headers.
import requests
my_dataset_key = "road_traffic_nvidia"

url = f"https://dashboard.tenyks.ai/api/workspaces/tenyks/datasets/{my_dataset_key}/images/annotations"

payload = {
    "type": "azure",
    "credentials": {
        "type": "connection_string",
        "value": "DefaultEndpointsProtocol=https;AccountName=tenyksapi;AccountKey=q1UXCLR5JKyG//hcZSztJu+l/cQhpIS2+ASt5FbjSw==;EndpointSuffix=core.windows.net"
    },
    "azure_uri": "https://azure_storage_account.blob.core.windows.net/tenyks-datasets/kitti/annotations.json"
}
headers = {
    "accept": "application/json",
    "content-type": "application/json",
     "authorization": "Bearer eyJraWQiO...JeTh-RN8WPOULqr7NFP8_ETrJ4ECpg"
}

response = requests.put(url, json=payload, headers=headers)

‍

  • Ingesting the images and the annotations into ĺ:

# Instructions
# - Use your {access_token} in the "authorization" key of the headers.
import requests

url = "https://dashboard.tenyks.ai/api/workspaces/tenyks/datasets/tshirts_nvidia_tao/ingest"

headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "authorization": "Bearer eyJraWQiO...JeTh-RN8WPOULqr7NFP8_ETrJ4ECpg"
}

response = requests.put(url, headers=headers)

‍

Voila! You will now see your data on the platform! (see Figure 2).

Figure 2. Images ingested on the ĺ platform

‍

4. Uploading our model to ĺ

4.1 Model Predictions Ingestion

Next, let’s upload our model to ĺ.

  • Creating a model:

# Instructions
# - Use your {access_token} in the "authorization" key of the headers.
import requests
my_dataset_key = "road_traffic_nvidia"

url = f"https://dashboard.tenyks.ai/api/workspaces/tenyks/datasets/{my_dataset_key}/model_inferences"

payload = {
    "iou_threshold": "0.5", # you can change this value
    "confidence_threshold": "0.5", # you can change this value
    "display_name": "yolo_v8", # replace with your own value
    "key": "yolo_v8" # replace with your own value
}
headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "authorization": "Bearer eyJraWQiO...JeTh-RN8WPOULqr7NFP8_ETrJ4ECpg"
}

response = requests.post(url, json=payload, headers=headers)

‍

  • Uploading model predictions:

# Instructions
# - In "payload", for {value} and {azure_uri} use your own values. 
# - Use your {access_token} in the "authorization" key of the headers.
import requests
my_dataset_key = "road_traffic_nvidia"
my_model_key = "yolo_v8"

url = f"https://dashboard.tenyks.ai/api/workspaces/tenyks/datas