University of South Florida - Health Informatics Institute

Data Exchange API Documentation - hdExchange v1.0.0

Purpose
The University of South Florida Health Informatics Institute has engaged in external analytical research partnerships (EAPs) to facilitate analysis of our data resources. This web-enabled API has been created to provide data to these partners in a programmatic fashion.

Authentication
Each External Analytical Partner (EAP) will be provided with a set of authentication credentials (username & password), which may be consistent with those used to access the Data Sharing Platform. These credentials can be used to acquire a unique API token, which can then be used to access all necessary functionality. This token must be included as part of the HTTP token authorization header value. Tokens have a set expiration period; once inactivated, a new token must be requested. All requests will be transferred over HTTPS to ensure that any credentials and sensitive information is protected.

Root URL
https://exchange.hiidata.org/api/

Resources

Resource Details

Accounts - Request New Token

• Description
  Generates and returns a new, active, non-expired token for the requesting application.
• URL
  api/{project}/{subproject}/account/token/new (GET)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

• Results
  An active API token which can be used for authentication on subsequent API requests.

  JSON Output:

  {
    "api_token":"AAAABBBBCCCC111122223333"
  }

  The API token should be included in the http header for all API requests and will need to be base64 encoded (example below is using the bash shell):
  $ printf "AAAABBBBCCCC111122223333" | base64
  Output: DDdEeeFFFggGhhH4HHHIIIjJJkK=
• Additional examples to base64 encode API token using additional programming languages:
  Python2.x:
  Import base64
  un_enc_token = "AAAABBBBCCCC111122223333"
  enc_token = base64.b64encode(un_enc_token)
  print(enc_token)
  
  Python3.x:
  un_enc_token = b"AAAABBBBCCCC111122223333"
  enc_token = base64.b64encode(un_enc_token)
  print(enc_token)
  
  Using r(3.2.1):
  library("base64enc")
  x <- charToRaw("AAAaaaBAAnNa2n4")
  base64encode(x)
• Authentication: Basic - A username and password is required for this call to generate a token (If your project is configured with the hdExplore Data Sharing Platform, then this username and password will be the same as that account).

• Description
  Returns the currently active token for the requesting application. If no current token exists, then a new token is generated.
• URL
  api/{project}/{subproject}/account/token (GET)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

• Results
  An active API token which can be used for authentication on subsequent API requests.

  JSON Output:

  {
    "api_token":"AAAABBBBCCCC111122223333"
  }

• Authentication: Basic - A username and password is required for this call to view a token (If your project is configured with the hdExplore Data Sharing Platform, then this username and password will be the same as that account).

• Description
  Provides a json object representing the specified user's membership in HDX groups and projects.
• URL
  api/account/{username}/membership (GET)
• Alternate URL
  api/{project}/{subproject}/account/membership (GET)
• Parameters
  

  {username}

  The username to use when returning group and project membership

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

• Results
  

  JSON Output:

  {
    "user_name": "sampleuser",
    "projects": [
      {
        "project_name": "Project Name",
        "parent_project_name": "Parent Project Name",
        "project_url_name": "parent_projectname/projectname",
        "groups": [
          {
            "group_name": "Test App Group",
            "group_url_name": "testgroup"
          },
          {
            "group_name": "Sample Data Warehouse",
            "group_url_name": "sampledw"
          }
        ]
      },
      {
        "project_name": "All Participants Project",
        "parent_project_name": "Parent Project",
        "project_url_name": "parent/all,
        "groups": [
          {
            "group_name": Test App Group",
            "group_url_name": "testgroup"
          }
        ]
      }
    ]
  }

• Authentication: Basic - a username and password is required for this call to generate a token. Data Sharing Platform credentials can be used to request a token.

• Description
  Returns basic information about an External Analytical Partner by name.
• URL
  api/{project}/{subproject}/eap/{eapname}/ (GET)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "usf" (For the University of South Florida)

• Results
  Basic information about the External Analytical Partner.

  JSON Output:

  {
    "name":"University of South Florida"
  }

• Authentication: Anonymous

• Description
  Return a list of data available to the EAP
• URL
  api/{project}/{subproject}/eap/{eapname}/access?release={release}&name={name}&type={type} (GET)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "demo_eap" (For the University of South Florida) Returns list of data accessible to the EAP along with each associated mask
  example:https://exchange.hiidata.org/api/project/subproject/eap/eapname/access

  {release} optional

  Optional Data Release filter in the querystring.

  {name} optional

  Optional Name filter in the querystring.

  {type} optional

  Optional Type filter in the querystring. Available options: [data_set, data_files].

• Results
  List of available data elements which can be accessed for analysis

  JSON Output:

  [{
    "name": "INITIAL_INFEC",
    "data_type": "data_set",
    "items":
    [{
      "release": "DEMO",
      "variable_count" 4
    }]
  },{
    "name": "STOOL-WGS",
    "data_type": "data_files",
    "items":
    [{
      "release": "DEMO",
      "file_count": 24
    }]
  }]

• Authentication: Token

• Description
  Request download of a specific data set by name and data release.
• URL
  api/{project}/{subproject}/eap/{eap_name}/datasetrequest/{release_name}/{dataset_name}?output={format} (POST)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "usf" (For the University of South Florida)

  {release_name}

  Name of mask associated with data

  {dataset_name}

  Name associated with the dataset For example: "INITIAL_SNP_DATA"

  {format} optional

  returned format of data For example: "CSV"

• Results

  Data associated with request
  CSV Output:

  subject_id,diagcode,fever,infec_age_months ,R50.9,1,8
  ,J06.9,1,12
  ,R50.9,1,14
  ,R50.9,1,17
  ,J05.0,1,18
  ,J02.9,1,19
  ,R50.9,1,26
  ,R50.9,1,32
  ,K12,0,36
  ...

• Authentication: Token

• Description
  Request transfer of a specific type of data files. Once the file transfer is complete, requested files will be available in the EAP's datacache on the USF high-performance computing cluster.
• URL
  api/{project}/{subproject}/eap/{eapname}/datafilerequest/{release_name}/{dataset_name} (POST)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "usf" (For the University of South Florida)

  {release_name}

  Name of mask associated with data

  {dataset_name}

  Name associated with the dataset For example: "STOOL-WGS"

• Results

  Job Id of Asynchronous Transfer Request
  Output:

  {
    "message":"job_id:44499844-4de5-414b-aa1c-e8420b0bcdef",
    "success": true
  }

• Authentication: Token

• Description
  Return status and information about a data file move job by identifier.
• URL
  api/{project}/{subproject}/eap/{eapname}/datafilerequestcheck/{jobnumber} (GET)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "usf" (For the University of South Florida)

  {jobnumber}

  Job Number generated when the job was created.
  

• Results

  Job Information
  JSON Output:

  {
    "details": {
      "job_id":"0f287c4e-c182-4b87-b045-e77f8b3035bd",
      "job_status":"successful"
    },{
      "message":"",
      "status":"success"
    }
  }

• Authentication: Token

• Description
  Requests for data or material transfers off the HPC. In order to make the data transfer call, a data transfer code must be requested from the DCC by submitting a completed Data Transfer Request form, which is available for download on the Data Sharing Platform. Upon review and approval, a data transfer code will be provided with a sub path to retrieve the files on the Data Sharing Platform if applicable.
• URL
  api/{project}/{sub-project}/eap/{eap_name}/datatransfer (POST)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "usf" (For the University of South Florida)

• Example of the body of a request:

  {
    "release":"MP###",
    "description":"This is a test data transfer request",
    "transfer_code":"44F8497F6B",
    "directory_path":"test/datatype"
  }

  release - The data release name which was used to request the source data.

  description - A clear and concise description of the data or materials being transferred off the HPC.

  transfer_code - The code that was granted to the analytical partner to authorize the data ingest.

  directory_path - The sub-directory path under the external analytical partner's HPC data transfer location. This path will be cleared out once the data transfer is complete.

• Results

  Job ID of Asynchronous Transfer Request

• Description
  Provides Infromation about the external resource being exposed via the hdExchange platform. An external resource could be a dowloaded database, a third-party web api, or a custom script.
• URL
  api/{project}/{sub-project}/external_ref_set/{eap_name}/{external_resource_path} (GET)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "usf" (For the University of South Florida)

  {external_resource_path}

  Url Path to the External Resource. For example: umls/snomed/termsearch

• Results

  External Resource Information
  JSON Output:

  {
    "name": "External Resource Example",
    "description": "Example External Resource for documentation purposes",
    "url_path": "external/resource/example",
    "parameters": [
      {
        "name": "parameter1",
        "data_type": "int",
        "required": true
      },
      {
        "name": "parameter2",
        "data_type": "string",
        "required": false
    }
    ]
  }

• Description
  Provides interactive access to an external resource being exposed via the hdExchange platform. An external resource could be a dowloaded database, a third-party web api, or a custom script.
• URL
  api/{project}/{sub-project}/external_ref_set/{eap_name}/{external_resource_path} (POST)
• Parameters
  

  {project}

  Name of the project. For example: TEDDY, RDN, TrialNet

  {subproject}

  Abbreviation of the specific project within the main study. For example: cc2012, all20120531, SMT

  {eapname}

  Short EAP name. For example: "usf" (For the University of South Florida)

  {external_resource_path}

  Url Path to the External Resource. For example: umls/snomed/termsearch

• Example of the body of a request:
  

  {
    "parameter1":"param-value",
    "parameter2":123,
    "parameter3":["value1","value2","value3"],
  }

• Results

  External Resource Information
  JSON Output:

  *** Varies By Each Resource ***

• Common External Resource Examples

  • Chemical Identifier Resolver (CIR)
      Resource Documentation: https://cactus.nci.nih.gov/chemical/structure_documentation
    
    •   List of Available Representations:
        url: https://exchange.hiidata.org/api/{project}/{sub-project}/external_ref_set/{eap_name}/cir/representations
        Post Body
        empty
        Response
        [
          {
            "id": 1,
            "name": "stdinchi",
            "description": "Returns the Standard InChI of the structure."
          },
          {
            "id": 2,
            "name": "stdinchikey",
            "description": "Returns the Standard InChIKey of the structure."
          },
          ...
        ]
      
    •   Resolve Alternative Chemical Representations:
        url: https://exchange.hiidata.org/api/{project}/{sub-project}/external_ref_set/{eap_name}/cir/resolve?view=[simple, detailed]
        Post Body
        {
          "identifiers":["InChIKey=HVYWMOMLDIMFJA-DPAQBDIFSA-N","InChIKey=BSYNRYMUTXBXSQ-UHFFFAOYSA-N","InChIKey=APDLCSPGWPLYEQ-KRUWYHBYSA-N","InChIKey=LFQSCWFLJHTTHZ-UHFFFAOYSA-N"],
          "representation":"names"
        }
        Response
        [
          {
            "request_string": "InChIKey=HVYWMOMLDIMFJA-DPAQBDIFSA-N",
            "representation": "names",
            "result": [
              "(3beta)-cholest-5-en-3-ol",
              "(3H)-Cholesterol",
              ...
            ]
          },
          {
            "request_string": "InChIKey=BSYNRYMUTXBXSQ-UHFFFAOYSA-N",
            "representation": "names",
            "result": [
              "11126-35-5",
              "2-(Acetyloxy)benzoic acid",
              ...
            ]
          },
          ...
        ]
  • UMLS Term Search
      Resource Documentation: https://www.nlm.nih.gov/research/umls/
    
    •   SNOMED Term Search:
        url: https://exchange.hiidata.org/api/{project}/{sub-project}/external_ref_set/{eap_name}/umls/snomed/termsearch
        Post Body
        {
          "term": "search-term",
          "category": "Clinical finding (finding);Procedure (procedure)",
          "resp_limit": "10"
        }
        Response
        {
          "version": "20160301",
          "results": [
            {
              "concept_id": "1234567",
              "name": "Search Result 1",
              "rank": 100
            },
            {
              "concept_id": "9876543",
              "name": "Search Result 2",
              "rank": 24
            },
            ...
          ]
        }