REST API Connections

INFO

Add the following command after the username and password to receive more meaningful HTTP errors:

-w"\n%{http_code}\n"

This command can be added to each call.

Create a Connection

Request Method POST
Description

Adds a connection in Datameer.

URI Syntax
curl -u '<username>:<password>' -X POST -d @<job-payload>.json 'http://<Datameer-serverIP>:<port-number>/rest/connections'
Example Response
{
"configuration-id": 8,
"status": "success"
}
Additional Notes

<job-payload>.json

  • The payload is a JSON representation of all information entered during the creation process using the web interface.
  • An easy method of creating a payload is to use REST to read an import job, data link, or workbook and change the JSON that is returned.

Encryption

  • When creating a connection and the password or SSH key is in plain text within the JSON file, the sensitive data will not be secure while passing from the local host over the network until encrypted by Datameer.
  • Datameer X provides the tool encrypt.sh that can encrypt sensitive information before it is sent via REST to Datameer.
    • The tool is called encrypt.sh and is located in the bin folder within Datameer.

      Syntax:

      encrypt.sh -v <password to encrypt>

      Example Response:

      SECURE:0:hnDNk3ZOWZn9bwDiGp7MNuvyEZPHIinQ
       

  • This encrypted password or SSH key can then be added to the JSON file used in creating a connection to securely transfer the information.

Knowledge Base Article:

How to create a connection via REST API calls

Creating a missing folder

A new folder is created when all connection capabilities and permissions are sufficient.

Read a Connection

Request Method GET
Description

Returns information on a specified connector within Datameer X including version number, class name, file information (uuid, path, description, name), typeID, and properties information.

URI Syntax
curl -u '<username>:<password>' -X GET 'http://<Datameer-serverIP>:<port-number>/rest/connections/<job-configuration-id>'
Example Response
{
  "version": "2.1",
  "className": "datameer.dap.common.entity.DataBaseConnection",
  "file": {
    "uuid": "23905938-5bd0-42da-afff-f39bd4b11ac0",
    "path": "/Data/Connections/MYSQL_QA1.dst",
    "description": "MySQL Connection",
    "name": "MYSQL_QA1"
  },
  "typeId": "das.JdbcDataStoreType",
  "properties": {
    "user_name": [
      "extuser"
    ],
    "password": [
      "L5lhAwwiuHjl/h08r0pVAg==\r\n"
    ],
    "key.connectionUrl": [
      "jdbc:mysql://ec2-23-22-91-33.compute-1.amazonaws.com:3306/world"
    ],
    "dataStoreTemplate": [
      "false"
    ],
    "dataStoreUsage": [
      "IMPORT_EXPORT"
    ]
  },
  "dbType": {
    "name": "MySQL"
  }
}

List All Connections

Request Method GET
Description

Returns a list of all the connections in Datameer. This list displays the description, id number, name, path, and uuid of the connector. 

URI Syntax
curl -u '<username>:<password>' -X GET 'http://<Datameer-serverIP>:<port-number/rest/connections'
Example Response
[
{
    "description": "hidden datastore to upload files: FileUploadDataStore",
    "id": 1,
    "name": "FileUploadDataStore",
    "path": "/.system/FileUploadDataStore.dst",
    "uuid": "708d54ed-6d73-4de7-b332-f26f5246c972"
}
,{
    "description": "A data store for the local filesystem where Datameer X is running.  Note, this works ONLY for Datameer X Personal, Workgroup, and Trial editions.\n",
    "id": 2,
    "name": "Datameer X server filesystem",
    "path": "/Data/Connections/Datameer X server filesystem.dst",
    "uuid": "0eb55632-85e2-404c-85f6-dea1a392dc2c"
}
,{
    "description": "Data files for some examples are hosted in Amazon's cloud (S3). Using this data store will download these files into your system and requires internet access",
    "id": 3,
    "name": "Examples in S3",
    "path": "/Examples/Public Data/Examples in S3.dst",
    "uuid": "af2cb4de-40f5-4441-b82e-9b67136d0d6a"
}
,{
    "description": "Data files for some examples are hosted by Infochimps.com in Amazon's cloud (S3). Using this data store will download these files into your system and requires internet access",
    "id": 4,
    "name": "Examples from Infochimps",
    "path": "/Examples/Public Data/Examples from Infochimps.dst",
    "uuid": "140ac500-2697-484f-a937-1ac2c617bd22"
}
,{
    "description": "",
    "id": 5,
    "name": "diConnection",
    "path": "/Applications/peter/diConnection.dst",
    "uuid": "532e8a9d-87c2-42f1-9af3-af2e06a99034"
}
]



Update a Connection

Request Method PUT
Description

Edits an existing connection in Datameer.

Prerequisites
URI Syntax
curl -u '<username>:<password>' -X PUT -d @<job-payload>.json 'http://<Datameer-serverIP>:<port-number>/rest/connections/<job-configuration-id>'
Example Response
{"status": "success"}
Additional Notes

<job-payload>.json

  • The payload is a JSON representation of all information entered during the creation process using the web interface.
  • An easy method of creating a payload is to use REST to read an import job, data link, or workbook and change the JSON that is returned.

Creating a missing folder

A new folder is created when all connection capabilities and permissions are sufficient.

Delete a Connection

Request Method DELETE
Description

Removes a connection from Datameer.

URI Syntax
curl -u '<username>:<password>' -X DELETE 'http://<Datameer-serverIP>:<port-number>/rest/connections/<job-configuration-id>'
Example Response
{"status": "success"}

Connection Dependencies

Request Method GET
Description

Returns the hierarchy of up- and downstream dependencies for a given connection.

Prerequisites

INFO

This API requires enabling /wiki/spaces/DASSB100/pages/32569527063.

URI Syntax
curl -u '<username>:<password>' -X GET 'http://<server>:<port>/api/connections/<configurationId>/dependencies[?direction=<upstream | downstream>][&level=<number>]'
Example Response
{
 "dependencies": [
 {
 "id": 13,
 "file": {
 "path": "/Data/ImportJobs/TechCrunch_continental_USA",
 "name": "TechCrunch_continental_USA",
 "type": "imp",
 "description": "A listing of company funding records reported by TechCrunch.",
 "uuid": "47c25d9a-6a71-4778-8f88-c42217b39a08"
 }
 },
 {
 "id": 10,
 "file": {
 "path": "/Data/ImportJobs/Sales_Jan_2009",
 "name": "Sales_Jan_2009",
 "type": "imp",
 "description": "Sales transactions during the month of January 2009.",
 "uuid": "158dc464-db46-4da8-af51-cf8be06cadc4"
 }
 }
Additional Notes

Path parameters:

  • configurationId:
    • Data type must be a number
    • Use the configuration ID

Query parameters:

  • Direction:
    • Data type must be a string
    • Optional
    • Shows either upstream or downstream
    • If left blank, shows both
  • Level:
    • Data type must be a number
    • Optional
    • If left blank, shows all levels