REST Interface

How to configure your automated tests to request test data via REST.

Flow

To request a test dataset two calls are required:

POST /3/requests (single input, inputKind: OBJECT) or POST /3/request-collections (multiple inputs, inputKind: LIST) - to create a request
GET /3/requests/{requestId}?waitUntil={deadline} - to get the request including the result

The first call is synchronous and returns immediately with a request id (for example r_...). The dataset status is "ACCEPTED". The second call is synchronous and returns with the dataset once it is ready or when the deadline is reached. In case the deadline is reached and the dataset is not in its final state, the call can be repeated.

The reason of this setup is to allow requesting all required data at the test suite start and then to pick each dataset just-in-time. This allows to generate test data that would not yet be available at the time of the test suite start while minimizing the time of the test suite execution.

Request and Response

Request body

The request body when doing a POST request for a new request has the following structure:

{
  "environment": "STAGE",
  "supplier": "CRM",
  "item": "customer",
  "input": {
    "country": "DE",
    "language": "de",
    "currency": "EUR"
  },
  "inputKind": "OBJECT"
}

For collection input (POST /3/request-collections), input is a JSON array and inputKind must be LIST:

{
  "environment": "STAGE",
  "supplier": "CRM",
  "item": "customer",
  "input": [
    { "country": "DE", "language": "de", "currency": "EUR" },
    { "country": "FR", "language": "fr", "currency": "EUR" }
  ],
  "inputKind": "LIST"
}

environment - the environment name where the dataset is requested
supplier - the supplier name of the dataset
item - the item name of the dataset
input - input parameters (OBJECT) or list of input parameters (LIST)
inputKind - OBJECT or LIST

When doing a GET request for a request, the query parameter waitUntil is optional and can be used to specify a deadline as epoch milliseconds. If the request is not ready at the time of the call, the call blocks until the deadline is reached. The backend currently enforces a maximum wait of 20 seconds per call, even if a later waitUntil is provided. If needed, poll repeatedly with multiple calls.

Response body

The response body has the following structure:

{
  "requestId": "r_123456",
  "datasetId": "654321",
  "status": "ACCEPTED",
  "progress": "Waiting on validation",
  "environment": "STAGE",
  "supplier": "CRM",
  "item": "customer",
  "input": {
    "country": "DE",
    "language": "de",
    "currency": "EUR"
  },
  "inputKind": "OBJECT",
  "output": null,
  "outputKind": null,
  "userRequestDate": "2021-01-01T00:00:00Z",
  "generationQueuedDate": null,
  "generationFinishedDate": "2021-01-01T00:00:00Z",
  "userAllocationDate": "2021-01-01T00:00:00Z"
}

requestId - the request id to use in GET /3/requests/{requestId}
datasetId - the dataset id associated with the request
status - the request status, initially "ACCEPTED"
progress - optional progress message, relevant for iterative generation
(...)
inputKind - OBJECT or LIST
output - null until fulfilled, then the generated output payload
outputKind - null until fulfilled, then one of FLAT, LIST, LIST_OF_LISTS
userRequestDate - the date when the request was created
generationQueuedDate - when generation entered queue
generationFinishedDate - when generation finished
userAllocationDate - when the dataset was allocated to the requester

outputKind depends on both item output kind and input kind:

Item output FLAT + input OBJECT -> response FLAT
Item output FLAT + input LIST -> response LIST
Item output LIST + input OBJECT -> response LIST
Item output LIST + input LIST -> response LIST_OF_LISTS
Item output LIST_OF_LISTS + input OBJECT -> response LIST_OF_LISTS
Item output LIST_OF_LISTS + input LIST -> not allowed

Authentication

Authentication requires several headers to be passed with each request including a HMAC signature. The HMAC signature is a HMAC-SHA256 signature of the concatenation: timestamp + '|' + nonce. Headers to be passed are:

SX-Account - The name of the organization
SX-User - The id of the service user
SX-Timestamp - The unix epoch time in seconds
SX-Nonce - A random string to not be repeated across requests for some time
SX-Signature - HMAC-SHA256 in hex of timestamp|nonce, signed with the service user API key

For example the headers will look like follows:

# Helper functions
callsixpack() {
  # Arguments: method, resource path, optional JSON body
  local method=$1
  local path=$2
  local body=$3
  local url="$SIXPACK_API_URL$path"
  local organization=$SIXPACK_ORGANIZATION
  local user=$SIXPACK_API_USER
  local api_key=$SIXPACK_API_KEY
  local nonce=$(openssl rand -hex 12)
  local timestamp=$(date +%s)
  local data="\${timestamp}|\${nonce}"
  local signature=$(echo -n "$data" | openssl dgst -sha256 -hmac "$api_key" -hex | cut -d " " -f2)
  curl -X "$method" "$url" \\
    -H "Content-Type: application/json" \\
    -H "SX-Account: $organization" \\
    -H "SX-User: $user" \\
    -H "SX-Timestamp: $timestamp" \\
    -H "SX-Nonce: $nonce" \\
    -H "SX-Signature: $signature" \\
    -d "$body"
}

sixpack-request() {
  # Arguments: environment, supplier, item, input JSON
  local environment=$1
  local supplier=$2
  local item=$3
  local input=$4

  local req_path="/3/requests"
  local input_kind="OBJECT"
  if [[ "$input" == \[* ]]; then
    req_path="/3/request-collections"
    input_kind="LIST"
  fi

  if [ -z "$input" ]; then
    input="{}"
  fi
  local data="{\\"environment\\":\\"$environment\\",\\"supplier\\":\\"$supplier\\",\\"item\\":\\"$item\\",\\"input\\":$input,\\"inputKind\\":\\"$input_kind\\"}"
  callsixpack "POST" "\${req_path}" "$data"
}

sixpack-refresh() {
  local id=$1
  local deadline=$(( $(date +%s%3N) + 10000 ))
  local req_path="/3/requests/\${id}?waitUntil=\${deadline}"
  callsixpack "GET" "\${req_path}"
}

# Definitions to run before (in your terminal)
export SIXPACK_API_URL= ...# "https://api.sixpack.dev" unless using self-hosted
export SIXPACK_ORGANIZATION= ... 
export SIXPACK_API_USER= ...
export SIXPACK_API_KEY= ... 

# How to call it
request=$(sixpack-request "ENVIRONMENT" "SUPPLIER" "ITEM" "{
		\"input\": \"123\"
	}")
echo $request

request_id=$(echo $request | jq -r '.request_id')
request=$(sixpack-refresh "$request_id")
echo $request

Flow​

Request and Response​

Request body​

Response body​

Authentication​

Flow

Request and Response

Request body

Response body

Authentication