list_datasource_dataflow_executions

DatasourcesApi.list_datasource_dataflow_executions(datasource_id, id, role_id, request_type=None, limit=None, offset=None, sortorder=None, sortby=None, flow_runs_token=None, dataflow_start_time=None, dataflow_stats_marker=None, _request_timeout=None, _request_auth=None, _content_type=None, _headers=None, _host_index=0)

List dataflow executions or table statistics

Returns execution history for a dataflow, or JDBC table statistics when request_type is set. The caller (identified by role_id) must have at least read/view permission on the datasource (else AUTH-1012). Supports JDBC Bulkload, JDBC Advanced Load, SaaS, and ArcGIS. Not supported for Streams (no handler branch). JDBC list history is full-load only; CDC Bulkload can still use request_type stats modes.

Parameters:

• datasource_id (str) – UUID of the parent datasource. Rejected with IPV-1045 if missing. Advanced Load / ArcGIS disabled by admin return DTSC-1025. (required)

• id (str) – UUID of the dataflow (DataflowId). Rejected with IPV-1045 if missing. SaaS also returns IPV-1002 if the flow item is not found. (required)

• role_id (str) – Amorphic role ID the request is authorized against. Must be a role the user belongs to with at least read/view access on the datasource. (required)

• request_type (str) – JDBC Bulkload and Advanced Load only. When present, returns table statistics instead of execution history. getStats pages stats (limit min 20). downloadStats returns a presigned url. getLatestStats (Bulkload only, non-full-load) triggers a background refresh; not supported for Advanced Load or full-load Bulkload.

• limit (str) – Page size. JDBC list pagination: default 100, max 1000 (GE-1034 if over). Applied only when any of limit/offset/sortorder/sortby is set. SaaS AppFlow runs: default 20. getStats: Bulkload defaults to 20 if omitted; Advanced requires limit (min 20).

• offset (str) – 1-based page offset for JDBC Bulkload and Advanced Load execution lists. Converted to zero-based internally (offset-1). Default page start is 0 when omitted. Ignored for SaaS and ArcGIS.

• sortorder (str) – Sort direction for JDBC Bulkload and Advanced Load execution lists when pagination params are used. Default desc. Only desc reverses; any other value sorts ascending.

• sortby (str) – Sort field for JDBC Bulkload and Advanced Load execution lists when pagination params are used. Default StartTime. Must be an attribute present on the run items.

• flow_runs_token (str) – SaaS only. URL-encoded AppFlow nextToken from a prior response FlowRunsToken. Pass to fetch the next page. Last page when next_available is no or FlowRunsToken is absent.

• dataflow_start_time (str) – JDBC Bulkload and Advanced Load only. StartTime of one run; returns that DynamoDB run object at the top level (not wrapped in dataflowExecutions). IPV-1045 if no matching run.

• dataflow_stats_marker (str) – JDBC Bulkload and Advanced Load getStats pagination marker. Live DMS markers are opaque; DynamoDB-backed pages use custom_dataflow_stats_marker–pos=N. More pages when next_available is yes and DataflowStatsMarker is returned.

• _request_timeout (int, tuple(int, int), optional) – timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of (connection, read) timeouts.

• _request_auth (dict, optional) – set to override the auth_settings for an a single request; this effectively ignores the authentication in the spec for a single request.

• _content_type (str, Optional) – force content-type for the request.

• _headers (dict, optional) – set to override the headers for a single request; this effectively ignores the headers in the spec for a single request.

• _host_index (int, optional) – set to override the host_index for a single request; this effectively ignores the host_index in the spec for a single request.

Return type:

DatasourceExecutions

Returns:

Returns the result object.

Request and Response Examples

HTTP: GET /datasources/{datasource_id}/dataflows/{id}/executions

Success (HTTP 200)

Shape depends on type and query params. Default JDBC/ArcGIS list: {dataflowExecutions}. JDBC with limit/offset/sort*: also next_available (yes|no), count, total_count — last page when next_available is no. JDBC dataflow_start_time: bare run object. SaaS: Message, FlowRuns, next_available, optional FlowRunsToken. getStats: DataflowStats, Message, optional DataflowStatsMarker and next_available. downloadStats: {url}. getLatestStats (Bulkload): Message only.

JDBC/ArcGIS list (dataflowExecutions)

{
  "dataflowExecutions": [
    {
      "DataflowId": "b1e6c2a0-9f4d-4c7a-8f2e-1a2b3c4d5e6f",
      "StartTime": "2024-06-01 12:00:00",
      "RunStatus": "completed",
      "Message": "Dataflow completed successfully",
      "LastModifiedBy": "user1"
    }
  ]
}

JDBC list with limit/offset/sort (pagination fields)

{
  "dataflowExecutions": [
    {
      "DataflowId": "b1e6c2a0-9f4d-4c7a-8f2e-1a2b3c4d5e6f",
      "StartTime": "2024-06-01 12:00:00",
      "EndTime": "2024-06-01 12:10:00",
      "RunStatus": "completed",
      "CurrentRetry": 0,
      "Message": "Dataflow completed successfully",
      "LastModifiedBy": "user1"
    }
  ],
  "next_available": "no",
  "count": 1,
  "total_count": 1
}

SaaS AppFlow runs

{
  "Message": "Flow runs retreived successfully",
  "FlowRuns": [
    {
      "executionId": "ex-123",
      "executionStatus": "Successful",
      "startedAt": "2024-06-01T12:00:00",
      "lastUpdatedAt": "2024-06-01T12:05:00"
    }
  ],
  "next_available": "no"
}

JDBC request_type=getStats

{
  "DataflowStats": [],
  "Message": "Dataflow stats retreived successfully",
  "next_available": "yes",
  "DataflowStatsMarker": "custom_dataflow_stats_marker--pos=20"
}

JDBC request_type=downloadStats

{
  "url": "https://amorphic-misc.s3.amazonaws.com/connection-task-stats/.../task.log?X-Amz-Algorithm=..."
}

JDBC Bulkload request_type=getLatestStats

{
  "Message": "Get latest dataflow stats is triggered and is running in background"
}

Errors

Documented error codes: AUTH-1012, IPV-1045, GE-1034, GE-1043, IPV-1002, DTSC-1015.

HTTP 400

Validation or authorization failure. Body is {“Message”: “<CODE> - <text>”}. Common codes: AUTH-1012 (no view access), IPV-1045 (missing datasource/ dataflow; JDBC non-full-load list history; missing run for dataflow_start_time; getLatestStats on full-load; stats download missing), GE-1034 (limit > 1000; invalid/missing request_type or getStats limit; bad stats marker), GE-1043 (getLatestStats when Bulkload status is deleted/ready), IPV-1002 (SaaS flow not found).

Caller lacks view access

{
  "Message": "AUTH-1012 - User: user1 requires at least read-only access on the resource to perform this action."
}

JDBC list history when ProcessType is not full-load

{
  "Message": "IPV-1045 - Run history can only be viewed for full-load type of dataflows"
}

JDBC list limit over 1000

{
  "Message": "GE-1034 - Out of range for items per page, Limit is 1000"
}

Invalid request_type for JDBC stats

{
  "Message": "GE-1034 - Invalid value received for the query param request_type"
}

HTTP 500

Backend failure. Body is {“Message”: “<CODE> - <text>”}. Example: DTSC-1015 (Bulkload unable to read live DMS table statistics). Streams has no handler and fails at runtime.

Bulkload live DMS stats call failed

{
  "Message": "DTSC-1015 - Unable to get the current status of the dataflow"
}