get_datasource_dataflow_details

DatasourcesApi.get_datasource_dataflow_details(datasource_id, id, role_id, projection_expression=None, action=None, _request_timeout=None, _request_auth=None, _content_type=None, _headers=None, _host_index=0)

Get a single datasource dataflow

Returns details of one dataflow under an existing datasource. The caller (identified by role_id) must have at least read/view permission on the datasource (else AUTH-1012). Supports JDBC Bulkload, JDBC Advanced Load, SaaS, Streams, and ArcGIS. Response fields vary by type (see DatasourceFlowsDetails).

Parameters:

  • datasource_id (str) – UUID of the parent datasource. Rejected with IPV-1045 if the datasource does not exist. Advanced Load / ArcGIS datasources that are disabled by admin return DTSC-1025. (required)

  • id (str) – UUID of the dataflow (DataflowId). Rejected with IPV-1045 if the item does not exist in the datasource-flows table. (required)

  • role_id (str) – Amorphic role ID the request is authorized against. Must be a role the user belongs to with at least read/view access on the datasource. (required)

  • projection_expression (str) – Comma-separated list of top-level attribute names to keep in the response (applied via item_with_projection_attributes after type handlers and MetadataUpdateStatus defaults). If omitted, the full shaped object is returned. Ignored when action=get-presigned-url because that path returns a string, not an object.

  • action (str) – Optional. When omitted, returns full dataflow details. Only honored for JDBC Bulkload: get-presigned-url returns a short-lived S3 PUT URL for the temporary large-payload path. Other datasource types ignore this query value and still return details.

  • _request_timeout (int, tuple(int, int), optional) – timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of (connection, read) timeouts.

  • _request_auth (dict, optional) – set to override the auth_settings for an a single request; this effectively ignores the authentication in the spec for a single request.

  • _content_type (str, Optional) – force content-type for the request.

  • _headers (dict, optional) – set to override the headers for a single request; this effectively ignores the headers in the spec for a single request.

  • _host_index (int, optional) – set to override the host_index for a single request; this effectively ignores the host_index in the spec for a single request.

Return type:

GetDatasourceDataflowDetails200Response

Returns:

Returns the result object.

Request and Response Examples

HTTP: GET /datasources/{datasource_id}/dataflows/{id}

Success (HTTP 200)

Success. Body is either DatasourceFlowsDetails (default) or a JSON string S3 PUT URL when JDBC Bulkload action=get-presigned-url.

Common keys on GET for all types. DataflowType is one of tasks | kafka-connect | flows | consumers | arcgis.

{
  "DataflowId": "b1e6c2a0-9f4d-4c7a-8f2e-1a2b3c4d5e6f",
  "DataflowName": "orders_full_load",
  "DataflowType": "tasks",
  "DatasourceId": "a2f7d3b1-8e5c-4a6f-9d1e-2b3c4d5e6f7a",
  "DataflowStatus": "ready",
  "Message": "Dataflow preparation completed",
  "CreatedBy": "user1",
  "CreationTime": "2024-06-01 12:00:00",
  "LastModifiedBy": "user1",
  "LastModifiedTime": "2024-06-01 12:05:00",
  "MetadataUpdateStatus": "completed",
  "MetadataUpdateMessage": "Dataflow updated successfully"
}

JDBC Bulkload action=get-presigned-url (string body)

"https://amorphic-misc.s3.amazonaws.com/a2f7.../a2f7..._temp.json?X-Amz-Algorithm=..."

Errors

Documented error codes: IPV-1045, AUTH-1012, DTSC-1025, IPV-1002, GE-1048, DB-1002.

HTTP 400

Validation or authorization failure. Body is {“Message”: “<CODE> - <text>”}. Common codes: IPV-1045 (datasource/dataflow not found, or JDBC Bulkload TaskS3Payload file missing), AUTH-1012 (no view access on the datasource), DTSC-1025 (JDBC Advanced Load or ArcGIS is disabled by admin), IPV-1002 (SaaS dataflow not found), GE-1048 (Streams dataflow does not belong to this datasource).

Datasource does not exist

{
  "Message": "IPV-1045 - Invalid parameter - DatasourceId(a2f7d3b1-8e5c-4a6f-9d1e-2b3c4d5e6f7a), value either missing or is invalid."
}

Dataflow does not exist

{
  "Message": "IPV-1045 - Invalid parameter - DataflowId(b1e6c2a0-9f4d-4c7a-8f2e-1a2b3c4d5e6f), value either missing or is invalid."
}

Caller lacks view access on the datasource

{
  "Message": "AUTH-1012 - User: user1 requires at least read access on the resource to perform this action."
}

JDBC Advanced Load or ArcGIS disabled by admin

{
  "Message": "DTSC-1025 - Advanced dataload datasource is currently disabled in the application. Please contact the administrator to enable."
}

Streams dataflow does not belong to datasource

{
  "Message": "GE-1048 - Invalid datasource dataflow association, the requested dataflow - clickstream_consumer, does not belong to the datasource."
}

SaaS dataflow not found

{
  "Message": "IPV-1002 - Invalid DataflowId - b1e6c2a0-9f4d-4c7a-8f2e-1a2b3c4d5e6f, resource not found."
}

JDBC Bulkload TaskS3Payload file missing in S3

{
  "Message": "IPV-1045 - Couldn't fetch the dataflow payload file from S3 to generate PresignedURL! error - ..."
}

HTTP 500

Backend failure. Body is {“Message”: “<CODE> - <text>”}. Example: DB-1002 (failed to update SaaS dataflow status in DynamoDB).

Failed to update SaaS dataflow status

{
  "Message": "DB-1002 - Failed to update entry in the dynamoDB table, please check for errors"
}