download_datasource_dataflow_execution_logs¶

DatasourcesApi.download_datasource_dataflow_execution_logs(datasource_id, id, execution_id, role_id, servicename, action, connector_type=None, filter_pattern=None, encoding=None, download_datasource_dataflow_execution_logs_request=None, _request_timeout=None, _request_auth=None, _content_type=None, _headers=None, _host_index=0)¶

View, generate, or download JDBC execution logs

Retrieves CloudWatch logs for one JDBC dataflow execution. The caller (identified by role_id) needs datasources.view and at least read-only access on the datasource (else AUTH-1010). Supported only for full-load JDBC Bulkload (servicename=jdbc-dataflow) and JDBC Advanced Load (jdbc-advanced-dataflow). Not supported for cdc / full-load-and-cdc (GE-1034), SaaS, Streams, or ArcGIS. Time window comes from the execution StartTime/EndTime; do not send startTime/endTime in the body.

Parameters:

datasource_id (str) – UUID of the parent datasource. Rejected with IPV-1002 if missing, AUTH-1010 if the user has no access. (required)
id (str) – UUID of the dataflow (DataflowId). Rejected with IPV-1002 if missing or not owned by the datasource. (required)
execution_id (str) – Execution StartTime (DynamoDB sort key), e.g. “2024-06-01 12:00:00”. URL-decoded. Rejected with IPV-1002 if no matching run exists. (required)
role_id (str) – Amorphic role ID. Must grant datasources.view. Missing role_id returns AUTH-1001. (required)
servicename (str) – jdbc-dataflow (Bulkload) or jdbc-advanced-dataflow (Advanced Load). (required)
action (str) – view-logs returns a preview page. generate-logs starts async generation and emails when ready. download-logs returns Presignedurl when LogStatus is available. (required)
connector_type (str) – Required when servicename is jdbc-advanced-dataflow. Selects Kafka Connect sinkConnector or sourceConnector log streams.
filter_pattern (str) – Optional CloudWatch Logs filter pattern for view-logs and generate-logs.
encoding (str) – Set to base64 when filter_pattern is base64-encoded. Otherwise omit.
download_datasource_dataflow_execution_logs_request (DownloadDatasourceDataflowExecutionLogsRequest) – Optional. For view-logs pagination only, pass nextToken from a prior response. startTime and endTime are ignored; the execution window is used.
_request_timeout (int, tuple(int, int), optional) – timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of (connection, read) timeouts.
_request_auth (dict, optional) – set to override the auth_settings for an a single request; this effectively ignores the authentication in the spec for a single request.
_content_type (str, Optional) – force content-type for the request.
_headers (dict, optional) – set to override the headers for a single request; this effectively ignores the headers in the spec for a single request.
_host_index (int, optional) – set to override the host_index for a single request; this effectively ignores the host_index in the spec for a single request.

Return type:

DownloadDatasourceDataflowLogsResponse

Returns:

Returns the result object.

Request and Response Examples¶

HTTP: POST /datasources/{datasource_id}/dataflows/{id}/executions/{execution_id}/logs

Request¶

Optional. For view-logs pagination only, pass nextToken from a prior response. startTime and endTime are ignored; the execution window is used.

The request body is optional.

view-logs next page

{
  "nextToken": "abc123"
}

Success (HTTP 200)¶

view-logs: logPreview, logTruncated, optional nextToken. generate-logs: Message. download-logs: PresignedUrl when ready, or Message if generate-logs has not completed.

action=view-logs

{
  "logPreview": "2024-06-01T12:00:01 INFO Replication starting...",
  "logTruncated": false
}

action=generate-logs

{
  "Message": "Log generation started, an alert will be sent to email after successful log retrieval"
}

action=download-logs when ready

{
  "PresignedUrl": "https://amorphic-logs.s3.amazonaws.com/dms-task-logs/...?X-Amz-Algorithm=..."
}

action=download-logs before generate completes

{
  "Message": "Log file is not available to download, please generate logs first."
}

Errors¶

Documented error codes: AUTH-1001, AUTH-1010, IPV-1052, IPV-1041, IPV-1002, LOG-1004, GE-1034, DB-1002.

HTTP 400

Validation or authorization failure. Body is {“Message”: “<CODE> - <text>”}. Common codes: AUTH-1001 (missing role_id), AUTH-1010 (no datasource access), IPV-1052 (missing servicename or action), IPV-1041 (invalid servicename/action/connector_type), IPV-1002 (missing datasource, dataflow, or execution), LOG-1004 (generate-logs already in progress).

No datasource access

{
  "Message": "AUTH-1010 - User user1 is not authorized to perform this operation"
}

No run for execution_id StartTime

{
  "Message": "IPV-1002 - Invalid ExecutionId - 2024-06-01 12:00:00, resource not found."
}

generate-logs already triggered

{
  "Message": "LOG-1004 - Cannot perform requested operation, log file creation is already in progress."
}

HTTP 500

Backend failure. Body is {“Message”: “<CODE> - <text>”}. Common codes: GE-1034 (cdc / full-load-and-cdc not supported, or log group missing), DB-1002 (failed to set LogStatus on generate-logs).

ProcessType cdc or full-load-and-cdc

{
  "Message": "GE-1034 - Execution-level logs are not supported for cdc dataflows."
}

Log group not available for the dataflow

{
  "Message": "GE-1034 - Logs are not available for this execution because the required log group does not exist."
}