generate_data_pipeline_node_logs

DataPipelinesApi.generate_data_pipeline_node_logs(id, executionid, node_name, action, servicename, role_id, post_data_pipeline_node_logs_request_body, logtype=None, filter_pattern=None, encoding=None, _request_timeout=None, _request_auth=None, _content_type=None, _headers=None, _host_index=0)

View, generate, or download node logs for a time window

v2-downloadLogs. Required query action=view-logs|generate-logs|download-logs and servicename=data-pipelines. Body startTime and endTime are UTC YYYY-MM-DDThh:mm:ss (IPV-1008 / IPV-1054). view-logs returns logPreview and logTruncated (optional nextToken). generate-logs starts Glue retrieval or tells AI callers logs are natively available. download-logs returns Presignedurl when a file matches the window. logtype error is required for some Glue node types (GE-1034). Optional filter_pattern and encoding=base64.

Parameters:

  • id (str) – DataPipelineId. (required)

  • executionid (str) – DataPipelineExecutionId (Glue workflow run id or AI flow execution id). (required)

  • node_name (str) – NodeName of the pipeline node. (required)

  • action (str) – (required)

  • servicename (str) – (required)

  • role_id (str) – Amorphic role ID the request is authorized against. Must be a role the user belongs to. Lambda returns AUTH-1001 (HTTP 403 on v2-dataPipelines, HTTP 400 on v2-downloadLogs) if missing after gateway validation. (required)

  • post_data_pipeline_node_logs_request_body (PostDataPipelineNodeLogsRequestBody) – (required)

  • logtype (str) – Required for Glue nodes that are not etl_job or datasource; typically error.

  • filter_pattern (str) – Optional CloudWatch filter pattern for view-logs.

  • encoding (str) – When base64, filter_pattern is decoded before use.

  • _request_timeout (int, tuple(int, int), optional) – timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of (connection, read) timeouts.

  • _request_auth (dict, optional) – set to override the auth_settings for an a single request; this effectively ignores the authentication in the spec for a single request.

  • _content_type (str, Optional) – force content-type for the request.

  • _headers (dict, optional) – set to override the headers for a single request; this effectively ignores the headers in the spec for a single request.

  • _host_index (int, optional) – set to override the host_index for a single request; this effectively ignores the host_index in the spec for a single request.

Return type:

GenerateDataPipelineNodeLogs200Response

Returns:

Returns the result object.

Request and Response Examples

HTTP: POST /data-pipelines/{id}/executions/{executionid}/nodes/{node_name}/logs

Request

One-hour UTC window

{
  "startTime": "2026-08-01T10:00:00",
  "endTime": "2026-08-01T11:00:00"
}

Success (HTTP 200)

Preview, generate Message, or download Presignedurl depending on action.

CloudWatch preview

{
  "logPreview": "2026-08-01T10:01:00 Starting job enrich_orders",
  "logTruncated": false
}

No events in the window

{
  "logPreview": "No logs available to retrieve for given time window!",
  "logTruncated": false
}

Glue log generation started

{
  "Message": "Log generation started, an alert will be sent send to email after successful log retrieval"
}

AI traces do not need generate-logs

{
  "Message": "Logs/Trace for Data-pipelines of type AI are natively available for viewing/downloading, no need to generate logs/trace."
}

Presigned download (handler field Presignedurl)

{
  "Presignedurl": "https://s3.amazonaws.com/bucket/logs.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256"
}

Generate first

{
  "Message": "Log file is not available to download for the given timeframe, please generate logs first."
}

Errors

Documented error codes: IPV-1052, IPV-1041, IPV-1008, DP-1021, AUTH-1010, LOG-1007.

HTTP 400

Invalid input. downloadLogs maps InvalidInputException and UnauthorizedUserException to HTTP 400.

Missing servicename

{
  "Message": "IPV-1052 - Missing mandatory parameter(s) servicename in queryStringParameters/pathParameters"
}

GET without logs=all and invalid action

{
  "Message": "IPV-1041 - Invalid parameter - action, allowed values/keys are [view-logs, generate-logs, download-logs]"
}

POST missing startTime/endTime

{
  "Message": "IPV-1008 - Invalid request body, missing key(s) - startTime/endTime"
}

Execution has not reached the node

{
  "Message": "DP-1021 - Data pipeline execution exec-1 is in running state and hasn't reached the node etl_orders yet."
}

No permission on the pipeline

{
  "Message": "AUTH-1010 - User user1 is not authorized to perform this operation"
}

HTTP 500

Log retrieval failed (LOG-1007 or other GenericFailureException).

Unhandled downloadLogs error

{
  "Message": "LOG-1007 - Unable to download log files, please try again later."
}