update_data_pipeline

DataPipelinesApi.update_data_pipeline(id, role_id, update_data_pipeline_request_body, publish=None, content_type=None, _request_timeout=None, _request_auth=None, _content_type=None, _headers=None, _host_index=0)

Update a Glue or AI Flow data pipeline

Updates by persisted DataPipelineType (body type is not used to switch). Requires editor access. Blocked while any execution is RUNNING or STOPPING (DP-1012) or while PipelineStatus is create_in_progress/update_in_progress (DP-1010). Glue fills omitted fields from the stored row and starts async update (Message “Data pipeline update initiated.”). Cyclic child-pipeline graphs return DP-1037. Query publish is not read. AI requires at least one of Description, Nodes, DefaultExecutionProperties, Graph, Keywords (GE-1010 overwritten, HTTP 500).

Parameters:

  • id (str) – DataPipelineId. (required)

  • role_id (str) – Amorphic role ID the request is authorized against. Must be a role the user belongs to. Lambda returns AUTH-1001 (HTTP 403 on v2-dataPipelines, HTTP 400 on v2-downloadLogs) if missing after gateway validation. (required)

  • update_data_pipeline_request_body (UpdateDataPipelineRequestBody) – (required)

  • publish (str) – Present in the live spec but not read by create_or_update_data_pipeline or AI update.

  • content_type (str)

  • _request_timeout (int, tuple(int, int), optional) – timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of (connection, read) timeouts.

  • _request_auth (dict, optional) – set to override the auth_settings for an a single request; this effectively ignores the authentication in the spec for a single request.

  • _content_type (str, Optional) – force content-type for the request.

  • _headers (dict, optional) – set to override the headers for a single request; this effectively ignores the headers in the spec for a single request.

  • _host_index (int, optional) – set to override the host_index for a single request; this effectively ignores the host_index in the spec for a single request.

Return type:

UpdateDataPipelineResponse

Returns:

Returns the result object.

Request and Response Examples

HTTP: PUT /data-pipelines/{id}

Request

Glue description and concurrency update

{
  "Description": "Enrich orders with customer attributes (v2)",
  "MaxConcurrentRuns": 2
}

AI Flow graph/nodes update

{
  "Description": "Updated AI Flow",
  "Nodes": [
    {
      "ModuleType": "Input",
      "NodeName": "input",
      "Outputs": [
        {
          "Name": "document",
          "Type": "String"
        }
      ]
    },
    {
      "ModuleType": "Output",
      "NodeName": "output",
      "Inputs": [
        {
          "Name": "document",
          "Type": "String"
        }
      ]
    }
  ],
  "Graph": {
    "input": {
      "success": [
        "output"
      ],
      "failure": []
    },
    "output": {
      "success": [],
      "failure": []
    }
  }
}

Success (HTTP 200)

Update initiated. AI may include LambdaUploadUrls.

Glue update started

{
  "Message": "Data pipeline update initiated."
}

AI Flow update started

{
  "Message": "Data pipeline update initiated.",
  "LambdaUploadUrls": {
    "custom_fn": {
      "presignedUrl": "https://s3.amazonaws.com/bucket/lambda.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256",
      "s3Location": "s3://bucket/ai-flows/kb_qa_flow/custom_fn.zip"
    }
  }
}

Errors

Documented error codes: DP-1012, DP-1010, DP-1037, AUTH-1001, GE-1044.

HTTP 400

API Gateway request validation (missing required parameters such as role_id). Lambda InvalidInputException from this function is HTTP 403.

Gateway rejected missing required header

{
  "Message": "Invalid request"
}

HTTP 403

Invalid input or unauthorized.

Execution is RUNNING or STOPPING

{
  "Message": "DP-1012 - Cannot update a data pipeline with execution status - {RUNNING}."
}

Create or update already in progress

{
  "Message": "DP-1010 - Data pipeline is already being modified (status: update_in_progress). Wait until the operation completes, then try again."
}

Child pipeline cycle

{
  "Message": "DP-1037 - Failed to update data pipeline since a cyclic dependency is being created with the following node(s) - ['child_flow']"
}

Missing role_id

{
  "Message": "AUTH-1001 - Missing role information in event header"
}

HTTP 500

Backend failure (GenericFailureException, InconsistentMetadataException, or unexpected GE-1044).

Unhandled exception

{
  "Message": "GE-1044 - Unexpected error - An unexpected error occurred."
}