list_data_pipelines

DataPipelinesApi.list_data_pipelines(role_id, limit=None, offset=None, sortorder=None, sortby=None, filter_expression=None, projection_expression=None, _request_timeout=None, _request_auth=None, _content_type=None, _headers=None, _host_index=0)

List data pipelines the caller can access

Returns Glue and AI Flow data pipelines the caller can access. Listing is in-memory after a batch get of authorized ids. Pagination uses next_available (yes/no), 1-based offset, default limit 500, maximum 1000 (GE-1028). Default sort is LastModifiedTime descending. Each item includes AccessType and LastRunStatus (null when the pipeline has never run). DataPipelineType defaults to glue on stored rows that omit it. Caller must send a valid role_id.

Parameters:

  • role_id (str) – Amorphic role ID the request is authorized against. Must be a role the user belongs to. Lambda returns AUTH-1001 (HTTP 403 on v2-dataPipelines, HTTP 400 on v2-downloadLogs) if missing after gateway validation. (required)

  • limit (int) – Maximum number of pipelines to return per page. Defaults to 500. Maximum allowed is 1000 (GE-1028 / HTTP 403 if exceeded).

  • offset (int) – 1-based page offset. offset=1 is the first page. Omitted offset is treated as 0 in the handler (first page).

  • sortorder (str) – Sort direction. Defaults to descending. Code does not validate other values.

  • sortby (str) – Field to sort by. Must be a listing key present on the items. Defaults to LastModifiedTime.

  • filter_expression (str) – Comma-separated field:value pairs. DataPipelineName uses wildcard matching; other fields use exact match.

  • projection_expression (str) – Comma-separated listing keys to include on each item. Allowed keys are DataPipelineId, DataPipelineName, Description, CreatedBy, CreationTime, LastModifiedBy, LastModifiedTime, DataPipelineType, Keywords, PipelineStatus, AccessType, LastRunStatus.

  • _request_timeout (int, tuple(int, int), optional) – timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of (connection, read) timeouts.

  • _request_auth (dict, optional) – set to override the auth_settings for an a single request; this effectively ignores the authentication in the spec for a single request.

  • _content_type (str, Optional) – force content-type for the request.

  • _headers (dict, optional) – set to override the headers for a single request; this effectively ignores the headers in the spec for a single request.

  • _host_index (int, optional) – set to override the host_index for a single request; this effectively ignores the host_index in the spec for a single request.

Return type:

ListDataPipelinesResponse

Returns:

Returns the result object.

Request and Response Examples

HTTP: GET /data-pipelines

Parameter examples

Name

In

Example

filterExpression

query

"DataPipelineType:glue,PipelineStatus:create_complete"

projectionExpression

query

"DataPipelineId,DataPipelineName,PipelineStatus,LastRunStatus"

Success (HTTP 200)

Paginated ListDataPipelinesResponse with DataPipelines, count, total_count, and next_available.

Paginated list of accessible pipelines

{
  "DataPipelines": [
    {
      "DataPipelineId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "DataPipelineName": "orders_enrichment_flow",
      "Description": "Enrich orders with customer attributes",
      "DataPipelineType": "glue",
      "PipelineStatus": "create_complete",
      "AccessType": "owner",
      "CreatedBy": "user1",
      "CreationTime": "2026-01-15 10:30:00",
      "LastModifiedBy": "user1",
      "LastModifiedTime": "2026-08-01 14:22:00",
      "Keywords": [
        "orders",
        "etl"
      ],
      "LastRunStatus": "COMPLETED"
    },
    {
      "DataPipelineId": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
      "DataPipelineName": "kb_qa_flow",
      "Description": "AI Flow over a knowledge base",
      "DataPipelineType": "ai",
      "PipelineStatus": "create_complete",
      "AccessType": "editor",
      "CreatedBy": "user2",
      "CreationTime": "2026-03-01 09:00:00",
      "LastModifiedBy": "user2",
      "LastModifiedTime": "2026-07-12 11:00:00",
      "Keywords": [
        "ai"
      ],
      "LastRunStatus": null
    }
  ],
  "count": 2,
  "total_count": 2,
  "next_available": "no"
}

Errors

Documented error codes: AUTH-1001, GE-1046, AUTH-1012, GE-1028, GE-1044.

HTTP 400

API Gateway request validation (missing required parameters such as role_id). Lambda InvalidInputException from this function is HTTP 403.

Gateway rejected missing required header

{
  "Message": "Invalid request"
}

HTTP 403

Invalid input or unauthorized. dataPipelines maps InvalidInputException and UnauthorizedUserException to HTTP 403.

Missing role_id after gateway

{
  "Message": "AUTH-1001 - Missing role information in event header"
}

Unknown pipeline or execution id

{
  "Message": "GE-1046 - Invalid DataPipelineId id received - 00000000-0000-0000-0000-000000000000"
}

Insufficient access on the pipeline

{
  "Message": "AUTH-1012 - User: user1 requires at least read-only access on the resource to perform this action."
}

Listing limit greater than 1000

{
  "Message": "GE-1028 - Out of range for items per page, Limit is 1000"
}

HTTP 500

Backend failure (GenericFailureException, InconsistentMetadataException, or unexpected GE-1044).

Unhandled exception

{
  "Message": "GE-1044 - Unexpected error - An unexpected error occurred."
}