get_file_upload_url_for_dataset¶

DatasetsApi.get_file_upload_url_for_dataset(role_id, dataset_file_upload_request, content_type=None, _request_timeout=None, _request_auth=None, _content_type=None, _headers=None, _host_index=0)¶

Generate a presigned URL for dataset file upload

Generates an S3 presigned URL for uploading a file to the specified dataset. Dataset requirements: - DatasetType must be internal - RegistrationStatus must be completed - DatasourceType must be api Partition keys: - Required when the dataset has PartitionKeys defined and is not Iceberg, Hudi, or Delta Lake. - Each partition key (except upload_date) must have a value in the request body. For example, if the dataset defines PartitionKeys as [region, year, upload_date], only region and year need to be supplied. - Iceberg, Hudi, and Delta Lake datasets manage partitions internally — do not pass PartitionKeys. - PartitionKeys values are used to build the S3 path. - upload_date is auto-generated from the current epoch timestamp and appended to the path. If upload_date is passed in the request body, it is ignored and overwritten with the system-generated value. Schema uploads: - When the dataset is in pending registration status, this endpoint is used for sample schema file uploads. - In this case, DatasetType and DatasourceType checks are skipped. File name handling: - Special characters like (& $ @ = ; : + , ? / { ^ } % ` ] > [ ~ < # | “ ‘) are replaced with underscores for S3 key compliance.

Parameters:

role_id (str) – The role ID of the user making the request (required)
dataset_file_upload_request (DatasetFileUploadRequest) – (required)
content_type (str)
_request_timeout (int, tuple(int, int), optional) – timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of (connection, read) timeouts.
_request_auth (dict, optional) – set to override the auth_settings for an a single request; this effectively ignores the authentication in the spec for a single request.
_content_type (str, Optional) – force content-type for the request.
_headers (dict, optional) – set to override the headers for a single request; this effectively ignores the headers in the spec for a single request.
_host_index (int, optional) – set to override the host_index for a single request; this effectively ignores the host_index in the spec for a single request.

Return type:

FileUploadPreSignedResponse

Returns:

Returns the result object.

Request and Response Examples¶

HTTP: POST /datasets/file

Request¶

Upload a file to a non-partitioned dataset

{
  "DatasetId": "abc123-def456-ghi789",
  "FileName": "sales_data_2024.csv"
}

Upload a file to a partitioned dataset

{
  "DatasetId": "abc123-def456-ghi789",
  "FileName": "transactions.parquet",
  "PartitionKeys": {
    "region": "us-east-1",
    "year": "2024"
  }
}

Success (HTTP 200)¶

Successfully generated S3 presigned URL

Presigned URL for non-partitioned dataset

{
  "Message": "https://s3.us-east-1.amazonaws.com/lz-bucket/sales/monthly_orders/upload_date=1691234567/user1/csv/testfile.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."
}

Presigned URL for partitioned dataset (region + year)

{
  "Message": "https://s3.us-east-1.amazonaws.com/lz-bucket/sales/monthly_orders/region=us-east-1/year=2024/upload_date=1691234567/user1/csv/testfile.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-Signature=..."
}

Errors¶

Documented error codes: GE-1003, IPV-1041, GE-1002, GE-1034.

HTTP 400

Invalid input - dataset not found, dataset type not internal, registration not completed, non-API datasource, missing partition keys, or insufficient permissions

Dataset does not exist

{
  "Message": "GE-1003 - abc123-def456-ghi789 does not exist."
}

Dataset type is not internal

{
  "Message": "IPV-1041 - Dataset type is not internal"
}

File upload not available for non-API datasources

{
  "Message": "GE-1002 - File upload is not available for non API datasources"
}

Partition keys required but not provided

{
  "Message": "GE-1002 - Dataset is of Partitioned type but partition keys are not passed in the request body"
}

Resource-synced external s3athena dataset

{
  "Message": "GE-1034 - File uploads are not supported for resource-synced external s3athena datasets"
}

HTTP 500

Unexpected internal error during presigned URL generation