Read Change Data Feed

This API returns change data feed (CDF) information, representing row-level changes between versions of a Delta table. It records change data for UPDATE, DELETE, and MERGE operations.

Overview

Change Data Feed provides three metadata columns in addition to table data:

_change_type (String): The type of change - insert, update_preimage, update_postimage, or delete
_commit_version (Long): The table version containing the change
_commit_timestamp (Long): Unix timestamp (milliseconds) when the change was committed

Request

string

required

GET

string

required

{prefix}/shares/{share}/schemas/{schema}/tables/{table}/changes

Headers

string

required

Bearer token for authentication

Authorization: Bearer {token}

Optional capabilities header for advanced features:

delta-sharing-capabilities: responseformat=delta;readerfeatures=deletionvectors

See Delta Sharing Capabilities for details.

URL Parameters

The share name to query. This parameter is case-insensitive.

string

required

The schema name to query. This parameter is case-insensitive.

string

required

The table name to query. This parameter is case-insensitive.

Query Parameters

At least one of startingVersion or startingTimestamp must be provided.

Long

The starting version of the query (inclusive). Either this or startingTimestamp is required.

String

The starting timestamp in ISO8601 format (UTC timezone), e.g., 2022-01-01T00:00:00Z. Will be converted to a version created at or after this timestamp.

Long

The ending version of the query (inclusive). If not provided, uses the latest table version.

String

The ending timestamp in ISO8601 format (UTC timezone). Will be converted to a version created at or before this timestamp.

Boolean

If true, return historical metadata seen in the delta log. Useful for streaming clients to check schema compatibility.Default: false

Response

Headers

string

Content-Type: application/x-ndjson; charset=utf-8

Long

The starting version of files in the response

Response Body

The response is newline-delimited JSON (NDJSON) with the following structure:

object

required

First line: Protocol version information

{
  "protocol": {
    "minReaderVersion": 1
  }
}

object

required

Second line: Table metadata (current and optionally historical if includeHistoricalMetadata=true)

Show Metadata Fields

string

Table UUID

object

File format (e.g., {"provider": "parquet"})

string

JSON-encoded table schema

array

Partition column names

object

Table configuration (should include {"enableChangeDataFeed": "true"})

object

Change data: File additions

Show Add Fields

string

Pre-signed URL for the added file

string

Unique file identifier

object

Partition column values

Long

File size in bytes

string

JSON-encoded file statistics

Long

Commit timestamp in milliseconds

Long

Table version for this change

object

Change data feed files (changes stored in _change_data directory)

Show CDF Fields

string

Pre-signed URL for the CDF file

string

Unique file identifier

object

Partition column values

Long

File size in bytes

Long

Commit timestamp in milliseconds

Long

Table version for this change

object

Change data: File removals

Show Remove Fields

string

Pre-signed URL for the removed file

string

Unique file identifier

object

Partition column values

Long

File size in bytes

Long

Commit timestamp in milliseconds

Long

Table version for this change

Example Request

curl -X GET \
  '{prefix}/shares/share_name/schemas/schema_name/tables/table_name/changes?startingVersion=0&endingVersion=2' \
  -H 'Authorization: Bearer {token}'

import requests
import json

url = "{prefix}/shares/share_name/schemas/schema_name/tables/table_name/changes"
headers = {
    "Authorization": "Bearer {token}"
}
params = {
    "startingVersion": 0,
    "endingVersion": 2
}

response = requests.get(url, headers=headers, params=params)

# Process NDJSON response
for line in response.iter_lines():
    if line:
        change = json.loads(line)
        print(change)

const response = await fetch(
  '{prefix}/shares/share_name/schemas/schema_name/tables/table_name/changes?startingVersion=0&endingVersion=2',
  {
    method: 'GET',
    headers: {
      'Authorization': 'Bearer {token}'
    }
  }
);

const text = await response.text();
const changes = text.split('\n')
  .filter(line => line.trim())
  .map(line => JSON.parse(line));

changes.forEach(change => console.log(change));

Example Response

HTTP/2 200
content-type: application/x-ndjson; charset=utf-8
delta-table-version: 0

{"protocol":{"minReaderVersion":1}}
{"metaData":{"id":"f8d5c169-3d01-4ca3-ad9e-7dc3355aedb2","format":{"provider":"parquet"},"schemaString":"{\"type\":\"struct\",\"fields\":[{\"name\":\"eventTime\",\"type\":\"timestamp\",\"nullable\":true,\"metadata\":{}},{\"name\":\"date\",\"type\":\"date\",\"nullable\":true,\"metadata\":{}}]}","partitionColumns":["date"],"configuration":{"enableChangeDataFeed":"true"}}}
{"add":{"url":"https://<s3-bucket-name>.s3.us-west-2.amazonaws.com/delta-exchange-test/table_cdf/date%3D2021-04-28/part-00000-8b0086f2.parquet?...","id":"8b0086f2-7b27-4935-ac5a-8ed6215a6640","partitionValues":{"date":"2021-04-28"},"size":573,"stats":"{\"numRecords\":1,\"minValues\":{\"eventTime\":\"2021-04-28T23:33:57.955Z\"},\"maxValues\":{\"eventTime\":\"2021-04-28T23:33:57.955Z\"},\"nullCount\":{\"eventTime\":0}}","timestamp":1652140000000,"version":0}}
{"cdf":{"url":"https://<s3-bucket-name>.s3.us-west-2.amazonaws.com/delta-exchange-test/table_cdf/_change_data/date%3D2021-04-28/part-00000-591723a8.parquet?...","id":"591723a8-6a27-4240-a90e-57426f4736d2","partitionValues":{"date":"2021-04-28"},"size":689,"timestamp":1652141000000,"version":1}}
{"remove":{"url":"https://<s3-bucket-name>.s3.us-west-2.amazonaws.com/delta-exchange-test/table_cdf/date%3D2021-04-28/part-00000-8b0086f2.parquet?...","id":"8b0086f2-7b27-4935-ac5a-8ed6215a6640","partitionValues":{"date":"2021-04-28"},"size":573,"timestamp":1652142000000,"version":2}}

Using Timestamps

curl -X GET \
  '{prefix}/shares/share/schemas/schema/tables/table/changes?startingVersion=5&endingVersion=10' \
  -H 'Authorization: Bearer {token}'

curl -X GET \
  '{prefix}/shares/share/schemas/schema/tables/table/changes?startingTimestamp=2022-01-01T00:00:00Z&endingTimestamp=2022-01-31T23:59:59Z' \
  -H 'Authorization: Bearer {token}'

Error Responses

Show 400 - Bad Request

The request is malformed (e.g., missing required start parameter).

{
  "errorCode": "string",
  "message": "string"
}

Show 401 - Unauthorized

The bearer token is missing or incorrect.

{
  "errorCode": "string",
  "message": "string"
}

Show 403 - Forbidden

The request is forbidden from being fulfilled.

{
  "errorCode": "string",
  "message": "string"
}

Show 404 - Not Found

The requested resource does not exist.

{
  "errorCode": "string",
  "message": "string"
}

Show 500 - Internal Server Error

The request is not handled correctly due to a server error.

{
  "errorCode": "string",
  "message": "string"
}

Change Types

The _change_type column in CDF files indicates the type of operation:

insert: New row added to the table
update_preimage: Row values before an update
update_postimage: Row values after an update
delete: Row removed from the table

Change Data Feed must be enabled on the table (enableChangeDataFeed = true in table configuration) for this API to work.

Protocol Specification

REST API Endpoints

Overview

Request

Headers

URL Parameters

Query Parameters

Response

Headers

Response Body

Example Request

Example Response

Using Timestamps

Error Responses

Change Types

​Overview

​Request

​Headers

​URL Parameters

​Query Parameters

​Response

​Headers

​Response Body

​Example Request

​Example Response

​Using Timestamps

​Error Responses

​Change Types

Overview

Request

Headers

URL Parameters

Query Parameters

Response

Headers

Response Body

Example Request

Example Response

Using Timestamps

Error Responses

Change Types