Bulk Activity Extract
The Bulk Activity Extract set of REST APIs provides a programmatic interface for retrieving large amounts of activity data out of Marketo. 聽For cases that do not require low latency, and must transfer significant volumes of activity data out of Marketo, such as CRM-integration, ETL, data warehousing, and data archiving.
Permissions
The Bulk Activity Extract APIs require that the API user have the 鈥淩ead-Only Activity鈥� or 鈥淩ead-Write Activity鈥� permissions.
Filters
startAt and endAt. startAt accepts a datetime representing the low-watermark, and endAt accepts a datetime representing the high-watermark. The range must be 31 days or fewer. Jobs with this filter type return all accessible records that were created within the date range. Datetimes should be in an ISO-8601 format, without milliseconds.activityTypeIds. The value must be an array of integers, corresponding to the desired activity types. The 鈥淒elete Lead鈥� activity is not supported (use the endpoint instead). Retrieve activity type ids using the the.primaryAttributeValueIds. The value is an array of ids that specify the primary attributes to filter on. A maximum of 50 ids may be specified. The ids are the unique identifier for either a lead field or an asset, and can be retrieved by calling the appropriate REST API endpoint. For example, to filter on a specific Form for the 鈥淔ill Out Form鈥� activity, pass the Form name to the endpoint to retrieve the Form Id. The following is a list of activity types where primary attribute filtering is supported.primaryAttributeValues. The value is an array of names that specify the primary attributes to filter on. A maximum of 50 names may be specified. The names are the unique identifier for either a lead field or an asset, and can be retrieved by calling the appropriate REST API endpoint. For example, to filter on a specific Form for the 鈥淔ill Out Form鈥� activity, pass the Form Id to endpoint to retrieve the Form name. The following is a list of activity types where primary attribute filtering is supported.primaryAttributeValueIds options primaryattributevalueids-options
When using primaryAttributeValueIds, the activityTypeIds filter must be present and only contain activity ids that match the corresponding asset group. For example, if you are filtering on Web Form assets, only the 鈥淔ill Out Form鈥� activity type id is allowed in activityTypeIds.
Example Request Body:
{
"filter": {
"createdAt": {
"startAt": "2021-07-01T23:59:59-00:00",
"endAt": "2021-07-02T23:59:59-00:00"
},
"activityTypeIds": [
2
],
"primaryAttributeValueIds": [
16,102,95,8
]
}
}
primaryAttributeValueIds and primaryAttributeValues cannot be used together.
primaryAttributeValues options primaryattributevalues-options
Note that you must use 鈥�<program>.<asset>鈥� notation to specify the name for the following asset groups: Marketing Program, Static List, Web Form. For example, a form with the name 鈥淢PS Outbound鈥� that resides underneath a program with the name 鈥淕L_OP_ALL_2021鈥� would be specified as 鈥淕L_OP_ALL_2021.MPS Outbound鈥�.
Example Request Body:
{
"filter": {
"createdAt": {
"startAt": "2021-07-01T23:59:59-00:00",
"endAt": "2021-07-02T23:59:59-00:00"
},
"activityTypeIds": [
2
],
"primaryAttributeValues": [
"GL_OP_ALL_2021.MPS Outbound"
]
}
}
When using primaryAttributeValues, the activityTypeIds filter must be present and only contain activity ids that match the corresponding asset group. For example, if you are filtering on Web Form assets, then only the 鈥淔ill Out Form鈥� activity type id is allowed in activityTypeIds. primaryAttributeValues and primaryAttributeValueIds cannot be used together.
Options
createdAt filter must be included in the array. An optional activityTypeIds filter may be included. The filters are applied to the accessible activity set, and the resulting set of activities is returned by the export job.marketoGUIDleadId activityDate activityTypeId campaignId primaryAttributeValueId primaryAttributeValueattributes. This parameter can be used to reduce the number of fields that are returned by specifying a subset from the list above. Example:鈥渇ields鈥�: [鈥渓eadId鈥�, 鈥渁ctivityDate鈥�, 鈥渁ctivityTypeId鈥漖An additional field 鈥渁ctionResult鈥� can be specified to include the activity action (鈥渟ucceeded鈥�, 鈥渟kipped鈥�, or 鈥渇ailed鈥�).Creating a Job
To export records, you first must define the job and the set of records that you want to retrieve.聽 Create the job using the endpoint.聽 When exporting activities there are two primary filters that can be applied: createdAt, which is always required, and activityTypeIds, which is optional. 聽The createdAt filter is used to define a date range in which activities were created, using the startAt and endAt parameters, both of which are datetime fields, and represent the earliest permitted creation date, and the latest permitted creation date respectively. 聽You may also optionally filter on only certain types of activities, using the activityTypeIds filter. 聽This is useful for removing results that are not relevant for your use case.
POST /bulk/v1/activities/export/create.json
{
"format": "CSV",
"filter": {
"createdAt": {
"startAt": "2017-07-01T23:59:59-00:00",
"endAt": "2017-07-31T23:59:59-00:00"
},
"activityTypeIds": [
1,
12,
13
]
}
}
{
"requestId": "e42b#14272d07d78",
"success": true,
"result": [
{
"exportId": "ce45a7a1-f19d-4ce2-882c-a3c795940a7d",
"status": "Created",
"createdAt": "2017-01-21T11:47:30-08:00",
"queuedAt": "2017-01-21T11:48:30-08:00",
"format": "CSV"
}
]
}
The job now has a status of 鈥淐reated,鈥� but it is not yet in the processing queue. 聽To put it in the queue so it can begin processing, call the endpoint using the exportId from the creation status response.
POST /bulk/v1/activities/export/{exportId}/enqueue.json
{
"requestId": "e42b#14272d07d78",
"success": true,
"result": [
{
"exportId": "ce45a7a1-f19d-4ce2-882c-a3c795940a7d",
"status": "Queued",
"createdAt": "2017-01-21T11:47:30-08:00",
"queuedAt": "2017-01-21T11:48:30-08:00",
"format": "CSV"
}
]
}
Now, the status is reporting that the job has been queued. 聽When a worker becomes available for this job, then the status is switched to 鈥淧rocessing鈥� and the job begins aggregating records from Marketo.
Polling Job Status
Job status can only be retrieved for jobs created by the same API user.
Marketo鈥檚 Bulk Activity Extract is an asynchronous endpoint, so the job status must be polled to determine when the job is complete.聽 Poll using the endpoint as follows:
GET /bulk/v1/activities/export/{exportId}/status.json
{
"requestId": "e42b#14272d07d78",
"success": true,
"result": [
{
"exportId": "ce45a7a1-f19d-4ce2-882c-a3c795940a7d",
"status": "Completed",
"createdAt": "2017-01-21T11:47:30-08:00",
"queuedAt": "2017-01-21T11:48:30-08:00",
"startedAt": "2017-01-21T11:51:30-08:00",
"finishedAt": "2017-01-21T12:59:30-08:00",
"format": "CSV",
"numberOfRecords": 15423,
"fileSize": 12342,
"fileChecksum": "sha256:c16514c7e80fcac5ea055dacae9617fc3c29aff5365e3743071313ce0ed2a815"
}
]
}
The status聽field may respond with one of the following values:
- Created
- Queued
- Processing
- Canceled
- Completed
- Failed
Retrieving Your Data
Once the job is complete, retrieve your data using the endpoint.
GET /bulk/v1/activities/export/{exportId}/file.json
The response contains a file formatted in the way that the job was configured.聽The endpoint responds with the contents of the file.
If a requested lead field is empty (contains no data), 迟丑别苍听苍耻濒濒 is placed in the corresponding field in the export file.聽 In the example below, the campaignId field for the returned activity is empty.
marketoGUID,leadId,activityDate,activityTypeId,campaignId,primaryAttributeValueId,primaryAttributeValue,attributes
783957693,5414087,2022-02-13T14:06:20Z,104,8497,1670,MembershipTest1,"{""Reason"":""Changed by Smart Campaign MembershipTestCampaignStepChoice.MembershipTestCampaignStepChoiceSetUp action Change Data Value"",""Program Member ID"":3240303,""Acquired By"":true,""Old Status"":""Not in Program"",""New Status ID"":21,""Success"":false,""New Status"":""On List"",""Old Status ID"":20}"
783958220,5414094,2022-02-13T14:08:50Z,104,17240,3569,SuccessWebCPS,"{""Program Member ID"":3240305,""Acquired By"":false,""Old Status"":""Not in Program"",""New Status ID"":6,""Success"":true,""New Status"":""Attended"",""Old Status ID"":1}"
783958306,5414094,2022-02-13T14:09:16Z,104,17240,3569,SuccessWebCPS,"{""Program Member ID"":3240305,""Acquired By"":false,""Old Status"":""Attended"",""New Status ID"":6,""Success"":false,""New Status"":""Attended"",""Old Status ID"":6}"
783961924,5316669,2022-02-13T14:27:21Z,104,11614,2333,Nurture Automation,"{""Program Member ID"":3240306,""Acquired By"":false,""Old Status"":""Not in Program"",""New Status ID"":27,""Success"":false,""New Status"":""Member"",""Old Status ID"":26}"
To support partial and resumption-friendly retrieval of extracted data, the file endpoint optionally supports the HTTP header Range聽of the type bytes. 聽If the header is not set, the whole of the contents is returned. 聽You can read more about using the Range header with Marketo Bulk Extract.
Canceling a Job
If a job was configured incorrectly, or becomes unnecessary, it can be easily canceled using the endpoint:
POST /bulk/v1/activities/export/{exportId}/cancel.json
{
"requestId": "e42b#14272d07d78",
"success": true,
"result": [
{
"exportId": "ce45a7a1-f19d-4ce2-882c-a3c795940a7d",
"status": "Cancelled",
"createdAt": "2017-01-21T11:47:30-08:00",
"format": "CSV"
}
]
}
This response has a status indicating that the job has been canceled.