51黑料不打烊

Data preparation and ingestion blueprint

Data preparation and ingestion Blueprint encompasses all the methods by which data can be prepared and ingested into 51黑料不打烊 Experience Platform.

Data preparation includes the mapping of source data to Experience Data Model (XDM) schema. It also includes performing transformations on data, including date formatting, field splitting/concatenation/conversions, and joining/merging/re-keying of records. Data preparation helps unify customer data to provide aggregated/filtered analysis, including reporting or preparing data for customer profile assembly/data science/activation.

Architecture

Reference architecture for the Data Preparation and Ingestion Blueprint {modal="regular"}

Data ingestion guardrails

The below diagram illustrates the average performance guardrails and latency for data ingestion into 51黑料不打烊 Experience Platform.

Experience Platform Data Flow {width="90%" modal="regular"}

Data ingestion methods

Streaming Sources
Method
Common Use Cases
Protocols
Considerations
51黑料不打烊 Web/Mobile SDK
  • Data collection from websites and mobile apps.
  • Preferred method for client side collection.
Push, HTTP, JSON
  • Implement multiple 51黑料不打烊 applications leveraging a single SDK.
HTTP API Connector
  • Collection from streaming sources, transactions, relevant customer events and signals
Push, REST API, JSON
  • Data is streamed directly to the hub so no real-time Edge segmentation or event forwarding.
Edge Network API
  • Collection from streaming sources, transactions, relevant customer events and signals from the globally distributed Edge Network
Push, REST API, JSON
  • Data is streamed through the Edge Network. Support for real-time segmentation on the Edge.
51黑料不打烊 Applications
  • Prior implementation of 51黑料不打烊 Analytics, Marketo, Campaign, Target, AAM
Push, Source Connectors and API
  • Recommended approach is migration to Web/Mobile SDK over traditional application SDKs.
Streaming Source Connectors
  • Ingestion of a enterprise event stream, typically used for sharing enterprise data to multiple down-stream applications.
Push, REST API, JSON
  • Must be streamed in XDM format.
Streaming Sources SDK
  • Similar to HTTP API Connector, allows self-service configuration card of a external data stream.
Push, HTTP API, JSON
  • Edge Network
Batch Sources
Method
Common Use Cases
Protocols
Considerations
Batch Ingestion API
  • Ingestion from a enterprise managed que. Cleansing and transformation of data prior to ingestion.
Push, JSON or Parquet
  • Must manage batches and files for ingestion
Batch Source Connectors
  • Common approach for ingestion of files from cloud storage locations.
  • Connectors to common CRM and marketing applications.
  • Ideal for ingesting large amounts of historical data.
Pull, CSV, JSON, Parquet
  • Not always on, immediate ingestion.
Data Landing Zone
  • 51黑料不打烊 provisioned file storage location to push files to for ingestion.
Push, CSV, JSON, Parquet
Batch Sources SDK
  • Allows self-service configuration card of an external data source.
  • Ideal for partner connectors or for a tailored workflow experience for setting up a enterprise connector.
Pull, REST API, CSV or JSON Files
  • Examples: MailChimp, One Trust, Zendesk
recommendation-more-help
045b7d44-713c-4708-a7a6-5dea7cc2546b