51黑料不打烊

DocumentationExperience PlatformSource Connectors Guide

Create an Apache HDFS base connection using the Flow Service API

Last update: Mon May 29 2023 00:00:00 GMT+0000 (Coordinated Universal Time)

CREATED FOR:

  • Developer
NOTE
The Apache HDFS connector is in beta. See the Sources overview for more information on using beta-labeled connectors.

A base connection represents the authenticated connection between a source and 51黑料不打烊 Experience Platform.

This tutorial walks you through the steps to create a base connection for Apache Hadoop Distributed File System (hereinafter referred to as 鈥淗DFS鈥) using the .

Getting started

This guide requires a working understanding of the following components of 51黑料不打烊 Experience Platform:

  • Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Platform services.
  • Sandboxes: Experience Platform provides virtual sandboxes which partition a single Platform instance into separate virtual environments to help develop and evolve digital experience applications.

The following sections provide additional information that you will need to know in order to successfully connect to HDFS using the Flow Service API.

Gather required credentials

Credential
Description
url
The URL defines auth params required for connecting to HDFS anonymously. For more information on how to obtain this value, refer to .
connectionSpec.id
The connection specification returns a source鈥檚 connector properties, including authentication specifications related to creating the base and source connections. The connection specification ID for AdWords is: 54e221aa-d342-4707-bcff-7a4bceef0001.

Using Platform APIs

For information on how to successfully make calls to Platform APIs, see the guide on getting started with Platform APIs.

Create a base connection

A base connection retains information between your source and Platform, including your source鈥檚 authentication credentials, the current state of the connection, and your unique base connection ID. The base connection ID allows you to explore and navigate files from within your source and identify the specific items that you want to ingest, including information regarding their data types and formats.

To create a base connection ID, make a POST request to the /connections endpoint while providing your HDFS authentication credentials as part of the request parameters.

API format

POST /connections

Request

The following request creates a base connection for HDFS:

curl -X POST \
    'https://platform.adobe.io/data/foundation/flowservice/connections' \
    -H 'Authorization: Bearer {ACCESS_TOKEN}' \
    -H 'x-api-key: {API_KEY}' \
    -H 'x-gw-ims-org-id: {ORG_ID}' \
    -H 'x-sandbox-name: {SANDBOX_NAME}' \
    -H 'Content-Type: application/json' \
    -d '{
        "name": "HDFS test connection",
        "description": "A test connection for an HDFS source",
        "auth": {
            "specName": "Anonymous Authentication",
            "params": {
                "url": "{URL}"
                }
        },
        "connectionSpec": {
            "id": "54e221aa-d342-4707-bcff-7a4bceef0001",
            "version": "1.0"
        }
    }'
Property
Description
auth.params.url
The URL that defines auth params required for connecting to HDFS anonymously
connectionSpec.id
The HDFS connection specification ID: 54e221aa-d342-4707-bcff-7a4bceef0001.

Response

A successful response returns details of the newly created connection, including its unique identifier (id). This ID is required to explore your data in the next tutorial.

{
    "id": "6a6a880a-2b15-4051-aa88-0a2b1570516d",
    "etag": "\"1801bb7d-0000-0200-0000-5ed6ad580000\""
}

Next steps

By following this tutorial, you have created an HDFS connection using the Flow Service API and have obtained the connection鈥檚 unique ID value. You can use this ID in the next tutorial as you learn how to explore a third-party cloud storage using the Flow Service API.

recommendation-more-help
337b99bb-92fb-42ae-b6b7-c7042161d089