Manage data connectors using API

Last updated on December 23, 2024

Introduction

This article walks you through how you can manage data connectors in AccelByte Gaming Services (AGS) using API.

Retrieve list of topics or analytics data

Retrieve a list of all available namespaces and topics using [GET] /analytics-connector/v1/admin/tools/kafka/topics/{eventType}. Replace {eventType} with the appropriate event type. The API will return a response containing namespaces and their associated topics.

{
  "namespace1" : [
    "topic-analytics-event-1",
    "topic-analytics-event-2"
  ] 
}

Get connector status

Check the status of a specific connector using [GET] /analytics-connector/v1/admin/connectors/{id}/status. Replace {id} with the connector's ID obtained from the API response after successful creation. The API will return the connector's status and related information.

{
  "time": "2023-06-08T10:22:01Z",
  "connector": "<connector-name>",
  "status": "ACTIVE",
  "error": null
}

Get connector list

Retrieve a list of all connector configurations using [GET] /analytics-connector/v1/admin/connectors. The API will return a list of connector configurations.

Pause a connector

Pause a specific connector using [PUT] /analytics-connector/v1/admin/connectors/{id}/deactivate. Replace {id} with the connector's ID. The API will return a response that confirms the pause.

{
  "id": 1,
  "active": false
}

danger

When a connector is paused, the events are temporarily stored on Kafka. Note that the default data retention of Kafka is seven days. If you pause the connector for more than seven days, you may lose data. Ensure that you backfill your data.

Resume connector

Resume or reactivate a specific connector using [PUT] /analytics-connector/v1/admin/connectors/{id}/activate. Replace {id} with the connector's ID. The API will return a response that confirms the reactivation.

{
  "id": 1,
  "active": true
}

Edit a connector

Edit the configuration of a specific connector using [PUT] /analytics-connector/v1/admin/connectors{id}. Replace {id} with the connector's ID. Modify the connector's configuration in the provided JSON structure.

Example S3 connector

{
    "storageType": "KAFKA_S3",
    "config": {
        "bucketName": "<s3-bucket-name>", 
        "region": "<s3-bucket-region>",
        "customPath": "<s3-path>",
        "customPartitionDuration": "<s3-partition-duration>",
        "partSize": "5242880",
        "flushInterval": "<flush-sending-interval>",
        "eventType": "<event-type>", 
        "flushSize": "<flush-size>"
    },
    "filter": {
        "<namespace>": [
            "<topic>"
        ]
    }
}

note

The event type (source) and data warehouse (destination) configurations cannot be edited.

note

The event type, storage type, and connector name cannot be changed.

Delete a connector

Delete a specific connector using [DELETE] /analytics-connector/v1/admin/connectors/{id}. Replace {id} with the connector's ID. The API will return a response that confirms the deletion.

{
  "deleted": true,
  "id": 1
}