AIS Infrastructure

Last updated on December 17, 2024

Overview​

AccelByte Intelligence Service (AIS) is a data manager for analysis solutions. In the realm of data, the Extract Transform Load (ETL) process is the backbone for efficient data integration and analysis. This document will explain to you the behind-the-scenes and explore the high-level design principles that underpin its behaviors.

How does AIS ETL work?​

AIS ETL infrastructure

The following is a summary of the ETL process to make the data available for our customers:

  • AIS automatically collects AGS service data when players play your game that's built on AccelByte Gaming Services (AGS) backend services. You can also send customized messages to be processed through Custom Telemetry Service via client and server UE and Unity SDKs.
  • AGS and Custom Telemetry Service will produce Kafka messages that will be processed by AIS Streaming Service.
  • AIS Streaming Service will serialize and distribute the raw data either directly to the customer's Data Lake with AIS Connector Service or AIS Data Lake.
  • The raw data in the AIS Data Lake will be processed by AIS Clarity ETL to produce relational data in the AIS Data Warehouse.
  • Query-ready data will be stored in the AIS Data Warehouse and can be accessed through applications such as Customer Database IDE, Customer Dashboard, AIS Visualization API, and AIS Aggregation Engine.

AIS components​

AIS consists of the following components:

  • AGS: backend services that handle AGS features and they are consistently producing messages in their every operation. Such as Matchmaking, Authentication, Store, etc.

  • Customer Telemetry Service: an API service that can receive data and produce it as messages.

  • Streaming Service: a place where data is temporarily stored as messages.

  • AIS Data Connector: periodically processes messages, serializes them as JSON from the Streaming Service, and then sends them into the Data Lake. The objectives can be customized by customers into their Data Lake.

  • AIS Data Lake: storage that will keep raw data in a historical structure.

  • AIS Clarity: a data transformation program that is responsible for decoding, deserializing, and normalizing raw data into relational data.

  • AIS Data Warehouse: a database where the query-ready data is stored. It complies with Snowflake schema and Slowly Changing Dimension type 2. To connect to this data warehouse, see Connect to AIS Data Warehouse.

  • AIS Visualization API: an API service that you can use to obtain metrics data. It is currently being used by AGS Analytic Dashboard.

  • AIS Aggregation Engine: responsible for transforming relational data into business metrics by combining and summarizing multiple data points into a single meaningful business metric.

Last updated on

On this page