Skip to main content

Smart Build technical knowledge

Last updated on December 27, 2023

What is Smart Build?

Smart Build is a new feature that will replace the Legacy build distribution system, which will improve the speed of uploading and downloading a game binary. Smart Build works by understanding the game archive format, extracting it, and only uploading the differences from each version.

What is the difference between Smart Build and Legacy Build distribution system?

The Legacy build distribution system is just a regular file upload. There is no binary diffing or patching on the Legacy Build. Smart Build does not use binary diffing or patching. It works on the game asset file level inside the game archive file, so that it improves the speed and bandwidth for upload and download.

Refreshment about the data structure

The following image shows the data structure of Smart Build. It starts from Game, Version, and Build. Think about build as a platform for game binary to upload. The game binary can be pointed to Legacy or Smart Build.

Smart Build data structure

Example

The process for uploading a game binary for the Legacy build distribution is as follows:

  1. Create a version on the CLI.
  2. Register a build on the CLI.
  3. Compile or Build your game.
  4. Upload the game binary using the Legacy upload command.

The process for uploading a game binary for a Smart Build is as follows:

  1. Create a version on the CLI.
  2. Register a build on the CLI.
  3. Compile or Build your game.
  4. Upload the game binary using the Smart Build upload command.

Upload build using Smart Build

Use this CLI command to upload using Smart Build:

./BlackBoxCLI.exe smartbuild upload 
--apikey {API_KEY}
--namespace {NAMESPACE}
--game-project {GAME_PROJECT_DIR}
--entry-point=${ENTRY_POINT}
--upload-dir=${GAME_DIR}

The following are example logs when you upload using Smart Build.

[SmartBuild] [info] 2023-12-07 11:45:50: Log file stored at: C:/Users/irfan.r_accelbyte/.blackbox/smartbuild_20231207T044550Z.log
[SmartBuild] [info] 2023-12-07 11:45:50: Log enabled
version :0.13.1
commit :ff443d9
timestamp:2023-11-23T06:32:08Z
[SmartBuild] [info] 2023-12-07 11:45:50: Local cache: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache
[SmartBuild] [info] 2023-12-07 11:45:50: PHASE 1: checking manifest for current build
[SmartBuild] [info] 2023-12-07 11:45:50: Check manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960
[SmartBuild] [info] 2023-12-07 11:45:50: Fetching manifest with files...
[SmartBuild] [info] 2023-12-07 11:45:50: Fetching manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960
[SmartBuild] [info] 2023-12-07 11:45:51: Fetching manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960 ends and it took 1 seconds
[SmartBuild] [info] 2023-12-07 11:45:51: Check manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960 ends and it took 1 seconds
[SmartBuild] [info] 2023-12-07 11:45:51: E:/Avengers/smartbuild/C8AutomationBuild/EnemiesUnity-Init/Enemies2023.exe
[SmartBuild] [info] 2023-12-07 11:45:51: PHASE 1 result : Create new manifest
[SmartBuild] [info] 2023-12-07 11:45:51: PHASE 2A: [Create new manifest] scan folder, detect schema, calculate hash
[SmartBuild] [info] 2023-12-07 11:45:51: Fetching user specified ignore list...
[SmartBuild] [info] 2023-12-07 11:45:51: Fetching user specified ignore list finished and it took 0 seconds
[SmartBuild] [info] 2023-12-07 11:46:03: File: .ignoreconfig process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: buildID process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Config/Preset-Medium.txt process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Enemies2023_Data/app.info process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Enemies2023_Data/boot.config process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Enemies2023_Data/globalgamemanagers process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Enemies2023_Data/globalgamemanagers.assets process with unity processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Enemies2023_Data/globalgamemanagers.assets.resS process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Enemies2023_Data/globalgamemanagers.assets.txt process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:03: File: Enemies2023_Data/level0 process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:04: File: Enemies2023_Data/level1 process with unity processor.
[SmartBuild] [info] 2023-12-07 11:46:04: File: nvngx_dlss.dll process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:04: File: NVUnityPlugin.dll process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:04: File: README-Enemies.txt process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:04: File: UnityCrashHandler64.exe process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:04: File: UnityPlayer.dll process with default processor.
[SmartBuild] [info] 2023-12-07 11:46:04: PHASE 2: File Processor, extract, put into list
[SmartBuild] [info] 2023-12-07 11:46:04: Creating cached folder for package manifest: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache/manifest/1f34b359-6c3b-41a2-8365-5f97b36383a0/731b2696-f78b-4843-9e75-43d07e421960
[SmartBuild] [info] 2023-12-07 11:46:04: start processing all files...
[SmartBuild] [info] 2023-12-07 11:46:04: Parsing Unity Asset: E:\Avengers\smartbuild\C8AutomationBuild\EnemiesUnity-Init\Enemies2023_Data\globalgamemanagers.assets
[SmartBuild] [info] 2023-12-07 11:46:04: Saving package manifest to local cache: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache/manifest/1f34b359-6c3b-41a2-8365-5f97b36383a0/731b2696-f78b-4843-9e75-43d07e421960/48154c221f9fa3caf99c9a405e1912b33939b1f00d37e8f5078cb4143d33dccc_295305
[SmartBuild] [info] 2023-12-07 11:46:04: Parsing Unity Asset: E:\Avengers\smartbuild\C8AutomationBuild\EnemiesUnity-Init\Enemies2023_Data\level1
[SmartBuild] [info] 2023-12-07 11:46:06: Saving package manifest to local cache: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache/manifest/1f34b359-6c3b-41a2-8365-5f97b36383a0/731b2696-f78b-4843-9e75-43d07e421960/21d4eefeb73ca809a18c42f45e3fe53ddd03bbdf9cd37ca0b895dd0f3c45adea_1541596
[SmartBuild] [info] 2023-12-07 11:46:06: Parsing Unity Asset: E:\Avengers\smartbuild\C8AutomationBuild\EnemiesUnity-Init\Enemies2023_Data\Resources\unity default resources
[SmartBuild] [info] 2023-12-07 11:46:06: Saving package manifest to local cache: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache/manifest/1f34b359-6c3b-41a2-8365-5f97b36383a0/731b2696-f78b-4843-9e75-43d07e421960/82efe0c4fc2049bf0d274eab7e30eed361ee46d363e8b063c689ca00dad39b3d_9380
[SmartBuild] [info] 2023-12-07 11:46:06: Parsing Unity Asset: E:\Avengers\smartbuild\C8AutomationBuild\EnemiesUnity-Init\Enemies2023_Data\resources.assets
[SmartBuild] [info] 2023-12-07 11:46:06: Saving package manifest to local cache: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache/manifest/1f34b359-6c3b-41a2-8365-5f97b36383a0/731b2696-f78b-4843-9e75-43d07e421960/f7b5ea6d01117e9bc128fa10d71e09830d6ffef5bc109837803d1e8d7f3684b4_151026
[SmartBuild] [info] 2023-12-07 11:46:06: Parsing Unity Asset: E:\Avengers\smartbuild\C8AutomationBuild\EnemiesUnity-Init\Enemies2023_Data\sharedassets1.assets
[SmartBuild] [info] 2023-12-07 11:46:13: Saving package manifest to local cache: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache/manifest/1f34b359-6c3b-41a2-8365-5f97b36383a0/731b2696-f78b-4843-9e75-43d07e421960/cc43d3d03163ca71ea610290a61723fecce8eaf0caae8aa1ad9c33fde3b16f12_296778
[SmartBuild] [info] 2023-12-07 11:46:16: Create manifest
[SmartBuild] [info] 2023-12-07 11:46:16: getting info about build ID 731b2696-f78b-4843-9e75-43d07e421960
[SmartBuild] [info] 2023-12-07 11:46:16: getting info about build ID 731b2696-f78b-4843-9e75-43d07e421960 ends and it took 0 seconds
[SmartBuild] [info] 2023-12-07 11:46:16: Appending platform flag 1024 to build ID 731b2696-f78b-4843-9e75-43d07e421960
[SmartBuild] [info] 2023-12-07 11:46:17: Appending platform flag 1024 to build ID 731b2696-f78b-4843-9e75-43d07e421960 ends and it took 0 seconds
[SmartBuild] [info] 2023-12-07 11:46:17: Serializing manifest...
[SmartBuild] [info] 2023-12-07 11:46:17: Serializing manifest ends and it took 0 seconds
[SmartBuild] [info] 2023-12-07 11:46:17: Saving manifest to local is enabled and will be written to E:/Avengers/Cycle9/SB/ManifestCycle9.json
[SmartBuild] [info] 2023-12-07 11:46:17: Saving manifest to local...
[SmartBuild] [info] 2023-12-07 11:46:17: Saving manifest to local ends and it took 0 seconds
[SmartBuild] [info] 2023-12-07 11:46:17: Submitting initial manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960
[SmartBuild] [info] 2023-12-07 11:46:39: Submit response manifest response saved in: C:/Users/irfan.r_accelbyte/.blackbox/sm_cache/manifest/1f34b359-6c3b-41a2-8365-5f97b36383a0/1f34b359-6c3b-41a2-8365-5f97b36383a0.json
[SmartBuild] [info] 2023-12-07 11:46:40: Submitting initial manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960 ends and it took 22 seconds
[SmartBuild] [info] 2023-12-07 11:46:40: Uploading files (hash) : 20910 of 20911
[SmartBuild] [info] 2023-12-07 11:46:40: Uploading 20910 files
{"status":"uploading files","current":22,"total":20910,"filename":"[Memory Mapped File Buffer] ComputeShader_20.bin","filesize":4764,"current_byte":78766,"total_byte":2238132545,"rate":0.0,"time_remaining":0.0}
{"status":"uploading files","current":52,"total":20910,"filename":"[Memory Mapped File Buffer] MonoScript_1675.bin","filesize":148,"current_byte":250608,"total_byte":2238132545,"rate":7470.142180094786,"time_remaining":299576.88663050375}
{"status":"uploading files","current":89,"total":20910,"filename":"[Memory Mapped File Buffer] Transform_7784.bin","filesize":104,"current_byte":704515,"total_byte":2238132545,"rate":24853.94921812396,"time_remaining":90023.03860701647}
{"status":"uploading files","current":118,"total":20910,"filename":"[Memory Mapped File Buffer] GameObject_1279.bin","filesize":47,"current_byte":1395415,"total_byte":2238132545,"rate":63844.38368019782,"time_remaining":35034.20349711597}
{"status":"uploading files","current":150,"total":20910,"filename":"[Memory Mapped File Buffer] MeshRenderer_10180.bin","filesize":156,"current_byte":8692126,"total_byte":2238132545,"rate":137642.53237685465,"time_remaining":16197.32200869397}
{"status":"uploading files","current":175,"total":20910,"filename":"[Memory Mapped File Buffer] MonoScript_2464.bin","filesize":140,"current_byte":19826999,"total_byte":2238132545,"rate":1136713.4862397609,"time_remaining":1951.5080738050685}
{"status":"uploading files","current":210,"total":20910,"filename":"[Memory Mapped File Buffer] GameObject_4214.bin","filesize":43,"current_byte":25024191,"total_byte":2238132545,"rate":1891192.5344411817,"time_remaining":1170.2184276303415}
{"status":"uploading files","current":240,"total":20910,"filename":"[Memory Mapped File Buffer] GameObject_3481.bin","filesize":63,"current_byte":31209895,"total_byte":2238132545,"rate":2289023.1809970634,"time_remaining":964.1329403395113}
{"status":"uploading files","current":20910,"total":20910,"filename":"[Memory Mapped File Buffer] MeshRenderer_10468.bin","filesize":156,"current_byte":2238132545,"total_byte":2238132545,"rate":704684.7866437414,"time_remaining":0.0}
[SmartBuild] [info] 2023-12-07 11:56:23: Uploading 20910 files ends and it took 582 seconds
[SmartBuild] [info] 2023-12-07 11:56:23: Committing manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960
[SmartBuild] [info] 2023-12-07 11:56:23: Committing manifest for build ID 731b2696-f78b-4843-9e75-43d07e421960 ends and it took 0 seconds

Work with Channel and Track

The Channel and Track feature was introduced in the recent ADT Hub revamp. This section explains the data structure between the channel and track relationship.

Channel and Track structure

Channel is a group of tracks. You can manage multiple channels where each channel can be for a different purpose. For example, Channel for QA, Channel for a stable build, Channel for nightly build, etc.

A channel may contain one or more track. A track represents builds on different platforms that serve that channel purpose. The track will use the Game binary that was uploaded to the build as a reference.

You can think about stack, we called the top of the stack as HEAD, that will use track for latest build.

You can update the track with a new build and the previous build will become history.

The following is an example of a channel. In this example, the channel is called Daily build - Unreal, which consists of two tracks: Linux and Windows. The Windows build points to 2.51-demo-client-windows.

Legacy build in the same channel

Use Smart Build with Channel

There is no difference between Legacy or Smart Build if you want to use Channel and Track. To learn how to create a Channel and Track, see Use tracks to organize channels.

Use Legacy build distribution system and Smart Build side-by-side

Let's assume this is your build pipeline.

Legacy CLI command

If you want to make the Legacy build and Smart build to work side-by-side, you need to create another build after it finishes the upload to the Legacy build system.

Legacy and Smart Build CLI command

You need to create another build, under the same version, one build can only have one game binary either Legacy or Smart Build. So, by the end you will have two build under same version, one is Legacy build the other one is Smart Build.

To create another build, follow these steps:

  1. Use this command to create another build using CLI:

    ./BlackBoxCLI.exe build register
    --smartbuild
    --platform-name {PLATFORM}
    --platform-arch {ARCHITECTURE}
    --namespace {NAMESPACE}
    --apikey {APIKEY}
    --game-engine {GAME_ENGINE_DIR}
    --game-project {GAME_PROJECT_DIR}
  2. Once you create a build, upload the game binary using Smart build to the newly created build using this command:

    ./BlackBoxCLI.exe smartbuild upload 
    --apikey {API_KEY}
    --namespace {NAMESPACE}
    --game-project {GAME_PROJECT_DIR} --entry-point=${ENTRY_POINT}
    --upload-dir=${GAME_DIR}
  3. Verify the upload in the ADT Hub. Go to the Versions tab. When the upload is finished, under the same version you will see two builds: one with Legacy and the other one is using Smart Build.

    Legacy and Smart Build side by side in a Version