Skip to main content

AMS Launch Preparation Checklist

Last updated on February 3, 57448

This article helps you to transition your game from development to production using AccelByte Multiplayer Servers (AMS). Moving from a development phase to a live game with thousands of players requires careful planning to ensure your game stays highly available for all players.

Start your planning as early as possible. Accelbyte's server providers have limitations. If you need a large amount of capacity, please work with your Accelbyte Account Manager. A large numbers of machines would by anything greater than ten Bare Metal servers per region, and greater than one thousand vCPUs per region. Our Bare Metal providers and Cloud providers take a minimum of four weeks to respond to a capacity request increase.

Fill in our AMS Launch Readiness Questionnaire (Download) and use it as a checklist for you to make sure you considered the various aspects of a public game launch or event. It captures all topics discussed in this article, but we provide this article to elaborate on the question in the Launch Readiness Questionnaire, in case you want more context to answer certain questions. For reference, we include a copy of the questionnaire's contents below.

Questionnaire Contents (download above)

Fill in this questionnaire to prepare for a launch or event. If a question does not apply to your situation you can skip it. For instance, the questions about load test may not be relevant, if you choose not to perform load tests. Use this questionnaire as a checklist for you to make sure you considered the various aspects of a public game launch.

General

  • When will the public be able to play the game (date and time):
  • Type of event (e.g.: early access, platform launch, territory launch (e.g.: soft launch in a specific country), marketing beat (TV advertising push, free on Epic Game Store or live on GamePass)):
  • Peak concurrent user count (PCCU):
    • Bear case (OK):
    • Bull case (good):
    • Home run case (wildly successful):

Deployment

Which deployment will be used for each launch event?

  • AGS Environment URL:
  • AGS Namespace:
  • AMS Image(s):
  • AMS Fleet(s):
  • Please fill in which Game mode(s), and Instance Types (VM or Bare metal) you will use:
Game ModeAMS Instance TypeServers per VMMatch Configuration (i.e.: 1v1, 2v2)% playersTypical Session LengthMax Session Length

Capacity planning

Please fill in which AMS regions you are enabling for your Game:

RegionRegional CCU by % of target peak CCUAnticipated CCU upon gate openingUser ramp up / login queue rateFleet Max Size [DS]Fleet Buffer Size [DS]
North America East (us-east-2)
North America West (us-west-2)
Asia Northeast (ap-northeast-2)
Europe Central (eu-central-1)
Asia Southeast (ap-southeast-1)
South America East (sa-east-1)
Australia East (ap-southeast-2)

Capacity Limits

  • Are your account's capacity limits (per region and per instance type) sufficient for your target CCU?
  • Do you use bare metal?
    • Which fallback fleets do you enable?

Dedicated server performance and behavior

  • Does the DS terminate under the following conditions:
    • When the last player has left and the game has ended?
    • When the last player has left and the game is no longer viable, e.g: the players will not reconnect back to the game?
    • When the DS has been claimed but no player connects?
  • When a drain signal is sent to the DS what actions does it take:
    • When in session?
    • When not in-session?
  • Are timeouts in your fleet configured:
    • Session timeout is greater than your maximum session duration?
  • Does your game server dynamically extend the session timeout (e.g. for persistent game worlds)?
  • Does the DS spawn additional processes? If so, what is the trigger for spawning a child process, what is the child process responsible for and what is the trigger for terminating the child process?
  • Have you stress tested your Instance type for your Servers per VM configuration? And checked DS performance metrics on Grafana?
  • Have you tested your DS running on different instance types?

Deployment Strategy and updates

  • Have you tested your deployment strategy (blue-green or canary)?
  • When you update your game client, will it use the same claim keys (and therefore use the same dedicated server version) or new claim keys (and therefore require a new fleet to be active)?

Monitoring and Debugging

  • Can you view logs of a running DS? And can you download logs of a exited DS (crashed or successful)?
  • Do you collect DS logs and have you configured log sampling rules accordingly (e.g. 100% for crashes, 1-5% for successful servers)?
  • Do you collect artifacts beside logs and core dumps? Which?
  • Can you access Grafana Cloud for monitoring AMS metrics?
    • Do you know how to see claim failures in a given region?
    • Do you know how to monitor the number of Ready DS (buffer)?
  • Do you plan to (periodically) monitor and adjust your fleet's buffer size based on its requirement?

Load Testing

  • Have you tested that your fleet can scale to your target PCCU?
  • What are your external dependencies (Extend or otherwise)? If any, are they tested under load?

Access Rights

  • Can all team members who need to operate AMS Admin Portal or other tools, access them?
info

As the game studio, you are responsible for ensuring a successful launch. AMS provides the tools and infrastructure, but you must actively monitor and manage your game's health. The following guide helps you prepare your game for a successful launch.

info

For major launches, please submit the completed questionnaire to your Account Manager in advance so it can be reviewed by the Solution Architect team (available for Standard, Professional, and Enterprise support plans).

Fleet Sizing / Capacity Planning

Capacity planning requires estimating your peak CCU (PCCU) per region. Use the template in the provided above to estimate your fleet sizes per region.

Bare metal

If your fleet uses a bare metal instance type, Configure a Fallback Fleet to allow bursting to larger DS counts than your static capacity.

Capacity limits

Ensure that the capacity limit (soft-limit) for your AMS account is higher than your target. Be aware that these limits are per region and per instance type, so check all the regions and instance types that you use. For details, see Capacity Limitations. Consider mitigating risk of capacity limits by diversifying and having multiple fleets spread across different cloud providers.

Deployment Strategies (game updates)

Implement and test deployment strategies to ensure smooth updates and rollbacks of your dedicated server versions. Oftentimes, games require an update on day one. And when it's crucial to roll-out the update, you don't want it to negatively impact live players. Therefore, make sure you understand how to use these strategies for rolling out and performing an update on a live fleet:

  • Blue-Green Deployment: For updating your dedicated server without downtime by maintaining two identical fleets and gradually shifting traffic. Learn more.
  • Canary Deployment: For safely testing new versions with a small percentage of live traffic before full rollout. Learn more.

Dedicated server compatibility

As part of your deployment strategy, consider if a new game client will be compatible with a old dedicated server. Can you update your game client without updating the dedicated server image? (And vice-versa.) If not, ensure that the claim request generated by a game client only contains a claim key for a compatible server. Similar to the deployment strategies linked above, learn about Supporting multiple versions of a server.

Testing deployments

Assuming you use one of these strategies, we recommend testing it before launch. For example by running 2 (or more) fleets simultaneously with different claim keys, and two (incompatible) game clients, and test that they only claim a dedicated server from the compatible fleet.

Monitoring

Proper monitoring is essential for production success. Before launch, learn how you can actively monitor Metrics and DS crashes and logs during launch.

Metrics

Refer to AMS Service Dashboards. Play around with these metrics during development to get a better understanding of them, for example by simulating a claim failure and observing the effect in the metrics. We recommend to be aware of how to monitor the following:

  • Claim failures: Monitor for claim failures.
  • Fleet Sizing: Track server utilization, in particular ensure you have sufficient READY servers in each region.
  • Performance Metrics: Monitor CPU, memory, and network usage of your dedicated servers.
  • Claim Rates: Monitor the rate of server claims. A sudden drop could indicate an issue.

Crashes and Logs (dedicated servers)

Dedicated servers may crash (or experience less obvious issues during operation). Monitor for dedicated servers in failure states (e.g., CRASHED, or ones stuck in CREATING) through the fleet details page in the Admin Portal.

  • Configure Log Sampling Rules before launch to ensure you collect logs of exited dedicated servers:
    • Initially, set crash log collection rate to 100% for easier debugging, and keep successful server log collection rate low (1-5%) to manage storage costs.
  • Learn how to view real-time logs using the log streaming feature on running servers.

Dedicated Server Build Validation

Near launch, we assume that your dedicated server already integrates with AMS and can be claimed. As a extra measure, we recommend to test the following scenarios, as they are not necessarily tested during development, but are crucial when scaling up to a larger audience.

Before launch, ensure your dedicated server build:

  • Exits when the game session is finished. It should exit gracefully without error.
    • Exits when the last player left and the game has ended (i.e. the match finished).
    • Exits when the last player left and the game is no longer viable (e.g., players will not reconnect).
    • Exits when the DS has been claimed but no player ever connects.
  • Exits with a non-zero exit code when a fatal error occurs. (This results in the DS displaying a CRASHED status in AMS.)
  • Listens to the drain signal correctly: ignoring it when in-session and exiting gracefully otherwise. See Listening to the drain signal for details.
  • Is performance tested at the DS per VM packing configuration on the specified VM instance type. If the dedicated server supports different game modes, maps, and team configurations, test that none of these exceed your expected limits.
    • Test this by claiming all DS in an instance and running gameplay on them, to validate their behavior under full utilization.
  • Is validated in live AMS fleet, end-to-end, using the same production fleet configuration you will use in launch.

Timeouts

Before launch, also check that your fleet have configured the correct timeouts. We try to provide sane defaults in Admin Portal, but as every DS behaves differently, you will need to set them according to your game's needs. Having timeouts does not replace proper testing of the scenarios mentioned above, but it acts as a fallback to prevent DS from living indefinitely and causing scaling issues. See Fleet Creation for details on configuring timeouts.

Packing and instance selection

Select the optimal instance type and DS per VM configuration for your game. Typically, larger instance sizes are more cost effective, and allow for faster scaling because more DS can be packed on one machine. Too many DS per VM can lead to resource contention, so monitor carefully. Use the AMS DS Metrics dashboard to track CPU, memory, and network usage per VM. These metrics will help you identify if you're over-packing your instances or if you have room to increase DS density. Refer to Choosing an instance type for your fleet.

Load Testing

AMS infrastructure has been load tested and ensures stable performance for your dedicated servers. Yet, for a smooth launch we ask you to consider the following:

Capacity Testing

As every dedicated server has different start up times, scaling speeds differ. Before launch it may be a useful exercise to start a fleet at your max size (assuming your target PCCU) by briefly setting the min size equal to the max size, and note if there are unexpected delays in the scaling behavior. We especially recommend running such a test if using large fleets, to validate that your fleets can scale out to your desired size.

Planning for Capacity Limits

Cloud service providers have limitations. Always ensure your dedicate server can run on more than one instance type. AMS provides hosting on several different instance types on three clouds; we encourage you to test and verify your Dedicated Server process runs on more than one instance type and one cloud. Leveraging multiple clouds allows you to reduce the risk of running out of instances or capacity with single cloud. Cloud providers do not provide an absolute guarantee they will have capacity of specific instances, they will only make available capacity if the capacity exists in that region. Always configuring your fleet to take advantage of the ability to run your fleet on multiple cloud offered by setting up secondary fleets spread across AMS's different cloud offerings.

If you are looking to have a reserved capacity for a launch or event, consider bare metal servers. Bare metal servers are always on and dedicated to your AMS account. Bare metal servers provide the best performance at the lowest cost. If you have concerns about commitment on bare metal reach out to your account team to discuss additional options.

Gate Rushes

Dealing with gate rush events is very common, a Gate rush is a rapid and sudden increase in players typically planned such as a patch, game unlock or launch, or a planned promotional event at a scheduled time. Adjusting your AMS fleet configuration to use a higher minimum number of servers, ensure that you have additional capacity online and ready for these players when they arrive in the system. The AMS platform is constantly adjusting capacity, cloud instances can take up to ten minutes to start and be ready to handle traffic, we have observed that typical launch time for most cloud providers is around two and half minutes. Use the minimum servers configuration to absorb the sudden influx of thousands of players.

External dependencies and Extend

If your dedicated server relies on external dependencies not part of AMS, ensure these are properly scaled and tested before launch. These fall out of scope for this guide but be sure to include them in your load tests. Think of custom matchmaking services, analytics pipelines, or game state databases.

If you're using Extend (AGS's service for custom server-side logic), see Extend Performance Testing for guidance on preparing your Extend apps for launch.

Other things

  • Verify access rights for all team members who need to access AMS Admin Portal (and other tools).

Notify us

For major launches, please submit the completed questionnaire to your Account Manager in advance so it can be reviewed by the Solution Architect team (available for Standard, Professional, and Enterprise support plans). The minimum turn around time for all our server providers is four weeks, we recommend starting the capacity planning exercise at least three months before going live or planned events.