AMS troubleshooting guide

Overview

This page contains troubleshooting information for possible issues you may encounter while using AccelByte Multiplayer Servers (AMS).

Fleet issues

I don't see any VMs running

It may take up to 10 minutes for the VM to be provisioned. Please wait and check again.

I see a VM count greater than zero, but there are no dedicated servers (DS) running

• Check the History tab to see if DS were being created. If they are crashing, check the DS logs.
• Verify the DS startup command to ensure it is correctly configured.
• Confirm that your game build is not faulty (i.e. a "bad build").

How can I test and debug my game locally with your runtime environment?

You can replicate our runtime environment locally. Follow these steps:

important

This container definition is intended for debugging and troubleshooting purposes only. It disables the AMS software and will not function with AMS directly.

1. Create a Docker file with the following contents:

   FROM ubuntu:22.04 AS base-watchdog
   
   ARG FORCE_REBUILD=0
   ARG TARGETOS
   ARG TARGETARCH
   ENV DEBIAN_FRONTEND noninteractive
   
   RUN apt-get update && apt-get install -y ca-certificates curl unzip
   RUN if [ "$TARGETARCH" = "amd64" ] ; then curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" ; else curl "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "awscliv2.zip" ; fi
   RUN unzip awscliv2.zip && ./aws/install
   
   # crash reporter package
   RUN apt-get install -y libc-dev libc++-dev libc++abi-dev uuid-dev gdb
   
   ARG WATCHDOG_NAME=watchdog
   ARG WATCHDOG_UID=1000
   
   # The armada group exists in the host VM, so we match the GID here so it can access armada files.
   ARG ARMADA_NAME=armada
   ARG ARMADA_GID=1100
   
   # Create `watchdog` user in the `armada` group
   RUN groupadd --gid $ARMADA_GID $ARMADA_NAME
   RUN useradd --uid $WATCHDOG_UID --gid $ARMADA_GID -m $WATCHDOG_NAME
   
   RUN mkdir /artifacts /alloc /ds /latest
   
   # Added Logic to copy Local DS for debugging
   COPY /<local_ds_package> /ds/
   
   RUN chown $WATCHDOG_NAME:$ARMADA_NAME /artifacts /alloc /ds /latest
   USER $WATCHDOG_NAME:$ARMADA_NAME
   
   # Copy Watchdog
   #COPY deployment/start.sh /start.sh
   #COPY bin/*${TARGETOS}_${TARGETARCH}*/watchdog /watchdog
   
   WORKDIR /ds
   ENTRYPOINT ["/bin/bash", "/ds/<ds_start_script.sh>"]

2. Modify the contents as follows:

   • Specify and copy the contents of your DS package into the /ds directory in this image.
   • Change the entrypoint to your dedicated server startup script.
   • Pass in the appropriate parameters to your startup script.

3. Build the container using the docker build command. For more information about container assembly, refer to Docker's official documentation.

Does your DS have the necessary AGS permissions?

If your DS relies on AccelByte Gaming Services (AGS), ensure that you created and configured its IAM client with the correct permissions and namespace settings. For more information, refer to the creating an IAM client guide.

Dedicated server issues

Does the instance type have the sufficient resources to run the server?

• Ensure your VM selection is appropriate for your DS's CPU and memory requirements. Consider using a larger VM if crashing persists.
• Use the AMS DS Metrics dashboard in Grafana to monitor the resource usage of individual DS and VMs.

DS crashes on startup or is terminated by the creation timeout

Ensure the DS notifies AMS that it is ready to serve players by sending the Ready Message within the creation timeout to avoid being removed. To test it locally, use the AMS Simulator to verify that the DS becomes "Ready".

Logs and Artifacts issues

I can't find logs for my DS

Verify that log sampling is enabled for your fleet.

Log sampling is enabled, but logs are still missing

• Use Grafana to investigate the watchdog and artifact collector logs:
  1. Open the Explore view in Grafana.
  2. Select the Logs data source.
  3. Apply label filters:
     • ams_fleet = your_fleet_id
     • service_name = ams-watchdog or ams-artifact-collector
  4. Apply Logfmt operations to use the Logfmt parser.
  5. Run query to review the sequence of events and messages based on DS states.

The DS is reported to have "CRASHED," but there is no core dump

Ensure core dump sampling is enabled for your fleet.

注記

A DS is considered "CRASHED" if it exits with a non-zero code. For example, Exit code 137 indicates the process was killed due to running out of memory (OOM). Use Grafana to check for memory usage.

DS Claiming issues

Player sessions are reporting failures in finding a DS

• Check your fleet to see if it has DS instances in a "Ready" state. You can check this in the fleet details on the Admin Portal, or use Grafana for a historical view.
• Consider increasing the buffer and max server count to meet higher demand.

To learn more about tailoring your fleet size to your specific game requirements, refer to the Optimizing fleet size page.

I have DS instances in the ready state, but players can't claim them

• Use Grafana to confirm that your DS instances are running in the regions requested by players.
• Check the claim keys on your fleet to ensure they match the claim keys on the session template or the client version specified by the game client.
• Verify if Quality of Service (QoS) is enabled for the target region.
• Find your session in Session & Party History and verify that a DS was requested
  • Verify that RequestedRegions matches the region of your fleet

Players are being kicked out of sessions mid-game

• Ensure your DS correctly handles the drain signal, as recommended in the Listening to the drain signal section in the Integrate your dedicated server to AMS using the AGS SDK page.