modular/max-amd repository overview

⁠max-amd

The max-amd container delivers high-performance inference on AMD GPUs using the MAX framework, with full support for PyTorch GPU and ROCm. It provides a ready-to-use environment for running LLMs with optimized performance on AMD hardware.

The MAX container is compatible with the OpenAI API specification and optimized for GPU deployment. For details on container contents and hardware compatibility, see MAX containers⁠ in the MAX documentation.

⁠Quickstart

You can run an LLM on AMD GPUs using the latest MAX full container (AMD) with the following command:

docker run \
    --group-add keep-groups \
    --rm \
    --env "HF_TOKEN=$HF_TOKEN" \
    -p 8000:8000 \
    --device /dev/kfd \
    --device /dev/dri \
    modular/max-amd:latest \
    --model-path <model-provider/model-id>

You can run a MAX model⁠ by referencing its HuggingFace model ID. For example, google/gemma-3-1b-it⁠.

You can also use the MAX container to run a variety of PyTorch models⁠ hosted on Hugging Face, such as microsoft/Phi-3.5-vision-instruct⁠.

For more information on deploying popular models with MAX, see the model support⁠ documentation.

⁠Tags

Supported tags are updated to the latest MAX versions, which include the latest stable release and more experimental nightly releases. The latest tag provides you with the latest stable version and the nightly tag provides you with the latest nightly version.

Stable

• modular/max-amd:25.X

Nightlies

• modular/max-amd:25.X.0.devYYYYMMDD

⁠Documentation

For more information on Modular and its products, visit the Modular documentation site⁠⁠⁠.

⁠Community

To stay up to date with new releases, sign up for our newsletter⁠⁠⁠, check out the community⁠⁠⁠, and join our forum⁠⁠⁠.

If you're interested in becoming a design partner to get early access and give us feedback, please contact us⁠⁠⁠.