MACA¶

Requirements¶

OS: Linux
Python: 3.10 -- 3.12

Set up using pip (without UV)¶

Build wheel from source¶

Note

If using pip, all the build and installation steps are based on corresponding docker images. You can find them on quick start. We need to add -no-build-isolation flag (or an equivalent one) during package building, since all the requirements are already pre-installed in released docker image.

Setup environment variables¶

# setup MACA path
export MACA_PATH="/opt/maca"

# cu-bridge
export CUCC_PATH="${MACA_PATH}/tools/cu-bridge"
export CUDA_PATH=/root/cu-bridge/CUDA_DIR
export CUCC_CMAKE_ENTRY=2

# update PATH
export PATH=${MACA_PATH}/mxgpu_llvm/bin:${MACA_PATH}/bin:${CUCC_PATH}/tools:${CUCC_PATH}/bin:${PATH}
export LD_LIBRARY_PATH=${MACA_PATH}/lib:${MACA_PATH}/ompi/lib:${MACA_PATH}/mxgpu_llvm/lib:${LD_LIBRARY_PATH}

export VLLM_INSTALL_PUNICA_KERNELS=1

Build vllm¶

Clone vllm project:

git clone  --depth 1 --branch main https://github.com/vllm-project/vllm
cd vllm

Build with empty device:

python use_existing_torch.py
pip install -r requirements/build.txt
VLLM_TARGET_DEVICE=empty pip install -v . --no-build-isolation

Build plugin¶

Install the build requirments first:

python use_existing_metax.py
pip install -r requirements/build.txt

Build and install vLLM:

pip install . -v --no-build-isolation

If you want to develop vLLM, install it in editable mode instead.

pip install . -e -v --no-build-isolation

Optionally, build a portable wheel which you can then install elsewhere:

python -m build -w -n
pip install dist/*.whl

Set up using UV (experimental)¶

Todo