Published
Last updated
Data exposure via ZeroMQ on multi-node vLLM deployment
In a multi-node vLLM deployment, vLLM uses ZeroMQ for some multi-node communication purposes. The primary vLLM host opens an XPUB
ZeroMQ socket and binds it to ALL interfaces. While the socket is always opened for a multi-node deployment, it is only used when doing tensor parallelism across multiple hosts.
Any client with network access to this host can connect to this XPUB
socket unless its port is blocked by a firewall. Once connected, these arbitrary clients will receive all of the same data broadcasted to all of the secondary vLLM hosts. This data is internal vLLM state information that is not useful to an attacker.
By potentially connecting to this socket many times and not reading data published to them, an attacker can also cause a denial of service by slowing down or potentially blocking the publisher.
The XPUB
socket in question is created here:
Data is published over this socket via MessageQueue.enqueue()
which is called by MessageQueue.broadcast_object()
:
The MessageQueue.broadcast_object()
method is called by the GroupCoordinator.broadcast_object()
method in parallel_state.py
:
The broadcast over ZeroMQ is only done if the GroupCoordinator
was created with use_message_queue_broadcaster
set to True
:
The only case where GroupCoordinator
is created with use_message_queue_broadcaster
is the coordinator for the tensor parallelism group:
To determine what data is broadcasted to the tensor parallism group, we must continue tracing. GroupCoordinator.broadcast_object()
is called by GroupCoordinator.broadcoast_tensor_dict()
:
which is called by broadcast_tensor_dict()
in communication_op.py
:
If we look at _get_driver_input_and_broadcast()
in the V0 worker_base.py
, we'll see how this tensor dict is formed:
but the data actually sent over ZeroMQ is the metadata_list
portion that is split from this tensor_dict
. The tensor parts are sent via torch.distributed
and only metadata about those tensors is sent via ZeroMQ.
Prior to the fix, your options include:
XPUB
socket. Note that port used is random.