Cluster
A Geneva Cluster represents the backend compute infrastructure for the execution environment.
created_at: datetime = field(
factory=lambda: now(utc),
metadata={"pa_type": timestamp("us", tz="UTC")},
)
ray_init_kwargs: Optional[dict[str, Any]] = field(
default=None, metadata={"pa_type": string()}
)
Convert the persisted cluster definition into internal RayCluster model
create_kuberay(name: str) -> KubeRayClusterBuilder
Create a KubeRay cluster builder.
Example:
cluster = GenevaCluster.create_kuberay("my-cluster").namespace("ml").build()
create_local(name: str) -> LocalRayClusterBuilder
Create a Local Ray cluster builder.
Example:
cluster = GenevaCluster.create_local("local-dev").build()
create_external(
name: str, ray_address: str
) -> ExternalRayClusterBuilder
Create an External Ray cluster builder.
Example:
cluster = GenevaCluster.create_external("remote", "ray://host:10001").build()
Type-safe builder for KubeRay clusters deployed on Kubernetes.
Use this builder for clusters that will be deployed via KubeRay. For local development, use LocalRayClusterBuilder instead.
Example:
cluster = (
KubeRayClusterBuilder.create("my-cluster")
.namespace("ml-team")
.add_worker_group(
KubeRayClusterBuilder.gpu_worker(4)
.memory("64Gi")
.build()
)
.add_worker_group(
KubeRayClusterBuilder.cpu_worker()
.cpus(16)
.memory("32Gi")
.build()
)
.build()
)
config_method(
method: K8sConfigMethod,
) -> KubeRayClusterBuilder
Set the Kubernetes config method.
portforwarding(
enabled: bool = True,
) -> KubeRayClusterBuilder
Enable or disable port forwarding.
aws_config(
region: str | None = None, role_name: str | None = None
) -> KubeRayClusterBuilder
Configure AWS settings.
ray_init_kwargs(kwargs: dict) -> KubeRayClusterBuilder
Set arbitrary kwargs to pass to ray.init() when starting the cluster, such as env vars. Example:
.ray_init_kwargs({
"runtime_env": {
"env_vars": {
"MY_VAR": "value",
"AWS_ACCESS_KEY_ID": os.environ["AWS_ACCESS_KEY_ID"]
},
},
})
head_group(
*,
image: str | None = None,
cpus: int | None = None,
memory: str | None = None,
gpus: int | None = None,
service_account: str | None = None,
node_selector: dict[str, str] | None = None,
labels: dict[str, str] | None = None,
tolerations: list[dict[str, str]] | None = None,
) -> KubeRayClusterBuilder
Configure the head group with optional parameters.
add_worker_group(
worker: WorkerGroupConfig,
) -> KubeRayClusterBuilder
Add a worker group configuration.
cpu_worker() -> CpuWorkerBuilder
Create a CPU worker builder.
CPU workers do not have a gpus() method - use gpu_worker() for GPU workers.
Example:
cluster = (
KubeRayClusterBuilder.create("test")
.add_worker_group(KubeRayClusterBuilder.cpu_worker().cpus(8).build())
.build()
)
gpu_worker(gpus: int = 1) -> GpuWorkerBuilder
Create a GPU worker builder.
GPU workers have memory validation:
- Minimum 4GiB memory required (raises ValueError)
- Warning for memory > 100GB (may exceed node capacity)
Example:
cluster = (
KubeRayClusterBuilder.create("test")
.add_worker_group(KubeRayClusterBuilder.gpu_worker(4).memory("64Gi").build())
.build()
)
Builder for local Ray clusters.
Resources are managed by the local Ray runtime, so this builder does NOT have memory(), cpus(), or worker configuration methods.
Example:
cluster = LocalRayClusterBuilder.create("local-dev").build()
create(name: str) -> LocalRayClusterBuilder
Create a new builder with the given cluster name.
Builder for connecting to an existing external Ray cluster.
This builder requires a ray_address to be set. Does NOT have memory(), cpus(), or worker configuration methods.
Example:
cluster = (
ExternalRayClusterBuilder.create("remote")
.ray_address("ray://10.0.0.1:10001")
.build()
)
ray_address(addr: str) -> ExternalRayClusterBuilder
Set the Ray address (required). e.g., 'ray://host:port'
ray_init_kwargs(kwargs: dict) -> ExternalRayClusterBuilder
Set kwargs passed to ray.init() when connecting (e.g. env_vars). For example:
.ray_init_kwargs({
"runtime_env": {
"env_vars": {
"MY_VAR": "value",
"AWS_ACCESS_KEY_ID": os.environ["AWS_ACCESS_KEY_ID"]
},
},
})
create(
name: str, ray_address: str | None = None
) -> ExternalRayClusterBuilder
Create a new builder with the given cluster name and optional ray_address.
Bases: _WorkerResourceMixin
Builder for CPU-only worker groups in KubeRay clusters.
This builder does NOT have a gpus() method - use GpuWorkerBuilder for GPU workers.
Example:
worker = (
CpuWorkerBuilder()
.cpus(8)
.memory("16Gi")
.replicas(2)
.build()
)
name(name: str) -> CpuWorkerBuilder
Set the worker group name. Must be unique within the cluster.
Set the node selector for pod placement.
Set the minimum number of replicas for autoscaling.
Set the maximum number of replicas for autoscaling.
Set the idle timeout in seconds for autoscaling down workers.
Bases: _WorkerResourceMixin
Builder for GPU worker groups in KubeRay clusters.
Includes validation:
- Minimum 4GiB memory for GPU workers (raises ValueError)
- Warning for memory > 100GB (may exceed node capacity)
- Warning for high memory/CPU ratio (> 16 GiB/CPU)
Example:
worker = (
GpuWorkerBuilder()
.gpus(4)
.cpus(8)
.memory("64Gi")
.build()
)
name(name: str) -> GpuWorkerBuilder
Set the worker group name. Must be unique within the cluster.
Set the node selector for pod placement.
Set the minimum number of replicas for autoscaling.
Set the maximum number of replicas for autoscaling.
Set the idle timeout in seconds for autoscaling down workers.
Bases: RayGroupConfig
Configuration for Ray Worker pods
Bases: BaseManager
Bases: Enum
Method for retrieving kubernetes config:
- LOCAL: Load the kube config from the local environment.
- EKS_AUTH: Load the kube config from AWS EKS service (requires AWS credentials).
- IN_CLUSTER: Load the kube config when running inside a pod in the cluster.