Cluster
A Geneva Cluster represents the backend compute infrastructure for the execution environment.
created_at: datetime = field(
factory=lambda: now(utc),
metadata={"pa_type": timestamp("us", tz="UTC")},
)
Convert the persisted cluster definition into internal RayCluster model
Fluent builder for GenevaCluster. name is required, all optional
fields will use defaults.
example usage:
GenevaClusterBuilder
.name("my-cluster")
.head_group_builder(HeadGroupBuilder().cpus(8).memory("16Gi"))
.add_worker_group(WorkerGroupBuilder()
.gpu_worker(memory="8Gi", gpus=1))
.build()
cluster_type(
cluster_type: GenevaClusterType,
) -> GenevaClusterBuilder
Set the cluster type. This can be one of:
- GenevaClusterType.KUBE_RAY (default): Launches a Ray cluster in Kubernetes using KubeRay.
- GenevaClusterType.LOCAL_RAY: Starts a local Ray cluster in the current process.
- GenevaClusterType.EXTERNAL_RAY: Connects to an existing Ray cluster.
ray_addressmust be provided for this option.
config_method(
method: K8sConfigMethod,
) -> GenevaClusterBuilder
Set the Kubernetes config method.
portforwarding(
enabled: bool = True,
) -> GenevaClusterBuilder
Enable or disable port forwarding.
aws_config(
region: str | None = None, role_name: str | None = None
) -> GenevaClusterBuilder
Configure AWS settings.
ray_address(addr: str) -> GenevaClusterBuilder
Set the Ray address for external Ray clusters.
i.e. ray://{ray_ip}:{ray_port}
This must be provided when using cluster type EXTERNAL_RAY
ray_init_kwargs(kwargs: dict) -> GenevaClusterBuilder
Set arbitrary kwargs to pass to ray.init() when starting the cluster.
Commonly used for runtime_env configuration with conda or pip dependencies.
Example:
builder.ray_init_kwargs({
"runtime_env": {
"conda": {
"channels": ["conda-forge"],
"dependencies": [
"python=3.10", "ffmpeg<8", "torchvision=0.22.1"
]
},
"config": {"eager_install": True}
}
})
WARNING: This accepts arbitrary kwargs without validation allowing for injection of fault or potentially malicious configuration. Use with caution.
head_group(
*,
image: str | None = None,
cpus: int | None = None,
memory: str | None = None,
gpus: int | None = None,
service_account: str | None = None,
node_selector: dict[str, str] | None = None,
labels: dict[str, str] | None = None,
tolerations: list[dict[str, str]] | None = None,
) -> GenevaClusterBuilder
Configure the head group with optional parameters.
head_group_builder(
builder: HeadGroupBuilder,
) -> GenevaClusterBuilder
Configure the head group using a HeadGroupBuilder.
add_cpu_worker_group(
*,
image: str | None = None,
cpus: int = 4,
memory: str = "8Gi",
service_account: str | None = None,
node_selector: dict[str, str] | None = None,
labels: dict[str, str] | None = None,
tolerations: list[dict[str, str]] | None = None,
) -> GenevaClusterBuilder
Add a CPU worker group.
add_gpu_worker_group(
*,
image: str | None = None,
cpus: int = 4,
memory: str = "8Gi",
gpus: int = 1,
service_account: str | None = None,
node_selector: dict[str, str] | None = None,
labels: dict[str, str] | None = None,
tolerations: list[dict[str, str]] | None = None,
) -> GenevaClusterBuilder
Add a GPU worker group.
add_worker_group(
builder: WorkerGroupBuilder,
) -> GenevaClusterBuilder
Add a worker group using a WorkerGroupBuilder.
external_cluster(
name: str, ray_address: str
) -> GenevaClusterBuilder
Create a new builder configured for an external Ray cluster.
local_cpu_cluster(
name: str, namespace: str = "geneva"
) -> GenevaClusterBuilder
Create a builder for a local CPU-only cluster with good defaults.
gpu_cluster(
name: str, namespace: str = "geneva"
) -> GenevaClusterBuilder
Create a builder for a GPU cluster with good defaults.
Builder for HeadGroupConfig. This can be used to build the configuration for a Ray head group with reasonable defaults. See GenevaClusterBuilder for examples
service_account(service_account: str) -> HeadGroupBuilder
Set the Kubernetes service account.
node_selector(
node_selector: dict[str, str],
) -> HeadGroupBuilder
Set the node selector for pod placement.
tolerations(
tolerations: list[dict[str, str]],
) -> HeadGroupBuilder
Set the pod tolerations.
add_toleration(
key: str,
operator: str = "Equal",
value: str = "",
effect: str = "",
) -> HeadGroupBuilder
Add a single toleration.
cpu_head(
cpus: int = 2, memory: str = "4Gi"
) -> HeadGroupBuilder
Create a CPU-only head group.
Builder for WorkerGroupConfig. This can be used to build the configuration for a Ray worker group with reasonable defaults. See GenevaClusterBuilder for examples
min_replicas(min_replicas: int) -> WorkerGroupBuilder
Set the minimum number of replicas for autoscaling.
max_replicas(max_replicas: int) -> WorkerGroupBuilder
Set the maximum number of replicas for autoscaling.
service_account(service_account: str) -> WorkerGroupBuilder
Set the Kubernetes service account.
node_selector(
node_selector: dict[str, str],
) -> WorkerGroupBuilder
Set the node selector for pod placement.
tolerations(
tolerations: list[dict[str, str]],
) -> WorkerGroupBuilder
Set the pod tolerations.
add_toleration(
key: str,
operator: str = "Equal",
value: str = "",
effect: str = "",
) -> WorkerGroupBuilder
Add a single toleration.
cpu_worker(
cpus: int = 4, memory: str = "8Gi"
) -> WorkerGroupBuilder
Create a CPU worker group.
gpu_worker(
cpus: int = 8, memory: str = "16Gi", gpus: int = 1
) -> WorkerGroupBuilder
Create a GPU worker group.
Bases: RayGroupConfig
Configuration for Ray Worker pods
Bases: BaseManager
Bases: Enum
Method for retrieving kubernetes config:
- LOCAL: Load the kube config from the local environment.
- EKS_AUTH: Load the kube config from AWS EKS service (requires AWS credentials).
- IN_CLUSTER: Load the kube config when running inside a pod in the cluster.