lance.LanceFragment.create - Lance documentation

static lance.LanceFragment.create(dataset_uri: str | Path, data: ReaderLike, fragment_id: int | None = None, schema: pa.Schema | None = None, max_rows_per_group: int = 1024, progress: FragmentWriteProgress | None = None, mode: str = 'append', *, data_storage_version: str | None = None, use_legacy_format: bool | None = None, storage_options: dict[str, str] | None = None) → FragmentMetadata

Create a FragmentMetadata from the given data.

This can be used if the dataset is not yet created.

Warning

Internal API. This method is not intended to be used by end users.

Parameters:

dataset_uri : str¶: The URI of the dataset.
fragment_id : int¶: The ID of the fragment.
data : pa.Table or pa.RecordBatchReader¶: The data to be written to the fragment.
schema : pa.Schema, optional¶: The schema of the data. If not specified, the schema will be inferred from the data.
max_rows_per_group : int, default 1024¶: The maximum number of rows per group in the data file.
progress : FragmentWriteProgress, optional¶: Experimental API. Progress tracking for writing the fragment. Pass a custom class that defines hooks to be called when each fragment is starting to write and finishing writing.
mode : str, default "append"¶: The write mode. If “append” is specified, the data will be checked against the existing dataset’s schema. Otherwise, pass “create” or “overwrite” to assign new field ids to the schema.
data_storage_version : optional, str, default None¶: The version of the data storage format to use. Newer versions are more efficient but require newer versions of lance to read. The default (None) will use the latest stable version. See the user guide for more details.
use_legacy_format : bool, default None¶: Deprecated parameter. Use data_storage_version instead.
storage_options : optional, dict¶: Extra options that make sense for a particular storage connection. This is used to store connection parameters like credentials, endpoint, etc.