- class lance.LanceOperation.Append(lance.LanceOperation.BaseOperation)
Append new rows to the dataset.
- fragments¶
The fragments that contain the new rows.
- Type:
list[FragmentMetadata]
Warning
This is an advanced API for distributed operations. To append to a dataset on a single machine, use
lance.write_dataset()
.Examples
To append new rows to a dataset, first use
lance.fragment.LanceFragment.create()
to create fragments. Then collect the fragment metadata into a list and pass it to this class. Finally, pass the operation to theLanceDataset.commit()
method to create the new dataset.>>> import lance >>> import pyarrow as pa >>> tab1 = pa.table({"a": [1, 2], "b": ["a", "b"]}) >>> dataset = lance.write_dataset(tab1, "example") >>> tab2 = pa.table({"a": [3, 4], "b": ["c", "d"]}) >>> fragment = lance.fragment.LanceFragment.create("example", tab2) >>> operation = lance.LanceOperation.Append([fragment]) >>> dataset = lance.LanceDataset.commit("example", operation, ... read_version=dataset.version) >>> dataset.to_table().to_pandas() a b 0 1 a 1 2 b 2 3 c 3 4 d
Public members¶
- Append(fragments: Iterable[FragmentMetadata])
Initialize self. See help(type(self)) for accurate signature.
- __repr__()
Return repr(self).
- fragments : Iterable[FragmentMetadata]