Batch ^v2.7.0

Writes many rows to a database table in grouped statements instead of one at a time.

const batch = datasource.createBatch("Warehouse", "sales", ["year", "product", "amount"]);

batch.insert(["2026", "Widget", 1200]);
batch.insert(["2026", "Gadget", 950]);

batch.flush();

Constructor

datasource.createBatch datasource.createBatch(datasource, tableName, fields) Batch

Creates a batch that inserts rows, writing them 500 at a time. Use insert() to queue each row.

Parameter	Type	Description
datasource	string	The datasource's name or id. It must be a JDBC datasource.
tableName	string	The table to write to.
fields	array	The columns each row supplies values for, in order.

const batch = datasource.createBatch("Warehouse", "sales", ["year", "product", "amount"]);

datasource.createBatch datasource.createBatch(datasource, batchType, tableName, fields, updateFields, batchSize) Batch

Creates a batch with a choice of what happens to rows that already exist, and how many rows are written at a time.

Parameter	Type	Description
datasource	string	The datasource's name or id. It must be a JDBC datasource.
batchType	string	"INSERT", "INSERT_OR_UPDATE" or "INSERT_OR_IGNORE"; see the table below.
tableName	string	The table to write to.
fields	array	The columns each row supplies values for, in order.
updateFields	array	The columns to update when a row's key already exists, in "INSERT_OR_UPDATE" mode. Pass [] for the other modes.
batchSize	number	Optional. How many rows to queue before they're written automatically, from 1 to 10000. Defaults to 500.

Batch type	Behaviour
"INSERT"	Inserts each row.
"INSERT_OR_UPDATE"	Inserts each row, or updates the existing row's updateFields when its key already exists.
"INSERT_OR_IGNORE"	Inserts each row, or skips it when its key already exists.

// Insert rows, updating the name and price of products that already exist
const batch = datasource.createBatch("Warehouse", "INSERT_OR_UPDATE", "products", ["id", "name", "price"], ["name", "price"]);

// Insert rows, skipping ones that already exist, 1000 at a time
const batch = datasource.createBatch("Warehouse", "INSERT_OR_IGNORE", "products", ["id", "name", "price"], [], 1000);

Methods

insert insert(values)

Queues a row to insert. Writes the queue automatically when it reaches the batch size.

Parameter	Type	Description
values	array	The row's values, in the same order as fields.

flush flush()

Writes every queued row now. Call it at the end of a load; rows still queued when the process finishes aren't written.

truncate truncate()

Deletes every row in the table straight away. Use it before a full reload.

setAllowNulls setAllowNulls(allowNulls)

Sets whether null values are accepted. They're allowed by default. Turn it off to catch gaps in the source data: queuing a row with a null then throws an error.

Parameter	Type	Description
allowNulls	boolean	false to reject null values.

Examples

Insert rows with the default batch size

// Create an INSERT batch with the default batch size of 500
const batch = datasource.createBatch(
    "Internal Datastore",
    "test_table",
    ["field_1", "field_2", "field_3"]
);

// Clear every existing row in the table
batch.truncate();

// Queue rows; the batch writes them automatically each time it reaches the batch size
for (let i = 0; i < 15; i++) {
    batch.insert([
        "test_value_1", // Value for 'field_1'
        "test_value_2", // Value for 'field_2'
        "test_value_3", // Value for 'field_3'
    ]);
}

// Write the rows that didn't reach the batch size
batch.flush();

Update rows that already exist

// On a duplicate key, update "field_2" for that row; otherwise insert a new row
const batch = datasource.createBatch(
    "Internal Datastore",
    "INSERT_OR_UPDATE",
    "test_table",
    ["field_1", "field_2", "field_3"],
    ["field_2"]
);

Skip rows that already exist

// On a duplicate key, leave the existing row unchanged
const batch = datasource.createBatch(
    "Internal Datastore",
    "INSERT_OR_IGNORE",
    "test_table",
    ["field_1", "field_2", "field_3"],
    []
);

Load rows from a data process

let batch;

function begin() {
    batch = datasource.createBatch("Warehouse", "staging_sales", ["region", "month", "amount"]);
}

function data(record) {
    batch.insert([record.region, record.month, record.amount]);
}

function end() {
    batch.flush();
}

Related

• DatasourceClient: runs one-off queries and statements.