pyflink.datastream.connectors.kinesis.KinesisStreamsSink

class KinesisStreamsSink(j_kinesis_streams_sink)

A Kinesis Data Streams (KDS) Sink that performs async requests against a destination stream using the buffering protocol.

The sink internally uses a software.amazon.awssdk.services.kinesis.KinesisAsyncClient to communicate with the AWS endpoint.

The behaviour of the buffering may be specified by providing configuration during the sink build time.

• maxBatchSize: the maximum size of a batch of entries that may be sent to KDS

• maxInFlightRequests: the maximum number of in flight requests that may exist, if any more in

  flight requests need to be initiated once the maximum has been reached, then it will be blocked until some have completed

• maxBufferedRequests: the maximum number of elements held in the buffer, requests to add

  elements will be blocked while the number of elements in the buffer is at the maximum

• maxBatchSizeInBytes: the maximum size of a batch of entries that may be sent to KDS

  measured in bytes

• maxTimeInBufferMS: the maximum amount of time an entry is allowed to live in the buffer,

  if any element reaches this age, the entire buffer will be flushed immediately

• maxRecordSizeInBytes: the maximum size of a record the sink will accept into the buffer,

  a record of size larger than this will be rejected when passed to the sink

• failOnError: when an exception is encountered while persisting to Kinesis Data Streams,

  the job will fail immediately if failOnError is set

Methods

builder()
get_java_function()