Lightflus can support most event-stream sourcing like Kafka, EMQX and etc as its data source. It will consume each data and transform them into another type by the user-defined dataflow.

Kafka

This chapter will describe how to use Kafka Source in Lightflus.

Usage

Kafka source provides a builder to build the instance of Kafka source. The code below is the example how to use it:

// define the schema of data
class UserAction {
  userId: string;
  itemId: string;
  timestamp: string;
  action: number;
}

// define the source and the type of the source
let source = Kafka
    .builder()
    .brokers(["localhost:9092"])
    .topic("topic")
    .build<UserAction>(UserAction);

// define the source that with primitive type
let source = Kafka
    .builder()
    .brokers(["localhost:9092"])
    .topic("topic")
    .build<string>(undefined, type of "");

let source = Kafka
    .builder()
    .brokers(["localhost:9092"])
    .topic("topic")
    .build<number>(undefined, type of 1);