Lightflus provide Typescript API for all developers whose Runtime is implemented by Rust and powered by*(https://github.com/denoland/deno*) (A JS runtime, backed by V8 engine). However, for the purpose of performance, we turn to use Deno as an embedded module, not a runtime container. It makes Lightflus much different from most JavaScript Runtime like NodeJS.

To help you to write correct Lightflus code, it’s necessary for you to understand how Lightflus runtime works at first.

UNDERSTAND DATAFLOW OPERATOR CONTEXT

The context of a dataflow operator (like map, flatMap, etc) in Lightflus is consist of several components, like image below:

https://whimsical.com/dataflow-operator-context-R5zUSyEuTK8Lksf6Hf68UR

Each operator will has one or more v8 instances (or the runtime context of a JavaScript function). Every instance contains only one function’s context. It means if we define types or functions outside this function, it may not work normally.

Examples

Let’s see three examples to make this concept more clear.

1. Such code works normally:

// let dataflow: Dataflow<string> = ...

let sink = Redis.new<{ t0: number, t1: string }>()
    .host("localhost")
    .keyExtractor((v) => v.t1)
    .valueExtractor((v) => v.t0.toString());

dataflow.map(value=> value.split(" ").map(v => {
		// define object here is legal
    return { t0: 1, t1: v };
}))
// and this object will sink to redis
.sink(sink)
.execute();

1. And such code works too:

// let dataflow: Dataflow<string> = ...

// define interface here. 
// Because interface no need to intialize, it will be recogonized as object
interface Data {
	t0: number;
	t1: string;
}

// use interface as sink objective data structure
let sink = Redis.new<Data>()
    .host("localhost")
    .keyExtractor((v) => v.t1)
    .valueExtractor((v) => v.t0.toString());

dataflow.map(value=> value.split(" ").map(v => {
    return { t0: 1, t1: v };
}))
// and interface Data will sink to redis
.sink(sink)
.execute();

1. However, circumstance below will be out of your expectation:

// let dataflow: Dataflow<string> = ...

// define a class here
class Data {
	t0: number;
	t1: string;
}

// use class as sink objective data structure
let sink = Redis.new<Data>()
    .host("localhost")
    .keyExtractor((v) => v.t1)
    .valueExtractor((v) => v.t0.toString());

// but this code will not work as your expectation
dataflow.map(value=> value.split(" ").map(v => {
		// It looks like natural here, but it not works
		let data = new Data();
		data.t0 = 1;
		data.t1 = v;
    return data;
}))
// data never sink to redis
.sink(sink)
.execute();

// Then we rewrite the code to make it right
dataflow.map(value=> value.split(" ").map(v => {
		// we define the data here, 
		// so the class Data will be wrapped in the context of operator
		class Data {
			t0: number;
			t1: string;
		}
		
		let data = new Data();
		data.t0 = 1;
		data.t1 = v;
    return data;
}))
// data will sink to redis as you wish
.sink(sink)
.execute();

because the class Data is not contained in the context of map operator’s process function, the v8 instance initialized by Lightflus fails to link the class Data while compiling.