Overview

What is Lightflus?

• Lightflus is a Cloud-Native Dataflow framework designed for a common purpose;
• You can be free to maintain a complicated dataflow system with the help of Lightflus. Empower your business with a faster and resilient data infra;
• Lightflus can be integrated with mainstream cloud providers (AWS, GCP, Azure) very easy;

Features of Lightflus

• Provide Typescript DataStream API, more lightweight compared with Java and lower learning curve;
• Fault-Tolerance (In release version): we’re working on this important feature and will release it in next version. We use Checkpoint to implement Fault-Tolerance which can promote the scalability and make sure exactly-once schema;
• Implemented by Rust: Rust is a new programming language which keep good balance between memory security and performance. Thanks to Rust, we build Lightflus as a robust and high performance system.
• Powered by Deno: Deno is a Javascript runtime co-created by Ryan Dahl (He also creates Node.JS). We use Deno as an embedded runtime for UDF Operator.
• Source(Sink) Connector: We will support more Source (Sink) Connectors in the future. Even more, we can let users define their own connectors by themselves.
• Web UI (will release in next version): we will provide a web ui in next version.

Architecture

• Lightflus will fetch data from DataSource like Kafka;
• Lightflus process fetched data and sink to downstream storages;

Why You Should Try Lightflus?

1. Cloud-Native: Lightflus is born for cloud. It’s much more agile than Flink. Your team can manage streaming data and integrate with your cloud infrastructure more efficient;
2. Lower Cost: Mainstream streaming framework like Flink/Storm, you need to recruit a professional team to handle this. The price is very high (time-consuming, cost-expensive and management-risky). Lightflus can offer both big companies and small teams a resilient streaming data framework to help them only focus on the core business. Anyone in your team can be a dataflow engineer in one hour.