查看: 243|回复: 0

Apache Kafka

[复制链接]
论坛徽章:
21
投资理财徽章
日期:2015-04-02 14:37:21Hadoop研习者初级
日期:2017-01-18 16:14:39机器学习徽章
日期:2017-07-27 17:25:09机器学习徽章
日期:2017-08-10 15:34:18python徽章
日期:2018-06-07 16:21:43python徽章
日期:2018-06-21 17:15:38人工智能徽章
日期:2018-07-19 15:00:17OpenAI课程徽章
日期:2018-08-16 15:34:25Kaggle徽章
日期:2019-05-16 15:54:26Hadoop研习者中级
日期:2017-01-06 10:48:29Hadoop研习者初级
日期:2016-12-29 16:44:02Openstack徽章
日期:2016-11-17 15:59:19
发表于 2019-4-13 19:21 | 显示全部楼层 |阅读模式
Apache Kafka® is a distributed streaming platform. What exactly does that mean?
A streaming platform has three key capabilities:
  • Publish and subscribe to Streams of records, similar to a message queue or enterprise messaging system.
  • Store streams of records in a fault-tolerant durable way.
  • Process streams of records as they occur.
Kafka is generally used for two broad classes of applications:
  • Building real-time streaming data pipelines that reliably get data between systems or applications
  • Building real-time streaming applications that transform or react to the streams of data
To understand how Kafka does these things, let's dive in and explore Kafka's capabilities from the bottom up.
First a few concepts:
  • Kafka is run as a cluster on one or more servers that can span multiple datacenters.
  • The Kafka cluster stores streams of records in categories called topics.
  • Each record consists of a key, a value, and a timestamp.
Kafka has four core APIs:
  • The Producer API allows an application to publish a stream of records to one or more Kafka topics.
  • The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.
  • The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams.
  • The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.



回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册 新浪微博登陆

本版积分规则

 

GMT+8, 2019-10-15 04:55 , Processed in 0.165156 second(s), 30 queries .

关闭

扫一扫加入
本版微信群