apache apex与apache storm有何不同?

4dbbbstv  于 2021-06-21  发布在  Storm
关注(0)|答案(2)|浏览(398)

apacheapex看起来类似于apachestorm。
用户在这两个平台上以有向无环图(dag)的形式构建应用程序/拓扑。apex使用操作员/溪流,storm使用喷口/溪流/螺栓。
它们都是实时处理数据,而不是批处理。
两者似乎都具有高吞吐量和低延迟
所以,乍一看,两者看起来很相似,我不太明白其中的区别。有人能解释一下关键的区别吗?换句话说,我什么时候应该用一个而不是另一个?

t9aqgxwy

t9aqgxwy1#

架构上存在根本的差异,这使得每个平台在延迟、扩展和状态管理方面都非常不同。
在最基本的层面上,
apachestorm使用记录确认来保证消息传递。
apacheapex使用检查点来保证消息传递。
你可以在下面的博客中了解更多的不同之处,其中还包括其他主流处理平台。
https://databaseline.wordpress.com/2016/03/12/an-overview-of-apache-streaming-technologies/

3pvhb19x

3pvhb19x2#

体系结构和特点

  1. +-------------------+---------------------------+---------------------+
  2. | | Storm | Apex |
  3. +-------------------+---------------------------+---------------------+
  4. | Model | Native Streaming | Native Streaming |
  5. | | Micro batch (Trident | |
  6. +-------------------+---------------------------+---------------------+
  7. | Language | Java. | Java (Scala) |
  8. | | Ability to use non | |
  9. | | JVM languages support | |
  10. +-------------------+---------------------------+---------------------+
  11. | API | Compositional | Compositional (DAG) |
  12. | | Declarative (Trident) | Declarative |
  13. | | Limited SQL | |
  14. | | support (Trident) | |
  15. +-------------------+---------------------------+---------------------+
  16. | Locality | Data Locality | Advance Processing |
  17. +-------------------+---------------------------+---------------------+
  18. | Latency | Low | Very Low |
  19. | | High (Trident) | |
  20. +-------------------+---------------------------+---------------------+
  21. | Throughput | Limited in Ack mode | Very high |
  22. +-------------------+---------------------------+---------------------+
  23. | Scalibility | Limited due to Ack | Horizontal |
  24. +-------------------+---------------------------+---------------------+
  25. | Partitioning | Standard | Advance |
  26. | | Set parallelism at work, | Parallel pipes, |
  27. | | executor and task level | unifiers |
  28. +-------------------+---------------------------+---------------------+
  29. | Connector Library | Limited (certification) | Rich library of |
  30. | | | connectors in |
  31. | | | Apex Malhar |
  32. +-------------------+---------------------------+---------------------+

可操作性

  1. +------------+--------------------------+---------------------+
  2. | | Storm | Apex |
  3. +------------+--------------------------+---------------------+
  4. | State | External store | Checkpointing |
  5. | Management | Limited checkpointing | Local checkpointing |
  6. | | Difficult to exploit | |
  7. | | local state | |
  8. +------------+--------------------------+---------------------+
  9. | Recovery | Cumbersome API to | Incremental |
  10. | | store and retrieve state | (buffer server) |
  11. | | Require user code | |
  12. +------------+--------------------------+---------------------+
  13. | Processing | At least once | |
  14. | Semantic | Exactly once require | At least once |
  15. | | user code and affect | End to end |
  16. | | latency | |
  17. | | | exactly once |
  18. +------------+--------------------------+---------------------+
  19. | Back | Watermark on queue | Automatic |
  20. | Pressure | size for spout and bolt | Buffer server |
  21. | | Does not scale | memory and disk |
  22. +------------+--------------------------+---------------------+
  23. | Elasticity | Through CLI only | Yes w/ full user |
  24. | | | control |
  25. +------------+--------------------------+---------------------+
  26. | Dynamic | No | Yes |
  27. | topology | | |
  28. +------------+--------------------------+---------------------+
  29. | Security | Kerberos | Kerberos, RBAC, |
  30. | | | LDAP |
  31. +------------+--------------------------+---------------------+
  32. | Multi | Mesos, RAS - memory, | YARN |
  33. | Tenancy | CPU, YARN | full isolation |
  34. +------------+--------------------------+---------------------+
  35. | DevOps | REST API | REST API |
  36. | Tools | Basic UI | DataTorrent RTS |
  37. +------------+--------------------------+---------------------+

来源:网络研讨会:ApacheApex(下一代hadoop)vs.storm-比较和迁移概述https://www.youtube.com/watch?v=spjyo2hfd_i

展开查看全部

相关问题