Incremental, Iterative Data Processing with Timely Dataflow.
We describe the timely dataflow model for distributed computation and its implementation in the Naiad system. The model supports stateful iterative and incremental computations. It enables both low-latency stream processing and high-throughput batch processing, using a new approach to coordination t...
| Publicado en: | Communications of the ACM Vol. 59; no. 10; pp. 75 - 84 |
|---|---|
| Autores principales: | , , , , , |
| Formato: | Artículo |
| Publicado: |
Association for Computing Machinery
Oct2016
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=118436922&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 118436922 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 00010782 ACM jtl: Communications of the ACM issn: 00010782 maglogo: N pubinfo: dt: Oct2016 vid: 59 iid: 10 pid: 68 pub: Association for Computing Machinery artinfo: ui: 118436922 10.1145/2983551 ppf: 75 ppct: 9 formats: tig: atl: Incremental, Iterative Data Processing with Timely Dataflow. aug: au: Murray, Derek G. McSherry, Frank Isard, Michael Isaacs, Rebecca Barham, Paul Abadi, Martín su: Data flow computing Distributed computing Computer systems Batch processing Electronic data processing sug: subj: Data flow computing Distributed computing Computer systems Batch processing Electronic data processing ab: We describe the timely dataflow model for distributed computation and its implementation in the Naiad system. The model supports stateful iterative and incremental computations. It enables both low-latency stream processing and high-throughput batch processing, using a new approach to coordination that combines asynchronous and fine-grained synchronous execution. We describe two of the programming frameworks built on Naiad: GraphLINQ for parallel graph processing, and differential dataflow for nested iterative and incremental computations. We show that a generalpurpose system can achieve performance that matches, and sometimes exceeds, that of specialized systems. pubtype: Periodical doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y dt: @attributes: year: 2016 holdings: @attributes: islocal: N |
|---|