Say goodbye to the Lambda architecture
David Regalado, Founder @ Data Engineering Latam
In the Lambda Architecture, an immutable sequence of records is captured and fed into a batch system and a stream processing system in parallel. You implement your transformation logic twice, once in the batch and stream processing systems. You stitch together the results from both systems at query time to produce a complete answer.
The problem with the Lambda Architecture is that maintaining code that needs to produce the same result in two complex distributed systems is painful.
Apache Beam aims to solve this problem and David will share how that's working.
