Summary Recently I have been looking into ways to test my Apache Beam pipelines at work. Most common use cases of Beam generally involves either batch reading data from GCS and writing to analytical platforms such as Big Query or stream reading data from Pubsub and writing to perhaps Bigtable. A pipelines consists of transforms and its generally easy to test them in isolation as a independent unit test per stage, however I am personally a big fan of “end-to-end” testing or “Integration testing” and this is where things can sometimes get tricky.

A heat map is a graphical representation of data where the individual values contained in a matrix are represented as colors. This article will attempt to explain the process or creating and using GHEAT-JAVA, which is a port of famous aspen based gheat and took great inspiration from Gheat.net Writing a service which would serve heat map tiles is a bit tricky,there are three major components involved The client part i.