Summary Recently I have been looking into ways to test my Apache Beam pipelines at work. Most common use cases of Beam generally involves either batch reading data from GCS and writing to analytical platforms such as Big Query or stream reading data from Pubsub and writing to perhaps Bigtable. A pipelines consists of transforms and its generally easy to test them in isolation as a independent unit test per stage, however I am personally a big fan of “end-to-end” testing or “Integration testing” and this is where things can sometimes get tricky.

Most folks have been using Grafana or something similar to monitor Kubernaties cluster. If you are using Google cloud container engine , then the standard unified place to monitor all your application and services is Slackdriver. According to googles documentation, when you create a new container cluster using GUI or gcloud, Slackdriver is automatically configured with a sink, however It did not work out of the box for me hence this post :)

In this blog post, I will write about using Google cloud storage as a Raster Tile Server for static imagery. In the GIS domain, various techniques are used to add custom raster overlay on top of a base maps, or to even use custom imagery, as a base map itself. This approach is also useful if you have a large quantity of satellite or aerial imagery, that you need to serve at scale onto a Google map or any other GIS tool.

Introduction Isochrones are curves of equal travel time from a certain point of origin, another way of saying this would be that an Isochrone is an isoline for travel time, if the weighting factor is changed form time to distance, then the resulting curve us called an Isodistance. In this post I will present a rough way to create an Isochrone and an Isodistance using google’s directions API. Algorithm The actual algorithm to calculate Isocrone or Isodistance is perhpas a bit complicated than the simplistic approach being described here, if you are intrested in having a deeper look then I would recommend visiting this link by OpenStreetMap folks.