BigQuery Visualiser Shiny app now free and open sourced

A few weeks ago I tweeted a beta version of a BigQuery Visualiser Shiny app that was well received, and got some valuable feedback on how it could be improved, in particular from @felipehoffa - thanks Felipe!

Here is a screenshot of the app:

Motivation

The idea of the app is to enhance the standard BigQuery interface to include plots of the data you query.  It uses ggplot2, a popular R library; d3heatmaps, a d3 JavaScript library to display heatmaps; and timelyportfolio's listviewer, a nice library for viewing all the BigQuery meta data in a collapsible tree.   Other visualisations can be added fairly easily and will be done so over time, but if you have a request for something in particular you can raise an issue on the project's Github page.

I got into BigQuery once it started to receive exports from Google Analytics Premium. Since these exports carry unsampled raw data and include unique userIds, its a richer data source for analysis than the Google Analytics reporting API.

It also was a chance to create another Google API library called bigQueryR, the newest member of the googleAuthR family.  Using googleAuthR meant Shiny support, and also meant bigQueryR can be used alongside googleAnalyticsR and searchConsoleR under one shared login flow.  This is something exploited in this demo of RMarkdown, which pulls data from all three sources into a scheduled report.

Running your own BigQuery Visualiser

All set-up instructions are listed on the BigQuery Visualiser's Github project page

You can run the Shiny app locally on your computer within RStudio; within your own company intranet if its running Shiny Server; or publicly like the original app on shinyapps.io

Feedback

Please let me know what else could improve. 

I have a current pending issue on using JSON uploads for authentication that is waiting a bug update in httr, the underlying library.

In particular all the htmlwidgets() packages could be added - this wonderful R library creates an R to d3.js interface, which holds some of the nicest visualisations on the web.

In this first release, I favoured plots that could apply to as much different data sets as possible.  For your own use cases you can be more restrictive on what data is requested, and so maybe more ambitious in the plots.  If you want inspiration timelyportfolio (he who wrote the listviewer library) has a blog where he makes lots of htmlwidgets libraries.

Enjoy!  Hope its of use, let me know if you build something cool with it.

9 responses
Great tools! Very intuitive and useful!
Hi Mark. I think bigQueryR has a small bug which makes authentication redirection URL fail if the actual request is coming from a non-standard port number. Specifically, I tried my app on a server directory of the form http://example.com:3838/bq and the redirect url that was sent to Google was: http://example.com/bq/3838 which failed to redirect me after authentication. Any ideas how to fix?
Hi Alex, yes that's known behavior. You need to specify the port number when launching the app, which by default is 1221. This is due to google OAuth2 not allowing any old port. So don't use the RStudio launch app button, but shiny::runApp(port=1221)
Check out the setup guide on github https://github.com/MarkEdmondson1234/BigQuery-V... for a bit more info. :)
Appreciate the new app. Looking forward to using it!
4 visitors upvoted this post.