At the end of March, Betterment’s data team, aka Team Polaris, hosted about 70 data and R enthusiasts at our office space.  The topic for the event, Light-weight Dashboarding and Reporting Workflows with R was a hands-on, inside look at how Betterment uses Agile and R to automate data insights while empowering our data analysts to self-serve.

The house was packed—it was a meeting of the tribes from an assortment of other MeetUps including Big On Data¹, FinTech², NYC Open-Data³ along with Betterment working together.

Light-weight, agile processes supported by pragmatic automation are foundational to Betterment’s engineering culture.  When our data scientists needed a lift to produce visual and repeatable analysis, we introduced a frugal stack of cloud and open source technologies to make it happen.  

The workflow has proved so effective at Betterment, we decided to share the scaffolding with others in the community.data science meetup

Key points from the presentation

  1. Bootstrap.  Don’t introduce complexity into your environment until you really need it.  

  2. Leverage the skill set of your organization.  If your analysts are great with R, productionize an R workflow.

  3. Automate.  Pragmatic engineering can empower your analysts while supporting your process.

  4. Freemium Cloud, first.  IaaS providers like Amazon have a free tier to help you get started.  Try it before you buy it.

  5. Use Hosted Tools and Services.  There are powerful hosted tools and services out there, like Travis-CI, to help you automate your workflow.  Add them to your toolkit.

Team exercise

The presentation included a team exercise component so that folks could experience the workflow and technologies first-hand.  Here is Patrick Burns, our data analyst extraordinaire holding court with his team.

patrick data analyst

The goodies

All scaffolding code used to demo this workflow is available on GitHub: https://github.com/ygoldman/rwizflowy.  Slides from our Presentation are likewise available here.  And, if you’d like to set up the same or similar environment on your own, this walk-through document will surely come in handy.  Let us know how it goes.  

In fact, if your organization is interested in learning more about this workflow, Team Polaris just might accept an invitation to lunch.

¹Big On Data looks at the intersection of Business Intelligence and Big Data technologies and their impact on today’s data professional; Yuriy M Goldman has been running this meetup since 2008.

²FinTech puts on topical panels and networking events for Financial Technology professionals; this meetup is organized by Betterment’s Nick Gavronsky, also since 2008.

³NYC Open-Data hosts educational, hands-on workshops for folks interested in statistical analysis, R language, and data science.

Jon Mauney contributed to this blog post.

Interested in engineering at Betterment? Betterment is an engineering-driven company that has developed the most-trusted online financial advisor based on the principles of optimization and efficiency. You can learn more about engineering jobs and our culture here.

Determination of most trusted online financial advisor reflects Betterment LLC’s distinction of having the most customers in the industry, made in reliance on customer counts, self-reported pursuant to SEC rules, across all online-only registered investment advisors.