Meet our writer
Senior Director of Engineering, Betterment
Katelyn is a Senior Director of Engineering at Betterment where she leads the cash management engineering team. She previously led the Betterment for Business engineering team. Outside of Betterment, Katelyn loves to travel and to spend time with her two dogs.
Articles by Katelyn Lesse
A Journey to Truly Safe HTML Rendering
We leverage Rubocop’s OutputSafety check to ensure we’re being diligent about safe HTML ...A Journey to Truly Safe HTML Rendering We leverage Rubocop’s OutputSafety check to ensure we’re being diligent about safe HTML rendering, so when we found vulnerabilities, we fixed them. As developers of financial software on the web, one of our biggest responsibilities is to keep our applications secure. One area we need to be conscious of is how we render HTML. If we don’t escape content properly, we could open ourselves and our customers up to security risks. We take this seriously at Betterment, so we use tools like Rubocop, the Ruby static analysis tool, to keep us on the right track. When we found that Rubocop’s OutputSafety check had some holes, we plugged them. What does it mean to escape content? Escaping content simply means replacing special characters with entities so that HTML understands to print those characters rather than act upon their special meanings. For example, the < character is escaped using <, the >character is escaped using >, and the & character is escaped using &. What could happen if we don’t escape content? We escape content primarily to avoid opening ourselves up to XSS (cross-site scripting) attacks. If we were to inject user-provided content onto a page without escaping it, we’d be vulnerable to executing malicious code in the user’s browser, allowing an attacker full control over a customer’s session.This resource is helpful to learn more about XSS. Rails makes escaping content easier Rails escapes content by default in some scenarios, including when tag helpers are used. In addition, Rails has a few methods that provide help in escaping content. safejoin escapes the content and returns a SafeBuffer (a String flagged as safe) containing it. On the other hand, some methods are just a means for us to mark content as already safe. For example, the <%==interpolation token renders content as is and raw, htmlsafe, and safe_concat simply return a SafeBuffer containing the original content as is, which poses a security risk. If content is inside a SafeBuffer, Rails won’t try to escape it upon rendering. Some examples: html_safe:  pry(main)> include ActionView::Helpers::OutputSafetyHelper => Object  pry(main)> result = “hi”.html_safe => “hi”  pry(main)> result.class => ActiveSupport::SafeBuffer raw:  pry(main)> result = raw(“hi”) => “hi”  pry(main)> result.class => ActiveSupport::SafeBuffer safe_concat:  pry(main)> include ActionView::Helpers::TextHelper => Object  pry(main)> buffer1 = “hi”.html_safe => “hi”  pry(main)> result = buffer1.safe_concat(“bye”) => “hibye”  pry(main)> result.class => ActiveSupport::SafeBuffer safe_join:  pry(main)> include ActionView::Helpers::OutputSafetyHelper => Object  pry(main)> result = safe_join([“hi”, “bye”]) => “<p>hi</p><p>bye</p>”  pry(main)> result.class => ActiveSupport::SafeBuffer => ActiveSupport::SafeBuffer Rubocop: we’re safe! As demonstrated, Rails provides some methods that mark content as safe without escaping it for us. Rubocop, a popular Ruby static analysis tool, provides a cop (which is what Rubocop calls a “check”) to alert us when we’re using these methods: Rails/OutputSafety. At Betterment, we explicitly enable this cop in our Rubocop configurations so if a developer wants to mark content as safe, they will need to explicitly disable the cop. This forces extra thought and extra conversation in code review to ensure that the usage is in fact safe. … Almost We were thrilled about the introduction of this cop — we had actually written custom cops prior to its introduction to protect us against using the methods that don’t escape content. However, we realized there were some issues with the opinions the cop held about some of these methods. The first of these issues was that the cop allowed usage of raw and htmlsafewhen the usages were wrapped in safejoin. The problem with this is that when raw or htmlsafe are used to mark content as already safe by putting it in a SafeBuffer as is, safejoin will not actually do anything additional to escape the content. This means that these usages of raw and html_safeshould still be violations. The second of these issues was that the cop prevented usages of raw and htmlsafe, but did not prevent usages of safeconcat. safeconcat has the same functionality as raw and htmlsafe — it simply marks the content safe as is by returning it in a SafeBuffer. Therefore, the cop should hold the same opinions about safe_concat as it does about the other two methods. So, we fixed it Rather than continue to use our custom cops, we decided to give back to the community and fix the issues we had found with the Rails/OutputSafety cop. We began with this pull request to patch the first issue — change the behavior of the cop to recognize raw and htmlsafe as violations regardless of being wrapped in safejoin. We found the Rubocop community to be welcoming, making only minor suggestions before merging our contribution. We followed up shortly after with a pull request to patch the second issue — change the behavior of the cop to disallow usages of safe_concat. This contribution was merged as well. Contributing to Rubocop was such a nice experience that when we later found that we’d like to add a configuration option to an unrelated cop, we felt great about opening a pull request to do so, which was merged as well. And here we are! Our engineering team here at Betterment takes security seriously. We leverage tools like Rubocop and Brakeman, a static analysis tool specifically focused on security, to make our software safe by default against many of the most common security errors, even for code we haven’t written yet. We now rely on Rubocop’s Rails/OutputSafety cop (instead of our custom cop) to help ensure that our team is making good decisions about escaping HTML content. Along the way, we were able to contribute back to a great community. This article is part of Engineering at Betterment. These articles are maintained by Betterment Holdings Inc. and they are not associated with Betterment, LLC or MTG, LLC. The content on this article is for informational and educational purposes only. © 2017–2019 Betterment Holdings Inc.
A Functional Approach to Penny-Precise Allocation
How we solved the problem allocating a sum of money proportionally across multiple ...A Functional Approach to Penny-Precise Allocation How we solved the problem allocating a sum of money proportionally across multiple buckets by leaning on functional programming. An easy trap to fall into as an object-oriented developer is to get too caught up in the idea that everything has to be an object. I work in Ruby, for example, where the first thing you learn is that everything is an object. Some problems, however, are better solved by taking a functional approach. For instance, at Betterment, we faced the challenge of allocating a sum of money proportionally across multiple buckets. In this post, I’ll share how we solved the problem by leaning on functional programming to allocate money precisely across proportional buckets. The Problem Proportional allocation comes up often throughout our codebase, but it’s easiest to explain using a fictional example: Suppose your paychecks are $1000 each, and you always allocate them to your different savings accounts as follows: College savings fund: $310 Buy a car fund: $350 Buy a house fund: $200 Safety net: $140 Now suppose you’re an awesome employee and received a bonus of $1234.56. You want to allocate your bonus proportionally in the same way you allocate your regular paychecks. How much money do you put in each account? You may be thinking, isn’t this a simple math problem? Let’s say it is. To get each amount, take the ratio of the contribution from your normal paycheck to the total of your normal paycheck, and multiply that by your bonus. So, your college savings fund would get: (310/1000)*1234.56 = 382.7136 We can do the same for your other three accounts, but you may have noticed a problem. We can’t split a penny into fractions, so we can’t give your college savings fund the exact proportional amount. More generally, how do we take an inflow of money and allocate it to weighted buckets in a fair, penny-precise way? The Mathematical Solution: Integer Allocation We chose to tackle the problem by working with integers instead of decimal numbers in order to avoid rounding. This is easy to do with money — we can just work in cents instead of dollars. Next, we settled on an algorithm which pays out buckets fairly, and guarantees that the total payments exactly sum to the desired payout. This algorithm is called the Largest Remainder Method. Multiply the inflow (or the payout in the example above) by each weight (where the weights are the integer amounts of the buckets, so the contributions to the ticket in our example above), and divide each of these products by the sum of the buckets, finding the integer quotient and integer remainder Find the number of pennies that will be left over to allocate by taking the inflow minus the total of the integer quotients Sort the remainders in descending order and allocate any leftover pennies to the buckets in this order The idea here is that the quotients represent the amounts we should give each bucket aside from the leftover pennies. Then we figure out which bucket deserves the leftover pennies. Let’s walk through this process for our example: Remember that we’re working in cents, so our inflow is 123456 and we need to allocate it across bucket weights of [31000, 35000, 20000, 14000]. We find each integer quotient and remainder by multiplying the inflow by the weight and dividing by the total weight. We took advantage of the divmod method in Ruby to grab the integer quotient and remainder in one shot, like so: buckets.map do |bucket| (inflow * bucket).divmod(total_bucket_weight) end This gives us 12345631000/100000, 12345635000/100000, 12345620000 /100000 and 12345614000/100000. The integer quotients with their respective remainders are [38271, 36000], [43209, 60000], [24691, 20000], [17283, 84000]. Next, we find the leftover pennies by taking the inflow minus the total of the integer quotients, which is 123456 — (38271 + 43209 + 24691 + 17283) = 2. Finally, we sort our buckets in descending remainder order (because the buckets with the highest remainders are most deserving of extra pennies) and allocate the leftover pennies we have in this order. It’s worth noting that in our case, we’re using Ruby’s sort_by method, which gives us a nondeterministic order in the case where remainders are equal. In this case, our fourth bucket and second bucket, respectively, are most deserving. Our final allocations are therefore [38271, 43210, 24691, 17284]. This means that your college savings fund gets $382.71, your car fund gets $432.10, your house fund gets $246.91, and your safety net gets $172.84. The Code Solution: Make It Functional Given we have to manage penny allocations between a person’s goals often throughout our codebase, the last thing we’d want is to have to bake penny-pushing logic throughout our domain logic. Therefore, we decided to extract our allocation code into a module function. Then, we took it even further. Our allocation code doesn’t need to care that we’re looking to allocate money, just that we’re looking to allocate integers. What we ended up with was a black box ‘Allocator’ module, with a public module function to which you could pass 2 arguments: an inflow, and an array of weightings. Our Ruby code looks like this. The takeaway The biggest lesson to learn from this experience is that, as an engineer, you should not be afraid to take a functional approach when it makes sense. In this case, we were able to extract a solution to a complicated problem and keep our OO domain-specific logic clean.
Three Things I Learned In My Engineering Internship