With the amount of free resources, books, online classes coding bootcamps available right now everyone can learn to code. However, there’s still a quality gap between coding and software engineering. Must there be one?
I wrote my first “Hello world” over twenty years ago – that’s the answer I give if someone asks me how long I’ve been a coder. For the past ten years, I’ve been enjoying a career that has me touch code almost every day – that’s the answer I give if asked how long I’ve been a professional coder.
How long have I been a software engineer? I’d say about five years. Hold on, these numbers don’t seem to jive! So what’s changed? Whom would I consider a software engineer, and whom “merely” a coder?
The definition of a software engineer
Coding is relatively easy. It’s not all assembly mnemonics on ridiculously constrained systems anymore. And if you’re using something as expressive and powerful as Ruby, it’s even easier.
You just pick up a ticket, find where you need to insert your code, you figure out the logic you need to put there, and boom – done. If you’re slightly more advanced, you make sure your code is pretty. Logically split into methods. Has decent specs that don’t only test the happy path. That’s what a good coder does.
A software engineer doesn’t think in methods and classes anymore, at least not primarily. In my experience, a software engineer thinks in flows. They see the thunderous, raging river of data and interaction roaring through the system first and foremost. They think about what they need to do in terms of diverting or altering this flow. The pretty code, logical methods and great specs come almost as an afterthought.
It’s turtles all the way down
People generally think a certain way about most interactions with reality. For lack of a better term, let’s call it the “top-down” perspective. If what my brain is working on is getting myself a cup of tea, it will first figure out the general steps: go to the kitchen, put the kettle on, prepare the cuppa, pour water, return to desk.
It won’t first figure out which cup to use first, as I stand zoned out at my desk; that zone-out will come later, as I stand in front of the cupboard. It won’t consider that we might be out of tea (or at least, out of the good stuff). It’s broad, reactive and error-prone. All told – very human in nature.
As the software engineer considers changes to the somewhat mind-boggling data flow, they’ll naturally do it in a similar way. Let’s consider this example user story:
A customer orders a widget. In pricing the order, the following must be taken into account:
- Widget base price in user’s locality
- Widget shape (price modifier)
- Whether it’s a rush order (price modifier)
- Whether order delivery takes place on a holiday in user’s locale (price modifier)
This all might seem contrived (and obviously, it is), but it’s not a far cry from some actual user stories I’ve had the pleasure of crushing recently.
Now, let’s go through the thought process that a software engineer might employ to tackle this:
“Well we need to get the user and their order. Then we start calculating the total. We’ll start at zero. Then we’ll apply the widget shape modifier. Then the rush charge. Then we see if it’s on a holiday, boom, done before lunch!”
Ah, the rush that a simple user story can bring. But the software engineer is only human, not a perfect multi-threaded machine, and the recipe above is broad strokes. The engineer continues thinking deeper then:
“The widget shape modifier is… oh, that’s super dependent on the widget, isn’t it. And they may be different per locale, even if not now then in the future,” they think, burned previously by changing business requirements, “and the rush charge might be as well. And holidays are super locale-specific as well, augh, and timezones will be involved! I had an article here about dealing with times in different timezones in Rails here… ooh, I wonder if the order time is stored with zone in the database! Better check the schema.”
All right, software engineer. Stop. You’re supposed to be making a cup of tea, but you’re zoned out in front of the cupboard thinking about whether the flowery cup is even applicable to your tea problem.
Brewing the perfect cup widget
But that’s easily what could happen when you’re trying to do something as unnatural to the human brain as thinking at several depths of detail simultaneously.
After a brief rummage through their spacious armory of links regarding timezone handling our engineer pulls themselves together and starts breaking this down into actual code. If they tried the naive approach, it might look something like this:
def calculate_price(user, order)
order.price = 0
order.price = WidgetPrices.find_by(widget_type: order.widget.type).price
order.price = WidgetShapes.find_by(widget_shape: order.widget.shape).modifier
...
end
And on and on they would go, in this delightfully procedural manner, only to be heavily shut down on the first code review. Because if you think about it, it’s perfectly normal to think this way: broad strokes first, and details much later. You didn’t even think you were out of the good tea at first, did you?
Our engineer, however, is well trained and no stranger to the Service Object, so here’s what starts happening instead:
class BaseOrderService
def self.call(user, order)
new(user, order).call
end
def initialize(user, order)
@user = user
@order = order
end
def call
puts "[WARN] Implement non-default call for #{self.class.name}!"
user, order
end
end
class WidgetPriceService < BaseOrderService; end
class ShapePriceModifier < BaseOrderService; end
class RushPriceModifier < BaseOrderService; end
class HolidayDeliveryPriceModifier < BaseOrderService; end
class OrderPriceCalculator < BaseOrderService
def call
user, order = WidgetPriceService.call(user, order)
user, order = ShapePriceModifier.call(user, order)
user, order = RushPriceModifier.call(user, order)
user, order = HolidayDeliveryPriceModifier.call(user, order)
user, order
end
end
```
Good! Now we can employ some good TDD, write a test case for it, and flesh out the classes until all the pieces fall into place. And it’ll be beautiful, too.
As well as completely impossible to reason about.
Enemy is the state
Sure, these are all well-separated objects with single responsibilities. But here’s the issue: they’re still objects. The service object pattern with its “forcibly pretend this object is a function” is really a crutch. There’s nothing preventing anyone from calling HolidayDeliveryPriceModifier.new(user, order).something_else_entirely
. Nothing’s preventing people from adding internal state to these objects.
Not to mention user
and order
are objects as well, and messing with them is as easy as someone sneaking in a quick order.save
somewhere in these otherwise “pure” functional-objects, changing the underlying source of truth’s, a.k.a. a database, state. In this contrived example it’s not a big deal, but it sure can come back to bite you if this system grows in complexity and expands into additional, often asynchronous, parts.
The engineer had the right idea. And used a very natural way of expressing this idea. But to know how to express this idea – in a beautiful and easy to reason about way – was quite nearly prevented by the underlying OOP paradigm. And if someone who hasn’t yet made the leap into expressing their thoughts as diversions of the data flow tries to less skillfully alter the underlying code, bad things will happen.
Becoming functionally pure
If only there was a paradigm where expressing your ideas in terms of data flows was not only easy, but necessary. If reasoning could be made simple, with no possibility to introduce unwanted side effects. If data could be immutable, just like the flowery cup you make your tea in.
Yes, I’m kidding around of course. That paradigm exists, and it’s called functional programming.
Let’s consider how the above example might look in a personal favorite, Elixir.
defmodule WidgetPrices do
def priceorder([user, order]) do
[user, order]
|> widgetprice
|> shapepricemodifier
|> rushpricemodifier
|> holidaypricemodifier
end
defp widgetprice([user, order]) do
%{widget: widget} = order
price = WidgetRepo.getbase_price(widget)
[user, %{order | price: price }]
end
defp shapepricemodifier([user, order]) do
%{widget: widget, price: currentprice} = order
modifier = WidgetRepo.getshapeprice(widget)
[user, %{order | price: currentprice * modifier} ]
end
defp rushpricemodifier([user, order]) do
%{rush: rush, price: currentprice} = order
if rush do
[user, %{order | price: currentprice * 1.75} ]
else
[user, %{order | price: current_price} ]
end
end
defp holidaypricemodifier([user, order]) do
%{date: date, price: currentprice} = order
modifier = HolidayRepo.getholidaymodifier(user, date)
[user, %{order | price: currentprice * modifier}]
end
end
```
You might note it’s a fully-fleshed example of how the user story can actually be achieved. That’s because it’s less of a mouthful than it would be in Ruby. We’re using a few key features unique to Elixir (but generally available in functional languages):
Pure functions. We don’t actually change the incoming order
at all, we’re just creating new copies – new iterations on the initial state. We don’t hop to the side to change anything, either. And even if we wanted to, order
is just a “dumb” map, we can’t call order.save
at any point here because it simply doesn’t know what that is.
Pattern matching. Rather similar to ES6’s destructuring, this allows us to pluck price
and widget
off of the order and pass it on, instead of forcing our buddies WidgetRepo
and HolidayRepo
to know how to deal with a full order
.
Pipe operator. Seen in price_order
, it lets us pass data through functions in a sort of “pipeline” – a concept immediately familiar to anyone who ever ran ps aux | grep postgres
to check whether the darn thing was still running.
This is how you think
Side effects are not really a basic part of our thought process. After pouring water into your cup you don’t generally worry that an error in the kettle might cause it to overheat and explode – at least not enough to go poking through it’s internals to check whether someone didn’t inadvertently leave explode_after_pouring
flipped high.
The road from coder to software engineer – going beyond worrying about objects and states, and into worrying about data flows – can in some cases take years. It sure did for OOP-raised yours truly. With functional languages, you get into thinking about flows on your first night.
We’ve made software engineering complicated for ourselves and every single newcomer into the field. Programming doesn’t need to be hard and brain-wracking. It can be easy and natural.
Let’s not make this complicated and go functional already. Because that’s how we think.
Read also: