The Hype Market

• Bill Hunt

It takes time, money, and other resources to execute on any new information technology initiative. In theory, there should be a return on investment for any new IT development. However, in the current market landscape, a lot of effort is being wasted on reinventing the wheel or misapplying solutions, rather than driving towards proving greater value to citizens and customers. The relatively low cost of experimentation and implementation of processes result in a fetishization of abstraction, leading to further and further complexity even if a proportional benefit is not achieved.

Technology is a Bridge

Most businesses and government services aim to connect a person’s intention to an outcome. Whether we’re helping someone call a restaurant to make a reservation, or helping a Veteran to understand their benefits, we’re aiding someone to solve a problem. In short, we’re bridging the gap between a customer and a service.

Technology is introduced into the process of delivering a service to a customer to automate one or more steps, it effectively shortens the gap. In general, technology is not seamless in this process – that is, there are almost always steps between the customer’s intent and the resulting action that have to be filled through human intervention or additional services. You can’t think “pizza” and have a pizza instantly appear  – but an app to order pizza removes some of the burden of calling the pizza place and having them take your order, prepare it, and deliver it to your doorstep.

Today there are new automation tools available to practitioners who want to create access to services through technology. If you want to create a service so that someone is able to order a pizza, you can set up a website or a mobile app to do so, because there’s already computers, and the Internet, and web browsers on the computers, and protocols for everything to talk to each other. So for anyone trying to solve a problem, the real value is in making the process simpler or easier through further automation – say, having your website be able to actually show the order automatically at the pizza place on a screen, or automatically send you a text when the pizza is out for delivery with an ETA. Based on the previous diagram, the goal is to expand the boundaries of technology to further fill the gaps between the customer and the service.

However, that gap is not where the technology industry focus most of its time and money. Software engineers are spending more and more of our resources mucking about with tools that are simply thinner and thinner slices of micromanagement around already-solved problems. In practice, the industry is just creating additional layers that are largely unnecessary for the end customer.

Here’s a real world example, again using a website.  In 1998, it took a few hours to set up a website. Most of our time was spent on designing it, then hand-coding HTML to post on a webserver.  As our tools evolved, the time to perform these steps decreased, but the complexity of the tools themselves has increased dramatically. Today, it can take ten times as long to create a basic website as it did decades ago.

After twenty years, technology has moved a little closer to the person requesting the information due to improved User Experience, and the end user can get to the information a little faster and easier, but mostly there’s lots of additional complexity in the middle layers. These middle layers have created a very rich market of expensive services and skills based on these bleeding-edge technologies.

Blockchain: A Parable

As a result, the world has seen the rise of numerous solutions with overblown promises of impact, fueled by speculation of venture capital. Blockchain (or distributed ledgers) is a popular example, which has elevated itself with an innate brand of “value” due to being associated with Bitcoin, a popular cryptocurrency using the technology.

Most computer systems take data from one person, put it into a storage location (usually a database), and then output it for someone else’s review. One problem that people have with these systems is that there’s no way to validate that the data submitted by the first person is the same as what the person on the other end receives.

There are numerous points along that path where a malicious actor – or broken process – can modify or corrupt the data. That can be a hacker exploiting a vulnerable system, data corruption due to failed hardware, or any number of other failures. The submitting person could also just be lying about the data, or the person receiving it could lie about the results.

Companies selling blockchain as a solution make the promise that these points of failure can be eliminated by having an authoritative record – or ledger – of all changes to the data. However, in reality, blockchain fulfills only a small piece of this process.

In most cases, it simply replaces the database with another type of database, which provides additional checks. This still doesn’t solve the original problem of ensuring that the data is valid on both ends.

To visualize this, let’s say I have four apples.  I write down on paper that I have six apples. You read the paper and double check that I do in fact say that I have six apples. You have no proof I ever had six apples, only the promise that I did.  You could then go report that I have eight apples and that you double-checked that fact and you promise that it’s true. This is how blockchain gives a false sense of security.

Addressing the Market Forces

And still the sales pitch still works!  Companies are spending millions to adopt “blockchain” technologies – many of which are so stripped of features that they can’t even be called a distributed ledger anymore. But even if purchasers don’t know the difference, technologists must know that much of this additional complexity is merely hype. Why would software engineers want to use it in the first place?

One way to consider this is that engineers obsess on generating higher and higher orders of abstraction. Abstraction, however, generally increases complexity by moving concerns to a higher order function.  Think of going from a wheelbarrow to a bicycle. Bicycles can reduce the amount of energy needed to move a load around due to gears – but those gears are also more complex than just an axle and wheels. Suddenly you have extra parts to keep up and grease, and to build them from scratch you’d need to understand the math behind gearing ratios.

With physical processes, resource limitations and cost-effectiveness become natural restrictions on automation. But in software, there are rarely such restrictions; rather, software developers are actively encouraged to create more complex tools, often as “side projects” outside of their main working hours – though frequently eschewing any hope of a healthy work-life balance. These developers and tools are given prominence on stage at most technology conferences, becoming superstars of the technology world and further adding to the hype. Very few people garner attention for solving common problems with common tools. (E.g., there are four major competing Javascript bundlers today, while make, written in 1976, can do just about everything.)

To look at this cynically, extra complexity means higher cost.  Technology as a business is largely about hype, the newest and coolest tools are a selling point for most tech-reliant industries (which is most industries). The rise of outsourcing overseas, to areas where costs are much lower, continues to drive down the cost of technology development in the west. But using all of these rapidly-evolving technologies increases the cost to develop, so engineers can demand higher salaries.  It’s only by continuing to evolve the market of solutions to newer and trendier alternatives that the system can maintain its value, otherwise everyone would all settle on a low-bar minimum baseline and scaffold from there – like using dumb “feature” phones with mail and chat features, instead of smart phones. But that wouldn’t allow anyone to maintain the higher prices.

These solutions are, as a result, extremely short-lived. Newer model iPhones come out annually.  And a new major Javascript framework establishes market dominance on the web every 18 to 36 months or so. Although at the time of writing we are in the height of React’s reign, we’ve already seen that market splinter and fragment, adding new layers within React, including Redux, Sagas, and others. Every piece of software released today is the legacy technology of tomorrow.

This market churn makes it difficult for any new engineers to enter the workforce, as any skills learned in a developed curriculum will largely be out of date by the time the student graduates. Only full-time software engineers are able to compete, and they must dedicate a considerable part of their time to staying current, lest they fall behind and become unemployable.  Though, this will eventually give way to a secondary market of expensive engineers specializing in legacy systems, as seen during the Y2K panic. The Federal Government also continues to provide a welcome market for outdated technology skills.

Moving Past Hype Through Accountability & Simplicity

The hype cycle and automation cult isn’t going away anytime soon.  But we all – government and private sector alike – can work to be more cynical about the technologies presented to us, to dive deep on the topics, and look to experts to inform our opinions.  We can demand proven, reliable solutions with longevity instead of just the newest buzzwords. And we must plan for the inevitable economic downturns in the future that will impact the IT market, like we saw in the “dot com” crash of the late 1990s.  There will always be money to be had in IT, but the bubble must burst (again) eventually.

Another thing we can do to combat this trend is to stand up for simplification; using the easiest – and most well known – solution will often be the most cost-effective.  This will inevitably erode the market, driving down costs and associated salaries, and reduce technology practitioners to skilled tradespeople like plumbers or electricians.  In this way, we can plan for a more stable future technology market, with far greater longevity.

Through this dedicated effort and planning, we can focus our efforts on those areas of greater value instead, creating a more seamless Customer Experience which actually solves problems for people more efficiently and cheaply, instead of just making increasingly impressive profit margins for private companies.