posts

The Secret Equation of Job Satisfaction

2018.12.16 – Many lengthy books and articles have been written on how to be fulfilled at work, good management, and keeping your staff engaged and excited. However, I’ve found one simple equation that is the secret to all of these. Satisfaction = Volition / Friction If you’re not satisfied with your job, it’s probably because you don’t feel effective or that your work has the desired outcome – your volition, or because there’s too much resistance, bureaucracy, barriers, and day-to-day minutiae for you to enjoy the work – the friction.  Satisfaction is a measure of your volition over the job’s friction. More volition increases satisfaction, as does less friction. Let’s say that your control over your job is around 5 on some magical undefined scale – you can do the things you need to some of the time. However, the amount of paperwork you have to fill out to do the job is around a 10.  5/10 = ½, so you’re probably not going to be very satisfied. On the other hand, if you have lots of control and can make all the decisions you need – let’s call this 10 – and you have no daily design-by-committee meetings to wrangle – let’s give this one a 2, you’re at 10/2 = 5, which is looking pretty good. You don’t really need the scale or the points here, it’s just to show how the two relate to each other. The ratio of volition to friction will determine your level of job satisfaction. Research suggests that people feel most fulfilled when they are being challenged just beyond their current capacity. Too little challenge or too much challenge become either boring or overwhelming. Although being able to direct your work is the most obvious component of volition, it can include many supplementary factors. The alignment of your skills and background to your work can impact your volition. Feeling that your work has an impact on the world can also be a major element. Friction is comprised of several components as well. If your organization has unnecessary processes and procedures, those barriers to productivity will cause friction. If you work more than 40 hours per week or your commute is long that will likely cause friction by cutting into personal time. If your organization requires extensive reviews and buy-in from multiple stakeholders to accomplish tasks that may cause friction. Anything that causes you to become quickly exhausted at your job is likely adding to the friction. To increase your job satisfaction, you’ll need to increase your volition in the job or decrease your friction. To increase your volition you can take many steps in your current job.  This includes outsourcing or delegating unwanted tasks, taking on different projects that you enjoy more than your current projects, learn key information or skills, increase the level of challenge by taking on harder projects, decrease the level of challenge by collaborating with experts.  To dramatically increase your volition you may need to find a new job. To decrease the level of friction there are many things to do as well.  You can outsource and delegate tasks to decrease your weekly hours. You can work from home several days to decrease your weekly commute time.  You can decrease the number of weekly meetings with your teams. Any of these can make the job feel easier even if the workload hasn’t decreased. This isn’t to say that there aren’t terrible jobs in the world that no one would ever enjoy. You may be in a job where neither of these variables can be changed enough to make a major impact on your happiness. It’s a good idea to regularly assess where you’re at on this scale just to make sure your satisfaction isn’t slipping away. If you’re familiar with Agile methodologies, this should sound somewhat familiar. “Individuals and interactions over processes and tools” is a core tenet of the Agile approach. Putting the individual in control and reducing the cruft of processes is just good practice. As a manager, I often use this equation to help increase my staff’s satisfaction as well. I may not be able to always give a staff member more control over their work, but I can drop half of our check-in meetings to free up their time, or find ways to reduce the amount of paperwork they have to file to do their job. However, in many cases I can do both at once – by allowing staff to act independently in their projects with less oversight. This brings up another equation that I’m fond of: Trust + Autonomy = Delivery In general, if you hire good, talented people, all you need to do is trust _them to do their job and give them the _autonomy they need to do it, and they’ll deliver good work. (Buckminster Fuller talks about this in Operating Manual for Spaceship Earth, in describing the “Great Pirates.”) Giving your people the space they need to execute and support they need to see their work through will build trust and autonomy. Best of all, trust can increase volition, and autonomy can decrease friction, resulting in delivery and satisfaction. Each one of these components can be broken down into various topics for in depth study.  The high level equation helps identify what the missing component is to getting the desired results.  I hope you’ll keep this equation in mind when you’re approaching your own work, and see if you can find ways of creating a better balance.

Read This

The Hype Market

2018.10.24 It takes time, money, and other resources to execute on any new information technology initiative. In theory, there should be a return on investment for any new IT development. However, in the current market landscape, a lot of effort is being wasted on reinventing the wheel or misapplying solutions, rather than driving towards proving greater value to citizens and customers. The relatively low cost of experimentation and implementation of processes result in a fetishization of abstraction, leading to further and further complexity even if a proportional benefit is not achieved.

Technology is a Bridge

Most businesses and government services aim to connect a person’s intention to an outcome. Whether we’re helping someone call a restaurant to make a reservation, or helping a Veteran to understand their benefits, we’re aiding someone to solve a problem. In short, we’re bridging the gap between a customer and a service. Technology is introduced into the process of delivering a service to a customer to automate one or more steps, it effectively shortens the gap. In general, technology is not seamless in this process – that is, there are almost always steps between the customer’s intent and the resulting action that have to be filled through human intervention or additional services. You can’t think “pizza” and have a pizza instantly appear  – but an app to order pizza removes some of the burden of calling the pizza place and having them take your order, prepare it, and deliver it to your doorstep. Today there are new automation tools available to practitioners who want to create access to services through technology. If you want to create a service so that someone is able to order a pizza, you can set up a website or a mobile app to do so, because there’s already computers, and the Internet, and web browsers on the computers, and protocols for everything to talk to each other. So for anyone trying to solve a problem, the real value is in making the process simpler or easier through further automation – say, having your website be able to actually show the order automatically at the pizza place on a screen, or automatically send you a text when the pizza is out for delivery with an ETA. Based on the previous diagram, the goal is to expand the boundaries of technology to further fill the gaps between the customer and the service. However, that gap is not where the technology industry focus most of its time and money. Software engineers are spending more and more of our resources mucking about with tools that are simply thinner and thinner slices of micromanagement around already-solved problems. In practice, the industry is just creating additional layers that are largely unnecessary for the end customer. Here’s a real world example, again using a website.  In 1998, it took a few hours to set up a website. Most of our time was spent on designing it, then hand-coding HTML to post on a webserver.  As our tools evolved, the time to perform these steps decreased, but the complexity of the tools themselves has increased dramatically. Today, it can take ten times as long to create a basic website as it did decades ago. After twenty years, technology has moved a little closer to the person requesting the information due to improved User Experience, and the end user can get to the information a little faster and easier, but mostly there’s lots of additional complexity in the middle layers. These middle layers have created a very rich market of expensive services and skills based on these bleeding-edge technologies.

Blockchain: A Parable

As a result, the world has seen the rise of numerous solutions with overblown promises of impact, fueled by speculation of venture capital. Blockchain (or distributed ledgers) is a popular example, which has elevated itself with an innate brand of “value” due to being associated with Bitcoin, a popular cryptocurrency using the technology. Most computer systems take data from one person, put it into a storage location (usually a database), and then output it for someone else’s review. One problem that people have with these systems is that there’s no way to validate that the data submitted by the first person is the same as what the person on the other end receives. There are numerous points along that path where a malicious actor – or broken process – can modify or corrupt the data. That can be a hacker exploiting a vulnerable system, data corruption due to failed hardware, or any number of other failures. The submitting person could also just be lying about the data, or the person receiving it could lie about the results. Companies selling blockchain as a solution make the promise that these points of failure can be eliminated by having an authoritative record – or ledger – of all changes to the data. However, in reality, blockchain fulfills only a small piece of this process. In most cases, it simply replaces the database with another type of database, which provides additional checks. This still doesn’t solve the original problem of ensuring that the data is valid on both ends. To visualize this, let’s say I have four apples.  I write down on paper that I have six apples. You read the paper and double check that I do in fact say that I have six apples. You have no proof I ever had six apples, only the promise that I did.  You could then go report that I have eight apples and that you double-checked that fact and you promise that it’s true. This is how blockchain gives a false sense of security.

Addressing the Market Forces

And still the sales pitch still works!  Companies are spending millions to adopt “blockchain” technologies – many of which are so stripped of features that they can’t even be called a distributed ledger anymore. But even if purchasers don’t know the difference, technologists must know that much of this additional complexity is merely hype. Why would software engineers want to use it in the first place? One way to consider this is that engineers obsess on generating higher and higher orders of abstraction. Abstraction, however, generally increases complexity by moving concerns to a higher order function.  Think of going from a wheelbarrow to a bicycle. Bicycles can reduce the amount of energy needed to move a load around due to gears – but those gears are also more complex than just an axle and wheels. Suddenly you have extra parts to keep up and grease, and to build them from scratch you’d need to understand the math behind gearing ratios. With physical processes, resource limitations and cost-effectiveness become natural restrictions on automation. But in software, there are rarely such restrictions; rather, software developers are actively encouraged to create more complex tools, often as “side projects” outside of their main working hours – though frequently eschewing any hope of a healthy work-life balance. These developers and tools are given prominence on stage at most technology conferences, becoming superstars of the technology world and further adding to the hype. Very few people garner attention for solving common problems with common tools. (E.g., there are four major competing Javascript bundlers today, while make, written in 1976, can do just about everything.) To look at this cynically, extra complexity means higher cost.  Technology as a business is largely about hype, the newest and coolest tools are a selling point for most tech-reliant industries (which is most industries). The rise of outsourcing overseas, to areas where costs are much lower, continues to drive down the cost of technology development in the west. But using all of these rapidly-evolving technologies increases the cost to develop, so engineers can demand higher salaries.  It’s only by continuing to evolve the market of solutions to newer and trendier alternatives that the system can maintain its value, otherwise everyone would all settle on a low-bar minimum baseline and scaffold from there – like using dumb “feature” phones with mail and chat features, instead of smart phones. But that wouldn’t allow anyone to maintain the higher prices. These solutions are, as a result, extremely short-lived. Newer model iPhones come out annually.  And a new major Javascript framework establishes market dominance on the web every 18 to 36 months or so. Although at the time of writing we are in the height of React’s reign, we’ve already seen that market splinter and fragment, adding new layers within React, including Redux, Sagas, and others. Every piece of software released today is the legacy technology of tomorrow. This market churn makes it difficult for any new engineers to enter the workforce, as any skills learned in a developed curriculum will largely be out of date by the time the student graduates. Only full-time software engineers are able to compete, and they must dedicate a considerable part of their time to staying current, lest they fall behind and become unemployable.  Though, this will eventually give way to a secondary market of expensive engineers specializing in legacy systems, as seen during the Y2K panic. The Federal Government also continues to provide a welcome market for outdated technology skills.

Moving Past Hype Through Accountability & Simplicity

The hype cycle and automation cult isn’t going away anytime soon.  But we all – government and private sector alike – can work to be more cynical about the technologies presented to us, to dive deep on the topics, and look to experts to inform our opinions.  We can demand proven, reliable solutions with longevity instead of just the newest buzzwords. And we must plan for the inevitable economic downturns in the future that will impact the IT market, like we saw in the “dot com” crash of the late 1990s.  There will always be money to be had in IT, but the bubble must burst (again) eventually. Another thing we can do to combat this trend is to stand up for simplification; using the easiest – and most well known – solution will often be the most cost-effective.  This will inevitably erode the market, driving down costs and associated salaries, and reduce technology practitioners to skilled tradespeople like plumbers or electricians.  In this way, we can plan for a more stable future technology market, with far greater longevity. Through this dedicated effort and planning, we can focus our efforts on those areas of greater value instead, creating a more seamless Customer Experience which actually solves problems for people more efficiently and cheaply, instead of just making increasingly impressive profit margins for private companies.

Read This

The Myth of the Flat Organization

2017.06.12

“Floor 1997–2000” by artist Do-Ho Suh

Over the last few years, an increasing number of technology companies and agencies have adopted a flat hierarchical structure. This organizational system promises to improve the efficiency of organizations by removing management structures, leaving as few management layers as possible. I believe that it is also almost entirely imaginary.

Since the popular game company Valve published its employee handbook online in 2012, companies have looked to the text for ways to improve their own processes. Growing companies stopped hiring middle level managers, project managers, and other critical roles in favor of letting engineers manage themselves. This was a very attractive move for smaller organizations, since it reduced the number of positions they needed to hire for, reducing their operating costs dramatically. For engineers it can be initially compelling to be able to set their own priorities amongst themselves — and potentially get an inflated title such as “Lead” or “VP” (but often with no additional compensation).

Management is best done by managers

However, in practice, it’s rarely so simple. By serving dual roles on a team, performance often suffers as staff try to context shift. Good engineers do not automatically make good managers — working with computers and working with people require very different skillsets. A company wouldn’t expect a business manager to write code, so why should the reverse be encouraged?

“Flattening the org chart just means creating a hierarchy of emotional labor”
Steven Reilly

In a flat organization, it is no one’s dedicated job to handle many of the complex human interactions that a business must handle. At one organization, I found myself staying up late nights, writing human resource policies, vacation plans, and codes of conduct. From project management to conflict resolution, functions that are filled by dedicated staff in other businesses frequently become after-hours extra labor by engineering staff. So-called soft skills and human-oriented problems are treated as secondary to achieving product goals.

It may be necessary for staff to serve multiple roles initially for very small organizations to be able to function, but beyond the first dozen staff this is not a practical or ethical way to accomplish tasks. If you’re large enough to be thinking about insurance plans and snack delivery, you’re large enough to hire management. A good project manager or human resources officer is a much better investment than that a lavish office space will be.

Even in large companies, it’s very common for women to be expected to perform administrative duties outside of their roles, such as taking notes or scheduling meetings. In flat organizations, it’s very common for many of the “extra” tasks to be assigned to women and minorities first. The inherent cultural biases in technology only increases the odds that these groups will be expected to do more than their share. It’s also common for more junior staff to be assigned the less exciting work, for instance being assigned to fix bugs instead of writing new code.

Hierarchies form anyway, and unfairly

“A flat org just replaces vertical hierarchy with concentric levels of inner circleness… If you remove formality you can avoid accountability and responsibility with policy of openness that is a convention of silence in practice.”
Ozzy Johnson

In many organizations that aim to be flat, a hierarchy emerges based on social cliques and personal relationships, instead of an officially established order. The ideas of staff who spend time socially with owners and executives tend to be adopted more readily. In many organizations I’ve worked for, I’ve frequently seen talented staff passed over for promotion in favor of friends of company owners and managers.

People, by nature, surround themselves with like-minded — and like-cultured — individuals, creating echo chambers and consolidating power in in-groups. This almost always puts women and minorities at a disadvantage. In a structured organization formal policies on hiring and process can help to prevent the biases and inequalities that come from such in-groups, but a flat organization has no such defense from becoming a good ol’ boys club.

It has been reported that this was the situation at Valve as well. Since there is no official hierarchy, there is often no way to call out the favoritism that comes with these factions, and rarely any formal process for resolution. Even when group-based decision making is a part of the process, individuals outside of the power centers tend to speak up less, adding to the asymmetry.


Although I was initially very excited about the prospect of flat organizations, I have yet to work in one that was effective. Modern business practices and the laws that govern them have mostly evolved to help create a more level playing field for employees. Although tech culture fetishizes rule breaking, disdain for authority, and meritocracy, these only contribute to the very toxic culture that has evolved. For the good of our industry, we could all do with a few more rules — and a few more managers.

Read This

Lessons from the Sunset of Sunlight Labs

2016.11.23

A few months ago, I joined Sunlight Foundation as their Senior Technology Adviser. Kat Duffy, who was then the Sunlight Labs director, brought me on to help bolster their programming efforts and lend support across the organization. However, only a couple of weeks later we learned that Sunlight Labs was being shut down entirely. With the time we had until the official closure, we quickly shifted our focus to preserving the legacy of Labs’ work over the last decade.

A quick inventory of our technology assets revealed hundreds of repositories and over fifty servers in various states of decay. A lot of effort would be required to preserve the data and tools for future use. What follows are a few of the lessons from this work.

It’s important to note that Sunlight Labs was never a traditional software development shop – it was always an incubator in which interesting ideas were cultivated. Resources were frequently limited, and delivering new tools and concepts was the top priority. Many of the tools created were never designed to be released. As anyone who’s worked under these constraints can attest, best methods and rigorous practice are often put aside to focus on delivery.

Begin with an end

The most important thing I’ve learned in the last few years is to start all projects with an end in mind. I’ve written previously about the uncertain future of civic technology and nonprofit funding.  We can no longer take for granted that our projects will continue forever. Far too many civic technology projects have effectively become abandonware as a result of lack of foresight.

One effective way of preventing this is to begin your project by thinking about your end goals. Do you want to create a tool for other people to take and use? Is it good enough to simply release some data once and then move on? Is your project one that you’re willing to independently run indefinitely? How long can you expect to be able to work on it? What happens to the project if something happens to you or your organization?

Writing down all of your contingency plans in a public way can help to mitigate these problems when they inevitably come up later. Make sure that you would feel comfortable if you had to walk away from your work tomorrow. Setting a deadline for the end of your work, and communicating this information with stakeholders, can be useful as well.

Documentation

We all are aware of the critical importance of documenting our projects, yet few projects have good documentation. It can be very time-consuming to write down everything that a new person needs to know about your project, but this is absolutely critical work. I recommend doing as much of this work as possible up-front, and making it an integral part of your process moving forward. This is still useful even if your work isn’t writing code for computers – always take the time to write for humans.

In the same way that you have your peers review your work or that you write your tests, make sure to include writing documentation as a step in the process. Make a new rule for your team: if a feature doesn’t have documentation, it doesn’t get merged and it doesn’t ship.

Assume nothing. Although today you may have the luxury of showing every new team member how every system works as part of their onboarding process, that may change suddenly and without warning. Write as if your end user probably knows nothing about your internal process. Make sure you explain every step of using your tools – from getting the data to running and deploying the software. Give examples as frequently as possible, with screenshots and exact commands wherever it makes sense.

Licensing

As a general rule, if your code and/or data doesn’t have a license, no one else can use it. There are huge exceptions to this, but it is almost always easier to be explicit than to hire a legal team.

Since a large number of Sunlight’s projects did not have licenses, I created a tool to add licenses in bulk to all projects that were missing them.  We used the GPLv3 license for most of these, as it is a strong license that requires sharing of modifications to code, helping to preserve these tools for future use.  There are lots of other good licenses to consider, however.

Keep a clean house

One of the more laborious tasks was preparing the many projects to be shared publicly on GitHub. Since a large number of these project had only been stored on an internal GitLab server with no public access, many of them had secure secrets – passwords and API credentials – hardcoded into the source. Frequently, these credentials were stored outside of the usual settings files, and since the credentials were different on every project, tools like BFG were not enough to find all of them. Again, I ended up having to write a collection of tools to migrate repositories from GitLab to GitHub, to check for potentially problematic files, and to automatically remove credentials from those files. Even after that, it still required many hours of manually reviewing code to check for stray credentials.

For most projects, it’s a better idea to never check in any secure credentials to your repositories in the first place. Storing these in the environment instead is a common way of avoiding this situation. If that’s too complicated for any reason, you can always simply use standard configuration files and have Git (or whatever version control tool you prefer) ignore them.

Do remember to keep an example of your configuration file in the repository and up-to-date, though. In one case we were unable to deploy updated code to our server due to an incomplete Fabric configuration, and had to update the code manually on the live server as a result.

Pick the simplest solution

One of the biggest challenges was the large number of simple “brochure” sites that were custom applications written from scratch, instead of using a common off-the-shelf solution. These were difficult for staff to maintain and keep running – often taking three different people to update a single page. In one case, we lost all of the content of a site due to it being built on Heroku using a database service that ceased to exist – we had to scrape the Internet Archive to get the content back. The main Sunlight Foundation website itself was one of the most complicated applications that Labs had created, pulling data from seven different databases – each one a completely different type (including MySQL, PostgreSQL, Mongo, Redis, Memcache, and others) – to deliver the content.

Since Sunlight would no longer have any dedicated technical team, keeping these running wouldn’t be viable. After discussing with the rest of the staff, I picked WordPress as a simple platform for people to be able to manage the content moving forward. We picked WPEngine for managed hosting, so staff could rely on them to handle any technical issues that might arise.

For future projects, I would recommend that developers avoid reinventing the wheel – especially if all that is required is a simple content management system. Working with your staff to identify the needs of a project and finding the simplest solution helps to ensure that things will work long after the developer of that project is gone or is too busy to help. And in most cases, one database is plenty.

… But plan for the worst

A constant pain point was that the architecture of Labs’ software used individual, huge servers that hosted critical pieces of infrastructure shared by all applications, with almost no redundancy. This meant that when the only DNS server or database server went down, most of our applications all failed immediately. This happened several times over the few months while we were shutting down Labs.  This also meant that servers were consistently under-utilized while being incredibly expensive – the monthly AWS bill was over $6,000 per month!

A little bit of extra redundancy can go a long way. Moreover, designing your architecture with small “atomic” pieces can reduce shared dependencies and provide greater stability. This is usually cheaper than running huge servers as well. Running one application per server is a good rule of thumb for smaller tools, and you can always split out services (databases, task runners, etc) as your needs grow.

As a quick aside – over the years, the number of servers I’ve seen fail due to logs filling a hard drive is astonishing. Spend the time to learn how to use tools like logrotate and set up your servers so that something simple doesn’t take down your application.

The final steps

When the time came to shut things down and transfer content, we had the help of many partners. Most notably, the Internet Archive was incredibly helpful in scraping all of our content from a list of sites we provided. Their fantastic 301Works project took our our in-house url shortener domain name and a spreadsheet of our links, to preserve those forever so we were able to shut it down entirely (see above about not reinventing the wheel).

One thing that made the transfer process much, much easier was services such as Heroku and Namecheap which allow you to easily transfer a property to a new owner. For both of these, you simply need to provide a username and the transferee needs only to accept – it’s a very straightforward process. GitHub proved to be more complicated, as we first had to add the new owner as an admin of the repositories and then have them manually move the repositories to the new Organization owner – this took a while with so many repositories moving over.

Amazon was the most difficult piece, as we had to create AMIs of our EC2 instances (a very slow process with the aforementioned huge servers) and then grant access (by account number) to the new owners. I was never able to grant access to an S3 bucket to a separate account – whether the documentation is out of date or there was a system bug, I’ll never know –   for most of these I just made the content public wherever possible and sent out links.

Wrapping up

We should always hope that our projects don’t suddenly come to an end without warning.  But we also must plan for the future as it our work may stop tomorrow. Taking steps up early in a project can help to make sure your work lasts beyond the length of your involvement, and that it can be used by as many people as possible. Make sure you invest the time now, so that you won’t need to later.

Read This

The End of the Second Act of Civic Tech

2016.09.27

The End of the Second Act

On September 20th, Sunlight Foundation announced that it is officially closing down Sunlight Labs, downsizing, and considering how best to further its mission through potentially merging with another organization, leaving the future of dozens of technology projects uncertain. This follows on the heels of OpenGov Foundation discontinuing most of its work on America Decoded, its open-law project [1] [2] [3], and Code for America ceasing direct funding of its Brigades after previously having shut down its technology incubator program. The major funders of many of these and other organizations in the space have also been changing how they’ve been funding. Frustration with projects that are not completed on time and within grant budgets have created an apparent wariness in funding opportunities from many. Some others simply no longer wish to fund transparency efforts, especially at the federal level, believing that transparency is no longer an achievable – or even good – goal. 1 2 Government, at all levels from city to federal, has also stepped up significantly in recent years, hiring more and more technologists in-house to solve problems from the inside, seemingly bypassing the need for civic tech entirely; of course, many of these hires came from the civic tech world outside of government. The days of million-dollar checks for “new” or “cool” ideas are effectively over. In some cases, the criteria these funders have chosen for evaluation of projects’ success – such as the number of “likes” an organization has on Facebook or Twitter – seem woefully out-of-touch with the work of advocacy organizations. Moreover, the funding cycles of these organizations are relatively short – a year or two at most – while it may take five or ten years at least to have the long-lasting impact many nonprofits aim for. This all has contributed to a lack of a clear path for relevant goals and sustainability for many organizations – and even small projects. But the mere merit of having a project labeled “civic tech” simply isn’t enough to justify piles of cash and free reign to use it. If anything, the failure of much of the civic tech movement can be attributed to a strong lack of experience, oversight, and planning. The larger nonprofit world, especially on the service and advocacy side of things, has been using logic models for program evaluation for decades – but this work has gone largely ignored in the civic tech community. Even within the few for-profits that have emerged, impact analysis frequently takes a back seat to promotion. Instead, most projects are started without clear goals, realistic evaluation criteria, or even a firm concept of real impact. “Let’s put some things online” is, in many cases, as sophisticated a model as many projects ever reach; the actual people being served are an afterthought at best most of the time. Publishing tools and data for their own sake is a fine goal, especially those that support journalistic, educational, and research pursuits – all of which are critical to transparency efforts. But we should also stop to consider if our tools will help those who need them most, or if they merely will help the privileged, educated, and well-off – especially for those that are intended to impact equity and access. Is more transparency having the impact on people’s lives that we want it to? Are we really engaging with the people and improving policy or are we merely talking to our friends? How do we know we’re making things better for everyone, working towards equity and not just paying lip service? Simply put, we cannot know if we don’t talk to the people directly, and if we don’t have a way to study our impact. There are some fantastic organizations following through, meeting people where they are – in churches, schools, libraries – but these are few and far between. A few more enthusiastic people with backgrounds in social work and degrees in nonprofit management would be hugely beneficial in our space. At the very least, more time spent on outreach would improve just about every project. Similarly, the technical limitations of many project teams hamper their long term success. The “anybody can do this!” attitude which pervades the civic tech community, while extremely democratic and empowering and wonderful for making the tent bigger, often results in many projects that are under-designed and lack foresight into technical sustainability. Meanwhile, just as many of our projects are the products of more experienced developers wanting to try out new, trending technologies – leading to us over-engineering solutions for niche problems written with ephemeral technologies. In both cases, any likelihood of reusability or sustainability is dramatically reduced by poor planning and decision-making. Many problems being solved today by custom applications could just as easily be solved with a git repository, or WordPress site, or even just a spreadsheet on Google Drive. There are many fine projects that do not fall into these categories, but they’re few and far between. And there is an insanely huge amount of reinventing the wheel in our space, despite efforts to the contrary. As developers, we must escape our own hubris to find the most simple solutions whenever possible, if our projects are to thrive. Engineering is hard; we don’t need to make it harder. We have largely forgotten that the purpose of all of this technology is not technology itself, but rather culture change. Culture changes much slower than fashion, which is (sadly) our most common unit of time-measurement these days. It takes years to build the relationships necessary to change culture. But this is the work 1 2. It’s very attractive to work in silos, programming without doing the hard part of talking to people, but this isn’t a good way to change the world. This is not to say that there is not a value in a proof-of-concept. For many of the changes that the civic tech world would like to see in government or society, simply proving that it can be done is a very powerful tool. But tools should generally solve an immediate need for real people, not imagined “users”. And at the very least they should be used to start conversations, not to spin off new product companies. (There are certainly other very good and valid reasons to hack as well.) After you’ve done the work though, definitely consider finding ways to be paid for it – without money, nothing is sustainable. And though civic tech will always rely on volunteerism, it should not simply become a form of government spec work. But not everything needs to last forever. In many, if not most, cases, all projects should have a firm timeline, with waypoints for evaluation, and a date to cease operation. Having ways to measure real success and failure in terms of impact, with a real schedule, is absolutely critical for projects. Trying to make something last forever that was only designed to last a few months – or simply not working – is a fool’s errand. Meanwhile, finding funding for a short-term project is vastly easier than one that will run indefinitely without clear goals. It is important to note that there have been some incredible successes during the first and second acts of civic tech. The passing of the DATA Act though a bipartisan effort and countless hours of work from The Sunlight Foundation and other groups was a huge milestone towards openness and transparency. Carl Malamud’s tireless work has led to the opening of the SEC EDGAR database, the IRS’ Form 990 data, and much more. And most recently, the hundreds of civic technologists from Sunlight, the brigades, and elsewhere that have gone to work for government agencies have brought their passion to new innovation inside of government. 1 2

Setting the Stage for The Third Act

The future of civic tech, given all of this, is very uncertain. Local brigades and volunteer groups have become in many ways the core of the movement. These groups have increasingly been working with government directly to introduce new tools and policies to increase transparency and equity. These spaces will continue to foster new ideas for their communities, and provide a ready source of civic-knowledgeable technologists for government and nonprofits – this is critical for supporting the infrastructure of transparency. And rather than being fueled by grant funding, this work is largely due to the passion of thousands of volunteers all over the country (and world!) However it’s also, in part, due to contributions from the private sector. In the same way that for-profit companies have sustained the open source movement for decades, so too are they stepping up to support the civic tech movement. Some companies provide direct financial support to brigades and community groups – paying for food and event spaces. Moreover, companies like Esri and GovDelivery are using open data, open source, civic tech as central parts of their platforms. This will only continue to grow as other funding sources become increasingly scarce and the interest in open data and transparency expands. Government is also changing to become more [civic] tech-savvy. The work of government units such as 18F in the General Services Administration and the U.S. Digital Services in the White House have impacted the Federal government in substantial ways 1 2 3. But moreover, this has created new enthusiasm in cities and states – causing an unprecedented rise in new CTO and CDO positions all over the country, and policies reflecting the changing landscape of data needs. (As an aside, it’s noteworthy that many of these positions are being filled by former GIS data team members, a field where open data and civic tech has particularly long and deep roots!) Tools and data are more and more often being produced in-house rather than outsourced. Another place I would love to see civic tech find a home outside of government is within libraries. I must say that this is idealistic, at best. In many places, libraries have become – and really, have always been – a bastion of innovation and civic focus 1 2 3. It’s a very natural fit for both parties, and – most importantly – the people being served are physically present! There’s no way to avoid doing the work of meeting people where they are if you start where they are. Increasing investment in civic technology from libraries would be a huge boon.

What You Can Do

Civic Organizations, Project Owners, and Dreamers

For any new projects just starting out, or years along their path, I would strongly recommend answering a few key questions before spending another minute on your work. You may have done many of these already, but writing them down and posting them publicly is to everyone’s benefit. They also provide a solid starting point to have an honest conversation with funders.
  • Identify your outcomes – what do you really want to change as a result of your project.
  • Evaluate your impact – what measurements will you use to judge that your work is causing the outcomes you intend?
  • Who is the audience? Where are they? How can you meet them where they are? Be specific, think about who is in need.
  • Talk to your audience – what do they think they need, and how well does your idea match up?
  • How long will the project run? What are the milestone for evaluation along the way? Set dates to evaluate your actions.
  • What is the real level of effort and cost for your project? Be realistic, think about the hours needed, not just to write code, but go to meetings, do user testing, and everything else.
  • How can you use the work of others to help you with your work? What tools, data, and research already exists in your problem space? Who else is already actively trying to solve this problem, and how can you work with them?
  • How can you make your work reproducible? What can you put online and publish for other people to be able to make an impact using what you’ve learned? How will you license your work so others can use it? 1 2 3
Just to reiterate that last item – civic tech works best when we collaborate. Collaboration works best if we embrace radical transparency, to share all of our research, methodology, and findings with the world. Even our failures can become victories if it helps someone else to do things better the next time. And this is true in the for-profit world as well! A single vendor in a space will struggle, but fostering an ecosystem will drive more interest, demand, and need. Open data and open source help make more open data and open source!

Board Members, Founders, and Executive Directors

Listen to your team. They’re there because they believe in what you’re doing! For an organization to grow and thrive, it must continue to change and evolve – while still staying true to its principles. Stop and look at the shape your organization is taking, you may find that it has outgrown your old vision and become something radical and new – and that’s ok! Don’t get stuck on one idea, be flexible. Listen to your team. You can’t do everything, find people that you trust and let them do what they’re good at. When they give you advice, take it. If most of your advisors are telling you one thing and you believe another, it’s probably time to step out of the way and let them do their jobs.

Government Officials at All Levels

If you’re a government official, you’re probably already aware of what you must do. First, make sure you understand the open data and policy push that’s happening today. And then, get excited about it – find a local civic tech group or brigade and chat with them about your job, and hear how passionate they are! If you put a problem in front of a technologist, no matter how big or small, we will immediately start coming up with ways of solving it – that’s where our passion comes from. Get excited with us. Some of the most passionate people I know in the civic tech movement started in government, not as technologists! Second, start viewing your job through a lens of transparency. Ask yourself, in everything that you do, how can I expose this work? How can I share with the community the problems and solutions I encounter? What data can I share with the world – whether I personally find it interesting or not? How can I work with other departments to improve communication and share knowledge outside of my silo? Try to find an issue, something that you can push on and make more transparent, and follow through with it from start to finish. And make sure to tell the community about it! No matter how mundane, we love this stuff.

Funders and Charitable Givers

If you happen to be a funder of nonprofit projects and organizations, first and foremost I ask you to renew your faith in the transparency and civic technology movement. We still have a lot of work left to do! And government can’t do it alone, small non-profits can’t do it alone. Innovation must be fostered from a variety of sources. And please keep in mind that this process takes a very long time! One- and two- year cycles are only long enough for a proof of concept, not nearly enough to effect real change. Organizations desperately need funding for general support in addition to individual projects – the day-to-day stuff like meetings, promotion, calls, and outreach are all just as important to running an organization as the fancy new releases. However, I encourage you to hold projects accountable to the criteria for organizations and projects listed above! There are real-world metrics of impact that can effectively track the progress of our projects, make sure you know what these are and so that we can better judge efficacy of programs. Hint: it’s not likes on social media – find real, meaningful ways of evaluating the work. If you’re a board member, your task above and beyond everything else is to understand real impact, and not just numbers. And – in many ways the most important ask I have of you – be aware of your own, personal blindspots and work to expand them. There are thousands of good ideas that could improve the world today and just need someone to take a chance on them. Just don’t get too distracted by every new and shiny thing – again, older projects still need long-term investment and growth. It is only by working together that we can make the world a better place than we found it.

Read This