[book_front] [book_back]
night mode
[1][                    ] Subscribe
 
[3] RSS
Add this blog to your rss feed
[4] github logo
Pollute your mind with some of my other missguided creations on Github
[5] email icon
Contact me at: george@cerebralab.com
[6] twitter logo
Publicly berate me on Twitter

This blog is no longer active, you can find my new stuff at: [7]george3d6.com, 
[8]epistem.ink, [9]ontologi.cc, and [10]phenomenologi.cc



[11]Audio version (read by a tts bot)

Imaginary Problems Are the Root of Bad Software

https://upload.wikimedia.org/wikipedia/commons/c/c0/
De_Rebus_Bellicis%2C_XVth_Century_Miniature.JPG There are many factors which
can be a catalyst for bad software: from the tools being used, to team
communication, to the personal stake developers have in its success, to the
testing methodology.

I propose that there is one problem chief among them, an impetus for bad
software from which almost all others take root: imaginary problems.

Most complicated or broken software is not designed to be overly complex or
dysfunctional. It’s just designed to do something other than its intended
purpose.

Let’s say you’re a podcast host who wants a custom website where you can sell
your promotional products, make advertising money without a third party cutting
in, and, most importantly, deliver podcasts, videos, and blogs to your
audience.

The requirements for your little web-app might look something like this:

  • Fast load time in North America, with real-time podcast streaming and
    downloads
  • Doesn’t crash or freeze in the first 15 minutes for 99.99 percent of users,
    preferably never crashes or freezes
  • Integrates well with Google Adwords and maybe some other third-party ad
    providers as well, if there’s time
  • Dynamically links to the latest products in my Zazzle shop and, if
    possible, gives recommendations to users based on the content they’ve
    consumed
  • Integrates with Facebook live player. If it’s easy to create an alternative
    solution for streaming that doesn’t require Facebook, even better

You give these specs to a team of contractors, and you chat about them a bit.
It seems that everyone is on the same page. Yet, when they return with the
Minimum Viable Product two months later, your face turns red. You’ve just
wasted $15,000 on a piece of garbage; you want your money back.

The first time you open the app, the screen freezes. You ask how to select what
kind of ads should be allowed to run on the site and are pointed to an ugly,
hard-to-understand custom user interface (UI). Half the links to your
merchandise on Zazzle are broken or missing images, and the Facebook livestream
is laggy!

But the development team is confused at your anger — rightfully so, from their
point of view — because they’ve gone to hell and back for you.

They’ve put their heart and soul into creating this app, and it has some
amazing features:

  • A state of the art recommendation system
  • An algorithm generating the transcript of all your streams, in real time
  • Your front page loads in sub 200ms times all over the world
  • A streaming protocol and client build almost from scratch, in case you
    don’t want to rely on Facebook live
  • A service that allows you to easily integrate over 20 ad exchanges

The problem is that you thought you requested a core product with a couple of
extra features, if they were easy enough to implement. Meanwhile, the dev team
heard something else. They heard about some exciting challenges they could
tackle… and a slew of boring, basic features they couldn’t be bothered to test
properly or care about.

Even worse, you didn’t communicate directly with the devs — you communicated
through a game of Telephone. You spoke to a sales guy, who held a meeting with
some middle management chap, who wrote some business specs and gave those to a
PM, who wrote some technical specs and gave those to a team lead or architect,
who then, at last, began to design the product with his team — each one of them
putting a bit of his own twist on it along the way.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Imaginary problems are often more fun to solve than real ones. Extremely
intelligent people play competitive games, construct and solve math problems,
and write books that aim to answer abstract questions about the human
condition, all of them for free. A mediocre programmer, however, will probably
charge you a fair amount to build a simple Android app. That’s not because
mediocre programmers are harder to find than geniuses, but because the former
activities are all fun, while the latter can be quite boring.

Most programmers want to get paid and have fun at the same time. Of course, the
definition of “fun” is different for everyone, but for many engineers, it boils
down to tackling interesting and challenging problems that are within the realm
of solvability.

Give a somewhat intelligent person too many boring tasks that are impossible to
automate and you will eventually drive him mad. The human brain however, after
billions of years of evolution, is quite talented at keeping its sanity. Much
like victims of childhood hardship or abuse can find escape in fantasy books,
victims of enterprise programming or freelance web development can find their
escape in solving imaginary problems.

[1]

The amount of imaginary problems a software engineer can create for themselves
is a function of their imagination and of the difficulty of the real problems
they’re supposed to solve.

It should be noted that this issue isn’t unique to developers. Management,
sales, HR, support, legal, and even accounting departments have their own
unique ways of creating imaginary problems. They try to involve themselves too
much in a decision, when their presence at a meeting is just a formality or
wasn’t requested at all. They overemphasize a minute problem that is related to
their role, or hire teams much larger than necessary to illustrate their
importance.

When problems are dumb, intelligent individuals will find a way of coping.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

But imaginary problems aren’t just the result of bored developers. They’re also
the result of long chains of communication.

When I first began taking on freelance clients, I couldn’t afford to be
particular. This means I’ve had email chains lasting for over a 100 exchanges,
discussing insignificant details about internal MVPs. I’ve had people change
every single requirement on a project within the span of a week. I’ve had
clients ask questions such as “Could this be ICO-ed?” or “Can we add some A.I.
in here?”

Granted, most clients are savvier than that, but, even still, they often lack a
bit of the knowledge necessary to articulate or construct some of their
requirements. That is fine, as part of my job as “the computer guy” is to help
people figure out what they do and don’t need based on their use cases. But it
can become much harder to determine what’s needed when there are a few layers
between you and the client.

    Requirements get changed because someone either misunderstood an intention
    or because someone was trying to cope with that aforementioned boredom

Most companies like having a sales guy who pitches potential customers,
negotiates prices, and outlines possible features. They also have a [12]people
person to discuss more in-depth requirements and details with the customer,
usually another sales guy, but with a slightly different title. Then there’s
the internal chain of command, various levels of management, and possibly some
hierarchy, within the technical team.

When a list of client requirements goes through so many people, even if those
people have the best of intentions, some things will inevitably get lost in
translation. Sometimes that change happens because the original requirement
made no sense, or sometimes requirements need to be redefined. The sales guy
might have told the client, “for only 39,999 extra we can do this on the
Blockchain.” But that leaves everyone who encounters the requirements down the
line wondering what the definition is of “doing it on the Blockchain.”

More often than not, requirements get changed because someone either
misunderstood an intention or because someone was trying to cope with that
aforementioned boredom, trying to make his job or the work of his team more
interesting and impressive.

Through all of this, the original requirements — the real problems that have to
be solved — get lost. They are replaced with imaginary problems and with voids,
and you’ve got plenty of people ready and willing to fill those voids with
their own imaginary problems, because the problems they have to solve are
boring, and filling the voids gives them a way of coping.

Overcomplexity and natural selection

There can often be an even darker reason for the existence of imaginary
problems: problems can help a team or a company grow, and can even become an
integral part of its function.

    “People who are bred, selected, and compensated to find complicated
    solutions do not have an incentive to implement simplified ones.”

    — Nassim Nicholas Taleb

Have you ever heard about those three web engineers who figured out that secure
online banking is actually quite an easy problem to solve? They developed some
flawless banking software from scratch, using a functional design methodology
and memory safe languages, then started migrating major banks to their amazing
infrastructure.

Probably you haven’t heard of them, because they don’t exist. There are,
however, plenty of teams of [13]thousands of developers, who are unable to
grasp simple concepts such “rollbacks,” perpetually creating banking software.

The storage and transfer of numbers is not a particularly hard problem.
Indexing the whole content of the internet and providing relevant results to
natural language queries, in sub second times, is a hard problem. [14]But just
a few smart guys managed to solve that problem.

The persistent problem for online banking is that the banking ecosystem has
become really good at preserving its own money-grabbing hierarchy. Its leaders
are [15]corrupt leeches who prey on society — but the leaders in an
organization are just a symptom of its members.

I wouldn’t suggest that most underling workers for banks are evil or malicious
in any way. Far from it. They are usually friendly lads, working to provide
food, shelter, and an education for their families. But their chief incentive
is not to fix the banking software, it is to stay employed. Losing your job in
today’s economy is no joking matter for some; in the banking industry, a big
mouth or too much initiative is an easy way find yourself in front of a
disciplinary committee.

So banking systems remain the same — not because the systems are efficient, but
because of inertia. This inertia comes in the form of working on imaginary
problems in order to avoid fixing real problems — real problems which, once
pointed out, would threaten the jobs of other people. To focus on these real
problems could lead to getting fired, or, in the case of some particularly
nasty “institutions” like Goldman Sachs, [16]getting a few life-ruining brown
envelopes sent to a few FBI officers and prompting a strange suicide.

    “It is difficult to get a man to understand something, when his salary
    depends upon his not understanding it!”

    — Upton Sinclair

The [17]C-suite ignores the fact that their upper management workers spend 90
percent of their time on “client meetings” that involve tropical islands and
million-dollar budgets for “other expenses.” Upper management, in return, turns
a blind eye to corruption in C-suite.

Because middle management encourages them to live in their Wolf of Wall Street
fantasies, upper management ignores middle managers who buy eccentric offices
and hire themselves three secretaries and a dozen interns.

Because line management doesn’t complain about their dictatorial power
fantasies, middle management ignores the fact that line managers, instead of
cutting costs, spend their time working on PowerPoint presentations about
“Improving our Agile Methodology.”

Because the team leaders don’t seem to notice the fact that their superiors
can’t even use Excel properly and only hit the office every few weeks, line
managers ignore the team leaders and architects talking about “next generation
interfacing between our systems using JRPC and microserviceization using
Hibernate and Spring” when they should be getting those bloody Mysql queries to
take less than a day.

Because the developers don’t seem to notice that their leaders don’t really
write any code except DOT diagrams, team leaders don’t complain about their
developers, instead of looking at an EXPLAIN for the aforementioned slow query,
re-designing the UI for the tenth time that year using a new JavaScript
framework.

It’s a vicious cycle of solving imaginary problems, from the CEO who doesn’t
realize that stealing another 30 million won’t make his dad love him to the
user-experience intern who doesn’t realize that redesigning the “submit” button
using Angular-Material-Bootstrap 19.13.5 won’t make the fact that they store
passwords in plain text (and use them as part of the auth cookie) go away.

But everyone needs to keep solving the imaginary problems, because if they stop
creating and solving these problems, if they start focusing on the real
problems, they might realize the whole system is broken. They might realize
Debra has been sitting in that corner, staring at uptime graphs of the internal
server farm for 10 years, despite the fact that the company moved to AWS five
years ago. They might realize 99 percent of their job is to perpetuate the
existence of someone else’s job. And that’s a hard realization to
digest—impossible for most, I dare say. So, instead, most find a way of coping.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

If you enjoyed this article you may also like:

  • [18]The Red Queen
  • [19]Stop future proofing software

Published on: 2019-04-29



━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━


[20][                    ] Subscribe




Reactions:

angry
loony
pls
sexy
straigt-face
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[22] twitter logo
Share this article on twitter
[23] linkedin logo
Share this article on linkedin
[24] Fb logo
Share this article on facebook
















References:

[3] https://cerebralab.com/feed.xml
[4] https://github.com/George3d6
[5] mailto:george@cerebralab.com
[6] https://twitter.com/Cerebralab2
[7] https://george3d6.com/
[8] https://epistem.ink/
[9] https://ontologi.cc/
[10] https://phenomenologi.cc/
[11] https://youtu.be/9jODhmgkp3o
[12] https://www.youtube.com/watch?v=hNuu9CpdjIo
[13] https://www.theguardian.com/business/2018/apr/28/warning-signs-for-tsbs-it-meltdown-were-clear-a-year-ago-insider
[14] https://en.wikipedia.org/wiki/History_of_Google
[15] https://en.wikipedia.org/wiki/Ben_Bernanke
[16] https://en.wikipedia.org/wiki/Sergey_Aleynikov
[17] https://www.investopedia.com/terms/c/c-suite.asp
[18] https://cerebralab.com/The%20Red%20Queen
[19] https://cerebralab.com/Stop%20future%20proofing%20software
[22] https://twitter.com/intent/tweet?text=A%20shameless%20content-promotion%20script%20has%20asked%20me%20to%20share%20with%20you%20this%20amazing%20article%20I%20just%20read:%20https://cerebralab.com
[23] https://www.linkedin.com/shareArticle?mini=true&url=https://cerebralab.com&summary=A%20shameless%20content-promotion%20script%20has%20asked%20me%20to%20share%20with%20you%20this%20amazing%20article%20I%20just%20read&source=https://cerebralab.com
[24] https://www.facebook.com/sharer/sharer.php?u=https://cerebralab.com
