Knowing when to be dumb
There is such a thing as being too smart.
I’ve written over 10 drafts of my first book, one of them got to over 50% completion before I lost faith in it. I am in some ways, more of a dumbass than a dumbass.
You see, even long after letting go of the kind of perfectionism that most people talk about, the sort where you never start anything because you’re afraid it won’t be perfect, it is possible to be guilty of a second-order perfectionism in which nothing is ever finished because there is always a better way to do it. This is what the historian Peter Burke referred to as Leonardo Syndrome, after Da Vinci, who famously left a lot of unfinished work.
This is worse than ordinary perfectionism and worse than not having any ideas at all; from the subjective point of view. (of course such people can still accomplish much, the disease is literally named after Leonardo Da Vinci!, what I mean is that this way of living is not pleasant or sustainable)
At least an uninspired person saves their time, mental energy and physical energy and gets to enjoy a simple life; and the ordinary perfectionist, although mentally tortured, should have a surplus of physical energy since they never do anything! By contrast, the iterative perfectionist is always working and yet never satisfied; extending themselves in several directions at once, in search of an ephemeral something that always seems out of reach. It is both a waste of time and a waste of energy, rather than just a waste of time or the bliss of ignorance.
Moreover, it is self defeating. Trying to be too clever, to improve too quickly, can actually produce worse outcomes than being stupid!
This is actually an essential concept in the field of machine learning; it is one of those funny little insights we can glean from teaching the computers that actually reflects back onto us in an interesting way.
When you iteratively train a computational model, from simple statistical regression models all the way up to Claude, there is a concept called the learning rate. The learning rate is what we call a hyper-parameter (a parameter that helps us tune other parameters) whose job is to define how much the machine adapts its model parameters after making a mistake. You can think of this as setting how quickly a dog is able to learn a new trick. The lower the learning rate, the older the dog.
To explain the intuition behind this, imagine a ball rolling down into a valley. If the ball is only allowed to move really slowly (I guess it is made of a sticky material or something) then it will take a longer time for it to roll down to the bottom of the valley. If you allow it to move a little bit faster, then it will obviously get there in less time. If, however, you were to strap some kind of propulsion device to this ball, so that it could move at a great speed (like the snitch in Harry Potter), then all of a sudden there is no guarantee that the ball will ever reach the bottom of the valley because it is bouncing around all over the place.
This is what happens when you set the learning rate of any iterative model. Too slow gives you the right answer but at a higher cost, just right gets you the right answer as quickly and cheaply as possible. Too fast, too much adaptivity, and the system becomes chaotically divergent and fails to find the optimal solution .
This means that there is a very real mathematical sense in which in any iterative learning process it is better to be a little bit slow than to be too smart. Stupidity remains convergent to the (local) optimum, overcomplicating does not. If we are working towards the right goal, then there is such a thing as adapting too quickly.
As such, if you are fairly sure that the endeavour you are setting upon is worthwhile, it is actively a bad idea to look for the absolute ‘best’ or ‘most correct’ way to pursue it. You are actually more likely to accomplish something if you have a go at making something that is kind of dumb and janky and iterating it from there. It may be more physically exhausting to go through those additional iterations but you can take mental rest in the certainty that you will get there eventually.
Rather than trying to make a masterpiece, think of the simplest, most manual, least efficient but still functional way to start building what you want to build and just start doing it. The Goofiest Viable Product, if you will.
It will be inefficient, it will be an imperfect way of going about it, but at least you won’t overshoot. You will almost certainly find ways to improve on your plan as you go along, and those improvements will not be wild speculations on the basis of minor negative signals but actually concrete improvements to a process you’ve already undertaken, with a clear intuition about the value of the saving that you have in some way embodied by all of the grunt work you have had to take to get there.
n.b. That’s actually why I started this blog, to start producing complete pieces of writing that I had to push out on a regular deadline. To get into a better habit so that I could be more disciplined and less clever with my book, so that thing has a hope of seeing the light of day, sooner or later. Sure enough, most of the plot-holes have already to started to fill themselves in, so I should be able to stitch together an answer from the best of my drafts fairly soon.
If there is something that you feel you ought to do with your life, it is better that you did it imperfectly than didn’t do it at all because you were constantly trying to find ‘the best way’ to do it. There is no best way, or if there is, you cannot guarantee that you’ll get there by swinging wildly. Instead you have to get there gradually, by pursuing smaller problems that you reckon you can handle, building up small bits and pieces that you can use to make something bigger.
Yes, it may be that there are other people out there who are smarter, who are faster, who are doing all that and seeming to make it work. However, if you go to an actual networking event (as I sometimes do, working with tech) you will find that these people are never the most brilliant people in the room. They are often markedly average. They have got fast and lean and agile by having spent plenty of time not being those things, making steady improvement and learning as they go. I also suppose that’s why the 10x engineers never make their own unicorns; The Wozniaks need a Jobs to kind of slow them down, tether them to Earth a bit, keep them constrained within on a more limited problem-space that doesn’t completely throw them off the rails.
For those of us who wish to make some kind of positive impact in the world this is especially important. It is important to recognise that our actions will not always be perfect, that we may even make mistakes and cause problems just like everyone else. However, if we constantly try to leap to the absolute optimum, we will probably just overshoot and cause more problems than we solve.
If you know the problem you want to solve, don’t overthink it, just find the shortest path that will work, make a commitment and fulfil the commitment before introspecting further. Be like a river burrowing its way through the silt, steadily optimising for the shortest route (which mathematically is very similar to training a neural net btw).
If, however, you are unsure of the problem you wish to solve, that is where introspection, doubt and concern have their valor. This post will be followed up next week with a piece on just that. Stay subbed for more.