Compounded Self Improvement

Posted on June 22, 2018 by Gabe Parmer

Nothing in the world can take the place of persistence. Talent will not; nothing is more common than unsuccessful men with talent. Genius will not; unrewarded genius is almost a proverb. Education alone will not; the world is full of educated derelicts. Persistence and determination alone are omnipotent.

  • Calvin Coolidge

Learning how to do both research and system design and implementation is hard. We enter into a research project with some base-level of knowledge and capability1. Most interesting problems – including earning a PhD – require you to increase your knowledge to the point that the problems are tractable. I’ve come a long way in my own understanding of what it takes to make myself productive, a process which include both gaining knowledge, and the capability to deploying it. This post covers my model for understanding the factors that go into this process.

Unfortunately, the means for systematically acquiring knowledge and capability are an area far from my expertise, so this post should be interpreted as opinion. This is a dump of my own mental models about the factors that go into being effective.

Knowledge Acquisition as Compounded Interest

I’ve watched enough students wrestle with understanding how systems work, and how to design them, to have confidence in the following assertion:

The amount of time that you put into research yields super-linear increases in capability with respect to the amount of time spent.

The more you know \(\to\)
the more context you have to understand concepts \(\to\)
the more you can make connections between concepts \(\to\)
the more you can understand \(\to\)
the better your model of the world is \(\to\)
the more you know.

In short, the more you know, the deeper and faster you can dive into a system, which accelerates your understanding and knowledge about the system. This creates a “virtuous cycle”2 that must be leveraged to become a world expert in an area (i.e. completing a PhD).

A Model for Knowledge Gain

This knowledge gain (at some level of detail) seems quite similar to me to the impacts of compounded interest. The basic idea behind compounded interest is that a given annual percentage rate (APR) increase in your funds, will increase not only your principal investment, but also on any previous APR increases from previous years. The impacts of compounded interest are, somewhat unintuitively, very significant. The compounded interest formula, interpreted in the context of knowledge gain:

\(K = k' \times (1 + r/n)^{n \times t}\)

where:

I’ll get to where I think this model is wrong in a bit, but for now I’m going to discuss it’s utility. To make it concrete, I’ve plotted three different configurations:

The absolute value of “knowledge” for each of these is somewhat meaningless. Instead, I’ll focus on the relationships between the lines.

Impact of effort. The effort put in over time to better your knowledge and capability is represented by \(r\). The blue and red curves only differ in this parameter, and the red curve represents someone who puts in 75% of the effort of the blue. If someone works 6 hours a day for someone else’s 8, after 6 years, they will know less than half as much as the other. The intuition here is that relatively small, but consistent effort put in over time compounds to significant capability differences.

Impact of initial knowledge. Everyone comes into a domain with a base-level of knowledge. The green line represents someone with double the base knowledge of the other lines, but who puts in 75% of the effort of the blue line. The blue line passes and makes up for this initial disparity in knowledge and, given the slope of the line, strongly accelerates past the green. Note that sometimes having a higher base knowledge can structurally encourage lower knowledge-gain rates (see “Smart Kids” below).

Super-linear increases in knowledge. The exponent in the formula, if anything, is the key insight of the model. Note how quickly each curve increases in its rate of rise at around 3 years. Anyone who has done a PhD knows how quickly you feel like your progress increases at around this time.

Lies and Models

As I mentioned previously, I have no idea if this model is accurate or true. Sometimes we have to make models to understand ourselves and the world around us, and if they are outside of our domain of expertise, they could be completely wrong. Think of it as a hypothesis that needs to be tested and reflected on as we gain experience. That experience can refute the hypothesis, or it can allow us to gain some confidence in it.

In my experience, this model provides significant predictive capability. It predicted my experiences with myself and other PhD students when I was doing my doctorate. Most importantly, as I’ve mentored more and more students, it has predicted the outcomes of many – though not all – of them. More importantly, this model increases my own motivation to effectively climb the steep curve of knowledge acquisition.

If this model helps you, great. If not, shed it off, but make sure to ask yourself how you can effectively improve yourself in both the short term, and in the long term.

Time \(\neq\) Productivity

What factors go into any specific rate, \(r\), of knowledge acquisition? A clear factor is time. Spending sufficient amounts of time on self improvement will yield higher productivity, right? Well, no. We can use our time poorly, and knowledge gain does scale up perfectly with increased time expenditure. More importantly, if you try and optimize for \(r\) above all else, it will likely have the opposite effect.

If you try and fill your weeks with learning to optimize for \(r\), you’ll quickly find that \(r\) actually decreases. It is important to consider the relationship, \(r = f(h)\) where \(h\) is the hours per week you put into self-improvement. This is certainly not linear. In my experience it is logarithmic – you reach diminishing returns after a certain \(h\) – with a distinct “cliff”, \(c\) – where \(f(h > c) << f(h \leq c)\). See the section later on diminishing returns for an elaboration.

X Hours Toward Maximum Productivity

Just because you spend \(X\) hours working and researching, does not mean that you’re gaining an optimal amount of experience from those hours. I’ve previously discussed how to best use your time in the research domain. I’m only going to add one observation beyond that post.

“Smart” kids. In CS, there is a bad perception that students who will achieve the most, are those that start out with significantly higher knowledge than their peers. There’s a harmful stereotype about the geeks destined for success. Don’t buy into it. Of course, knowing more and being more capable is always preferable to the alternative, all other factors being the same. However, it is my experience that the rate of knowledge gain is higher for students who come in with significantly lower base knowledge. As the graph above shows, a higher rate can easily surmount base knowledge. I’ve observed two contributing factors to this:

Learning to fight for knowledge is hard. Research can be quite challenging, and should push anyone involved beyond their comfort zone. Some people have been able to leverage their innate intelligence and ability to get them to their current position. It can be difficult for these people to suddenly be quite challenged, and have to push their own knowledge acquisition rate up to compensate. In short, if you haven’t had a significant amount of practice having to fight for your knowledge, it can be difficult to learn how to summon that drive.

“Knowing” the answer is often counter-productive. Their high base knowledge is often accompanied by a high confidence. This confidence is, in many cases, quite justified given their past successful experiences. However, that confidence will not be justified in a domain that significantly pushes their capabilities. Research, by definition, should not be a domain approached with a sense that one either knows the answer, or can rely on their intuition to provide it. Difficult problems demand that you listen to your intuition, then you ignore it, and pivot to rigorously evaluate the situation. Intuitive, knee-jerk solutions often kill creativity, and rule out innovative solutions. Knowing “the answer” is counterproductive for finding the “best answer”.

A characteristic I’ve observed about people in this category: making assertions instead of asking questions. For example, a common behavior I’ve observed: someone doesn’t approach a technical discussion by asking questions, and instead provides assertions that (without some push-back) would kill further discussion. From a group perspective, I think this has a dangerous impact on effective discourse. I’ve seen this kill conversations about deep materials, stifle creativity, and result in ineffective communication that stunts the mentoring process. If someone approaches group discussions in this way, they likely approach their own inner monologue similarly.

I believe this can easily be remedied: prioritize asking questions, and foster a skepticism about intuitive assertions. A good way to do this, is to change your perspective on how to interact with others.

Diminishing and Negative Returns

The first six years of my higher education were characterized by what I think is an immature caricature of how to be productive. I valued hard work above almost anything else. I’ve always been work/life ratio-challenged as I generally enjoy the domain of my work so much, but this phase in my life took that to an extreme.

It took me two years of doctoral work to completely burn out, and realize that putting hours in after a certain point not only gave diminishing returns in terms of knowledge acquisition, but had negative returns. Spending an extra two hours in the lab on a Saturday inhibits the recovery of necessary “down time”. As it turns out, I can will myself to put outrageous amounts of effort in, but I cannot will myself to spend all of that time productively.

Constantly re-evaluating if you are spending enough time on the aspects of life that enable you to refresh and be productive is both important, and very personal. It took me a long time to converge on a pattern that allows me to be productive. Though I’ve aggressively pursued a knowledge gain curve with the highest possible slope, it was essential to recognize that the number of hours put in was not the only relevant variable. As part of understanding yourself, it is important for everyone to find all the variables that go into their own knowledge curve.

Summary

The take-aways:

  1. Knowledge and capability increase super-linearly. It is hard for humans to understand exponential relationships, and aggressively super-linear relationships, in general. This means that you might look at a challenge that is impossible now, but in a couple of years it might be trivial.
  2. Where you start out in your knowledge is less important than carefully ensuring that you grow over time. As intellectual growth is a compounding factor.
  3. It is important to understand how to maintain a healthy rate of weekly time expenditure. Too little, and you aren’t taking advantage of the compounding factor, and tasks that seem hard will remain hard for a long time. Too much, and you reach diminishing returns due to cognitive limitations and a lack of work-life balance negatively impacting your growth.

So, it is important to contemplate what we each should do to effectively gain capability in our chosen fields. This is meant to increase introspection into how you might plan and manage your own self improvement, and gives you a rough model through which you can reflect on and plan your own development.

  1. I believe that having knowledge, and having the capability to deploy it are key aspects of productivity. Regardless, to simplify the discussion, I’ll be using “productivity”, “knowledge”, and “capability” as proxies throughout this post. Further, “effort” is required to increase in any of these dimensions. Without putting in time and sweat, there is no knowledge, capability, nor productivity. The mapping between effort and the others is not straightforward and is discussed later in this post.↩︎

  2. The foundation for the systemic efficiency of for many aspects of capitalism.↩︎

  3. Note that I’m underspecifying what the “unit” of knowledge is so as to avoid epistemological distractions.↩︎