Empiricism and Bayesian Updating

How do I make a profitable trade or select a company to invest in?

How do I know if I'm working out too hard or if I need to push through?

How do I know if he is a moment away from submitting or I'm to gas out?

Talking to 10 different experienced traders or fund managers, personal trainers, or martial arts practitioners you will get 10 different answers to those questions.

Who is correct?  Well, they can all be correct.

The reward in each system comes from an amorphous expectation function, each agent may select actions that contribute to positive expected value in their own way; effectively, their unique experiences contribute to alpha in the subsequent realized outcomes, positive or negative.

There is mathematical structure that separates the experienced and capable from the incapable.  Borrowing from the reinforcement learning literature it comes down to a policy function, state, action, environment, expected and realized reward.  

Over time, agents act more optimally achieving highly desirable outcomes.  

But the road to get there?  Yeah, not pleasant, and walked by very few...

Progress is highly non-linear and stochastic.  

How do we know if we are acting sub optimally or if we are just experiencing downside variance from an optimal action? 

The world is relatively easily explained by mathematical structures in the reinforcement learning literature, so let's start there.  

This is not some academic exercise, rather, this framework discerns how rational and irrational agents select actions toward a desired outcome.

Who cares?  

This is functionally what separates the experienced and capable from the inexperienced and incapable.

Here is the framework...

Agents act in an environment given a policy function and a state.  Agents don't act randomly, rather with some expected reward in mind.  

Why expected?

Agents aren't sure if the action they select based on a function of their current state and environment will yield the desired outcome.  

There would literally be no point to anything if they did. 

Ok, so agents then have a set of actions, what they can do, and they select one that they believe will yield them closer to a desired outcome (expected reward).

Actions either yield the desired outcome, or they don't.  In fact, even if a reward isn't binary, there is a reasonably qualitative gauge for how close or far you are from the subsequent desired outcome (a distribution, if you will).

After the realization of an outcome, rational agents don't just sit around and go "cool, that didn't do what I thought it was going to so I'll repeat that action."  

No, they try to understand and learn, discerning whether the outcome was primarily a function of that action, chance, the list goes on...

This is the purest qualitative sense of Bayesian updating.  Agents don't have the parameters for their upstream policy functions so they are actively trying to learn them via their priors and noisy estimates in real time.

But there is a problem.  How do we know when and if we are right?

Did an action not work once or is it not going to work a majority of the time?

Welcome to the game.

With this framework in mind, when, do we update our expectations about whether an action at a given state will yield a particular reward?

Obviously extrapolation is not an easy problem, otherwise, again, there would be no point to anything.  So how can we become better at generalization?

Well, last week I mentioned I would discuss ego, altruism, and the independent creator.  Where does this fit into this framework?  Is it even relevant?  

Hang tight, it'll become more clear where these ideas fit into the bigger picture.

Social media sees founders talking about mindset and routine, and everyone loves to hate.  Arbitrary motivational platitudes and philosophy are dismissed.

But they exist for a reason: they work.  Otherwise, an agent's policy function will succumb to the state and environment before it has a chance to learn.  Before it has a chance to make it to the long-run.  Before it can survive and achieve its convex payoff.  [see my previous Guild letters]

Let's start with an analogy, a simple example, and work our way to more specific ideology on how I dynamically update my policy function.

Applying the Framework: Brazilian Jiu-Jitsu (BJJ) Example

Hopefully you're tired of the BJJ analogies by now.  Good.

That means you've read my letters on survival and convexity.  The application of this framework herein will illustrate the series of iterations required upstream to achieve desirable long term outcomes.

Starting with a trivial example is the best way to see this learning framework in action.  To win a BJJ competition you must submit your opponent.  In the Wild West (Expert) division every technique is legal.  Now you are in a fight.

Suppose you are actively applying a submission technique, it isn't quite working (yet?), should you stay on it, or should you move on to something else?

How often you can answer that question correctly will dictate how far you go, and if you are to become a champion in the sport.  That is the idea of an edge.

How do you know?  You trained thousands of hours, maybe tens of thousands of hours for this moment.  Should you raise?  Should you shove?  Your choice will dictate whether the binary outcome collapses to defeat or victory.

Your coach can scream from the corner anything he likes, it's you who has to decide.  Your coach can't experience what you are.  Your coach can't see that your grip is better than he thinks, that you believe it's going to work.

Then your grip slips.  

You gassed out, and now you're in trouble, you get submitted.  You lose.

"Why didn't you listen to me?"  Your coach insists.

You explain the grip was better than he realized, it was going to work.

But he assures you it didn't work.  You lost.  And dismisses your perspective.

But you're correct.  You were in the arena, not him.  Right?

You'll never be able to play re-play that fight simulation style thousands of times to determine which of you is correct.

That's where ego, altruism, and the notion of independent creation comes in.

You are at a crossroads after this loss, do you listen to yourself or your coach?  

Maybe your entire team is telling you that you should've dropped the submission.  Maybe they're even a great coach and team, and they usually have your back.  But this is empiricism, you are the only one that experienced that.

Your choice will dictate whether you make progress or not, and if the next time around you experience a similar state your policy function selects the correct action yielding the desired outcome: victory.

Of course, you'll never know for that fight.  You can't replay it.  But you can experience more fights, and put yourself in similar states.  

That is how you will learn how to be correct.

Notice, it won't be binary.  This requires constant active learning and experiencing.  It is not easy, in fact, it's quite painful and requires a lot of losing.

You have to put down your ego, and other times, pick it up.  

You have to try new things, you have to test if you're correct, you have to be creative.  A good coach will have your back, and happily watch you lose.  A bad coach will scream at your for not listening to him, time and time again.

But having someone's back is not a vacation home, you can't show up when you feel like it.  You either do or don't while they try to figure it out.

You can only sincerely have someone's back if you fully believe in them, if you believe they are trying their hardest.  Otherwise, you'll plague them with "why did you do that?" or "that wasn't a good idea." after the fact...

I was watching a retrospective the other day on Sam Darnold from the Seahawks after winning the Super Bowl; the Coach and team were discussing how if you're gonna have a confident gunslinging quarterback you better have his back when he throws an interception.  You don't get it both ways.

If you're the gunslinging quarterback, and your team doesn't understand this, you're on your own.  How can you possibly optimize your policy function in this?

Well, you certainly can't subvert your own experiences, thoughts, ideas for your coach or team if you believe they're incorrect.  This is easier said than done.  

Brotherhood and the social construct of a team is one hell of a drug.  You'll constantly be the contrarian if nobody else believes in you, and that sucks.

But being the contrarian, believing in yourself, is how you will make progress, and how you will force yourself to find a team that actually has your back.

Having some mental scaffolding now, how do I use this framework to optimize my policy function?  Obviously my goal is to continue to learn and grow, but when do I reject my previous assumptions in favor of new ones?  

When do I accept that I'm wrong and move on with my new objective truth?

It's a dynamic process, it changes all the time.  But due to a substantial number of requests to articulate my process I've done my best below.

To make optimal decisions, I leverage a few highly effective tools:

1. A Researcher's contrarian perspective
2. Wisdom of the crowds
3. Law of large numbers
4. Rejection of altruism and ego
5. Undying faith in myself

Here's how they all fit together...

1.) A Researcher's contrarian perspective

If something or someone can't stand basic criticism, it's a house of cards and unlikely to hold in practice or stand on their own.  It's a simple test, it may not win you many friends, but you want quality not quantity.  It'll win you the friends with the same curiosity and desire for progress.  Friends that have your back.

2.) Wisdom of the crowds

Wisdom of the crowds is breadth first not depth first, but let's see what everyone else thinks.  Is it rational?  Is it correct?  Maybe.  That's why I even bother to look here, this can be a benchmark.  It may even tell us how we can go against the grain to achieve a more desirable outcome.

3.) Law of large numbers

The more reps you do the better you'll become.  Asymptotics for processes that change over time in the classroom still hold, and it's no different in practice.  This is the real law of large numbers.  If I do 100 reps and the next 100 aren't like the first, am I going to sit idly by and do nothing?  "guess that stopped working, oh well"... No I have to update, I have to try to fix it, I have to see what went wrong and why.  Then I can achieve more desirable and similar results the next time around.  That's real statistical estimation, that's Bayesian updating.

4.) Rejection of altruism and ego

I refuse to let another man occupy a moment of my mind.  I will not subvert my subjective needs, or what I believe to be correct.  I will put my oxygen mask on before attempting to help others.  Too many in the western world are preached to about being arbitrarily kind: "don't be selfish".  Too many are walked all over, they "feel bad", we should be raising good men.  Society will never champion what is best for the individual, it is a social herd.  In rejecting this, I refuse to let my ego dominate when I must surrender and update.  But I also refuse to fall like a house of cards.  Some ego is necessary, otherwise you can be convinced it will rain gumdrops and marshmallows.  You must stand firm, but willing to look stupid standing your firm ground.  Necessary for progress, sometimes painful socially, trust me, you get over it.

5.) Undying faith in myself

I never act with malicious intent.  Given this truth, anyone can say anything they like about me.  The truth is binary and objective.  I will never stop because I know this truth exists, and I believe in my framework until I have reason not to where I will surrender and update again.

This ideology is what allowed me to create Quant Guild.  Nobody believed in it.

None of my cheerleaders, yet here we are.

That is empiricism.  I experienced something my "coaches" didn't and I kept iterating and updating.  Now I am grateful to share these letters with my community every week and offer lectures and edutainment on my YouTube channel.  I wouldn't have it any other way.

Life was hard for me as a Quant on W2.  The idea of justifying my existence to an employer made me sick.  Creating for them made me sicker.  Egotistical?  

Probably.  But now I only exist to create for my community.  I will never have to look back and wonder, I'll never fall in line, and I practice exactly what I preach.

I wouldn't be capable of organizing my thoughts in this manner if it weren't for a friend recommending The Fountainhead, a newfound pillar in my way of thinking alongside Meditations and Discourses.  

I'll die before I look anything like Peter.

Rather than being paralyzed with choice, it is oftentimes easier to paint a beautiful and vivid image of precisely what it is you don't want, Jordan Peterson's idea of a personal hell, then running as far away from that as possible.  Now that's motivation.  And it removes paralysis of choice and commands immediate action.  You won't get it right on the first try, but the sooner you start, the further along you'll be.

Regardless if I am to one day sit on a golden throne or die in a cardboard box, I'll happily accept my fate.  Because I am trying my absolute hardest, and this framework is just one tool in my toolbox to accomplish and articulate that.

The media loves to point at the failures.

- Logan Paul's XYZ that FAILED MISERABLY.

- Donald Trump's ABC that FAILED MISERABLY.

- [insert person here that failed at something and the media crucified]

People becoming symbols to a social slaughter, because it drives clicks.

Terrific.

But they're trying their hardest.  It's the only reason they receive any attention.

Call it egotistical.  Call it cruel.  Call it whatever you want.

But you have literally no idea what they are trying to accomplish.

BuT rOmaN THeY sAid "XYZ".

Yes, and that's what they wanted you to hear.  You know nothing of the covert contracts they hold with themselves or others.

You've walked 0% of their path.  A clip you've seen explains nothing.

Has nobody read The Death of the Author?  It's almost laughable.

TL;DR

In the words of Epictetus, are you going to sit around and complain that your nose is running?  Or did God give us two hands for a reason?  Isn't it easier to just wipe your nose?  Thanks Arrian.  

That's to say, get out there and experience.  Learn, iterate, and make progress.

With that I will leave you to the Weekly Guild Letter.  

I hope you enjoy, and I hope you learn something!

- Roman