Post FI Notes 014: $20 Million with a Unique Asset Allocation

Today’s interviewee was certainly good at earning money, and over a lengthy career as a doctor and practice owner, he made a lot of it.

Admittedly, though, he wasn’t very good at investing that money, always trying to beat the stock market and routinely failing to do so. His struggles with stock picking led him to embrace alternative investments, and some of them have given him outstanding returns.

As we often see, you don’t have to do everything just right to amass significant wealth. The most difficult question for him to answer was the one where he was asked to choose his worst investment, and he claims to have so many to choose from!

Yet, today as a retiree, he and his family spend multiple six figures a year with a sub-2% withdrawal rate.

If you’re interested in participating in one of three interview series, please download the most appropriate form for your life situation: FIRE Starter, FIRE Crossroads, or Post-FI Notes. To see other posts in the series, visit our Q&A archive.

 

post-fi-notes--014

 

Getting to Know You

 

You’re financially independent. About how much does your household spend in a typical year? How much could you spend while still abiding by the 4% rule?

We spend approximately $300,000 yearly. This has recently increased a bit because our oldest child (of 3) just began (a private) college this past year.

He has a partial scholarship, so we are drawing approximately $30,000 per year from his 529 plan, so I included that amount in our yearly expense. Our net worth is approximately $20,000,000 so we’re well under the 4% rule which would be $800,000 of yearly expenses.

 

Tell us about your household. How many people and at what ages? Are you supporting anyone outside of your home? Where do you live?

Our household consists of myself age 71, my wife, age 51, and three kids, one in college and two still in High School.

My wife doesn’t work, so I now share household duties with her since I retired one year ago. These duties include being the family chauffer for our youngest, which is actually quite nice, as it’s the time where we get to chat a bit without all the distractions of a busy household.

In the past, I supported my mother when she moved to assisted living, but she’s now deceased, and my wife’s parents are self-supporting, so we don’t any longer have any expenses beyond those for our nuclear family.

 

Are you still working? In what career? Did your work schedule or attitude towards work change once you knew you were FI?

I was probably FI in theory quite a while before I retired, but I was too busy working to really notice, and I also didn’t have the time to invest properly. Thus, I say I was FI “in theory” since I had the savings to create cash flow if it were properly invested.

The problem was, it wasn’t properly invested, so I didn’t have the retirement income required. I have always been self-employed, so I would have no pension income beyond what I created on my own.

Thus, as retirement approached, I tried to focus more on learning how to invest to create the retirement income stream we needed given the fact that I had a relatively young family with the higher expenses one would expect for that stage of family life.

At the same time, even before COVID, I had started to experience burnout, and I knew I didn’t have it in me to work much longer, so I also began to try and find a buyer for my small physician practice and was fortunate to sell the practice at a 2.5X multiple, as the practice netted about $1 million annually.

With this sale, I had the confidence to retire while continuing to work on configuring our portfolio to produce both income and growth.

I had purchased several limited partnerships prior to retirement and before I really know what I was doing and paid the price of now being stuck with a few underperforming and non-performing assets.

Fortunately, my education really took off when I discovered an online investment club (it’s called “The Private Investment Club”; you can search for it if you’re interested). This club’s focus is on teaching about and group vetting of investment opportunities, and the tone of the club is kept very professional and collegial.

 

Was financial independence a long-term goal of yours? Did you think you might retire early or be able to do so when you first got started in your career?

When I left academia early in my career and went into private practice, I had the naïve vision of an early retirement.

Soon, I realized that not only was I not interested in Lean FIRE, but that I wouldn’t know what to do with myself if I did retire early, so my goal instead became to just focus on my work, knowing I’d have more than enough to live on once I decided I was ready to retire based upon non-economic factors.

 

Investing

 

How is your nest egg invested? Approximately what percentage is allocated to stocks, bonds, real estate, and alternatives?

It might be best to show you a pie chart of my portfolio allocations first, and I can then go into some of the details.

 

Post-FI 014 asset allocation

 

I think the most noteworthy aspects of my asset allocation are that:

  1. I have a really quite small allocation to public equities of only 8% or about $1.2M.
  2. I have no bonds currently.
  3. The largest component of my portfolio is real estate (31%), predominantly multifamily value add, but also heavy value add and new development.
  4. I have a quite large allocation to venture capital (18%) and private equity (3%), something I felt I could safely do once I had created an adequate income stream that was recession-resistant.
  5. I have a relatively high allocation to cash (9%), but this is due to the fact that some of my investments call capital over time, so I need cash reserves for that purpose.
  6. My gold & silver allocation (2%) is actually in public small-cap stocks, but I broke this category out from general public equity due to its different characteristics and purpose within the portfolio.
  7. Non-correlated investments of 8%, discussed below.
  8. Rounding out the portfolio is 4% in cryptocurrency, 6% in an oil investment, and 11% in income-producing “hard money” and corporate loans.

 

I should probably also comment upon my overall quite small allocation to public equity of only 8% (not including my gold/silver stocks). I have found over the years that I do best with those investments I cannot touch.

But, I just can’t leave my stocks alone, always looking for some system to beat the market. All that has achieved for me is sub-par returns consistently. My alternative investments in contrast have been consistent performers for me.

From a risk perspective, I see no downside to being overweight in alternatives since these alternatives are well diversified. In particular, I also have 8% in “non-correlated” investments. These are investments like litigation finance, life settlement funds and music royalties.

These investments have proven over time to have no correlation to the economic cycle, and so would provide a buffer from losses in any investments sensitive to the economic cycle. In the meantime, I’m working on addressing my hubris in thinking that I can beat the market if I only found the right system!

 

Are your investments primarily in tax-deferred, Roth, or “taxable” post-tax accounts?

My pie chart includes both our corporate 401(k) assets and post-tax accounts; the current value of the 401(k) is approximately $6,000,000 and includes a Roth 401(k) which contains a single asset currently valued at approximately $1,000,000.

This Roth investment was a Venture Capital investment, transferred to the Roth before it began to appreciate, and thus the transfer only created a small tax liability. The post-tax accounts total approximately $11,000,000. The remainder of net worth is home value (as our home mortgage is paid off) and 529 plan assets.

 

Do you have investments in an HSA? How about 529 Plans?

We have no HSA, as all medical expenses were paid prior to retirement by our medical C-corporation as allowed medical expenses.

We do have a 529, and decided to be ultra-conservative and utilize “The Private College 529”. This 529 allows one to deposit tuition credits at today’s prices, so therefore has an approximate 7% return (as the effect of yearly tuition inflation), but the downside is that it doesn’t cover any non-tuition expenses.

For the 3 kids, we accumulated over the years about $650,000 which, with a few scholarships, should cover tuition for each of them and any additional funds in the plan can be withdrawn to be allocated to room & board expenses.

 

Incrowd_surveys

Physicians and pharmacists, Register with Incrowd before 2021 ends to qualify for 1 of 2 $150 Amazon gift cards to be awarded to new registrants referred by Physician on FIRE in December.

You'll also get the opportunity to earn money with quick "microsurveys" tailored to your specialty.

 

What has been your best investment?

At the moment, my best investment was actually made before I retired; I started fooling around with cryptocurrencies in 2017, investing about $40,000 in various currencies before the big crypto bubble.

This crypto portfolio appreciated over 6 months by 80X! I also, through absolute sheer luck, completely divested one week before the bubble burst. By divesting one week early (before the absolute top), I lost about $1,000,000.

At the same time, divesting 1 week before the top also saved approximately $2,500,000 which would have been lost if I rode the positions all the way back down to the bottom that occurred with the “crypto winter”.

My total gain was $3,200,000. Watching a portfolio that lost and gained up to $300,000 in a day was a roller coaster ride that I plan to avoid if at all possible in the future! The problem I confronted after the divestment was that almost the entire capital gain was short-term.

This left me with the need to protect the gains to the extent possible from capital gains taxes of about 60%. Opportunity Zone (OZ) funds were brand new at this point in time, and no OZ limited partnerships were yet available, so I put $1,000,000 of the gain into an oil exploration limited partnership.

With the perverse taxation rules for oil exploration, this provided shelter for almost the entire investment of $1,000,000, so my tax burden was lowered by about 1/3. Now, OZ investments are readily available and are a much safer means for shielding capital gains.

In fact, I used OZ funds extensively to protect the gains from the practice sale, as the sale was configured such that 80% of the gain was characterized as sale of “goodwill” which is taxed as capital gain.

The residual 20% was “sale of assets” which is taxable as income. Thus, the funds from the sale were very efficiently converted into real estate assets, producing both appreciation and an income stream for at least the next 10 years.

The cryptocurrency investment might ultimately be rivaled by a current seed investment in a cryptocurrency mining company. My initial experience with cryptocurrency made me quite comfortable making this sort of investment.

The company was to IPO just as the crypto market fell at the end of January, so the IPO was cancelled. If/when conditions are appropriate for an IPO, I’ll be able to value this investment which can then be sold 6 months after the IPO.

This is an investment that was group vetted by the club I mentioned earlier, the Private Investment Club, and we actually felt that given some unique characteristics of this company, it was a relatively safe investment given that it is, after all, crypto!

 

Your worst investment?

I, unfortunately, have a more difficult time identifying my worst investment, because there are so many to choose from! However, sitting at the top of the trash heap of failures is an investment done through an insurance policy into a hedge fund.

This was done long before I learned how to do due diligence, and I just followed the lead of a salesperson making large commissions from my account. Almost immediately after I invested, the SEC froze the “hedge fund’s”  accounts and began investigating the two principals.

It turns out that instead of investing my funds in the stock market, it was invested in a hotel development they owned after their plans for the hotel went south during the 2008-9 real estate collapse.

Long story short, they are both in prison for many years to come, and about 8-10 years after my investment, I received back 60 cents on the dollar when the Dept. of Justice appointed Receiver was able to sell the hotel and give the proceeds to me and the other fleeced investors.

So, very expensive but useful lesson-it is SO easy to be fooled so no matter what you are told by a sales team, and how sharp you think you are. So do your own independent due diligence! And if you can’t find enough information in order to compete that due diligence to your satisfaction, DO NOT invest no matter how appealing the investment seems to be.

 

Your next worst investment?

A follow up to my worst investment is, embarrassingly, whole life insurance.

I thought I knew all about why whole life is bad and to be avoided at all costs. So, then why did I succumb and purchase a policy? Because I followed the recommendation of a financial newsletter  pushing whole life as an “infinite loan”.

This is a variation of a typical whole life policy that minimizes the life insurance benefit (and thus cost), and which is overfunded, along with a few other changes that overcome the well known downsides of  Whole Life. You supposedly rapidly overcome the first year commission, and then build up principal.

Unfortunately, in an increasingly low interest environment, that build up lagged significantly from projections, and so after about 5-6 years I finally decided to terminate the policy, getting back about $700,000 of the $800,000 I originally invested.

Fortunately I exchanged the policy into a Vanguard low fee annuity, and thus over the following year, regained my losses on a tax free basis, so that ultimately what was lost was the opportunity cost of $800,000 over 5 years.

 

Start receiving paid survey opportunities in your area of expertise to your email inbox by joining the Curizon community of Physicians and Healthcare Professionals.

Use our link to Join and you'll also be entered into a drawing for an additional $250 to be awarded to one new registrant referred by Physician on FIRE this month.

 

Post-FI Life

 

What do you like to do with your free time? How much free time do you have these days?

Well, once retired, theoretically all my time is free time. I joined two book clubs and read a book every two weeks. I’m very proud of the fact that after reading so little during the years of my practice, I’ve been able to read 25 books over this first year of my retirement.

I also joined a local investment club which has been quite enjoyable. I spend a lot time vetting investment opportunities along with my fellow Private Investment Club members (this is a different club than the local club I just mentioned).

I also spend a lot of time working out, and have put lots of miles in on my Peloton! Finally, I get to spend time (that is when and if THEY have the time) with my kids before they’re all away in college-probably the greatest benefit of my retirement!

 

Do you enjoy travel? Tell us about a favorite trip you’ve taken.

Since two of our three kids are still home, our travel is mostly local or semi-local, though this year we’ve done two week long college reconnaissance trips with our middle son.

I imagine once all three of them are in college we’ll again be able to travel as we both did when we were younger, and look forward to doing that.

 

If retired, do you miss work? Do you get bored?

I really don’t miss work at all. I noted earlier that I felt I was burned out and just ready to stop. I had several offers to work in more limited, lower stress positions, and to do consulting, but I just didn’t feel motivated enough or interested enough to go in that direction.

Now, a bit over one year later, I can say I’m very comfortable with my decision and overall content.

 

What advice do you have for others hoping to achieve the financial success you’ve found?

I’ve learned a lot more from my mistakes than from my successes. I’ve mentioned a few earlier, but to recap:

Never buy Whole Life insurance, no matter what it is called and how it is sold.

If you invest in alternative investments (but maybe in some cases even if you only invest in individual stocks) learn how to properly do the due diligence process, and NEVER invest without completing thorough due diligence.

Don’t be afraid of alternative investments.

Don’t feel bound by formulas you’ve read or by advisors you have listened to telling you that most of your investments should be in public equities. Properly selected alternative investments can produce low volatility income and appreciation while allowing you to sleep well at night.

Due diligence is an art form that is not impossible, but is difficult to learn on your own-seek out the help of peers or mentors who can help you on the journey. Once you understand the process, it’s a tool that you will have for life and your investing will never be the same.

 

 

PoF: Catch all the future interviews from those just getting started, at a crossroads, or at the end of their FI journey with a free subscription to Physician on FIRE.

 

 



 

I thank today’s interviewee for sharing their story, and I’ve shared my feedback privately with them. I wouldn’t want my opinions to influence yours. Please give your take and answer any questions they have had in the space below!

Again, if you’d like to partake in a future Q&A, please download a FIRE Starter, FIRE Crossroads, or Post-FI Notes interview form.

 

19 thoughts on “Post FI Notes 014: $20 Million with a Unique Asset Allocation”

  1. Subscribe to get more great content like this, an awesome spreadsheet, and more!
  2. You stated that joining the PIC is free but when I tried to join it states there’s a $10,000 annual fee? Do they invest the 10k for you or is that just the membership fee?
    From the website below:
    I understand the annual fee to join as an Elite PIC Member is $10,000 with some special bonuses.

    Reply
  3. I ran across the Private Investment club on the fat fire subreddit . Do you feel it is a significant value add and how much time does it require?

    Reply
    • Hi Ryan,
      As I implied in my post, the club completely changed my approach to investing. It taught me how to evaluate real estate and business investments. It also presents to members investments not otherwise available to any group other than institutional investors such as pension plans and private offices (because group members invest together in “feeder” funds so that we can meet the high minimum investments accepted by these funds which is often $5M). It is also free to join. You are vetted however, and you have to sign a Non-Disclosure Agreement which is enforced, and some members have been ejected from the club for violating the club rules. However, these rules make the club safe and effective for the rest of the members, and keep the discourse both goal directed and polite. You can invest as much or as little time as you’d like. Some members are just lurkers, while others are very involved. The knowledge level varies quite a bit. We are large enough now that there are experts in pretty much every area who can contribute their knowledge. However, even if you are new to alternative investments, there is a ton of material available that will slowly bring you up to speed. What’s so great about the club is how willing members are to share what they know and take the time to educate others.

      Reply
  4. I’m not anywhere close to as wealthy as you, and with a daughter heading to an expensive private college next year, I didn’t bother to fill out the FAFSA, as I determined there’s no chance I’m eligible for any aid. How does a family with this much wealth receive a college scholarship?

    Reply
    • Don’t confuse scholarships for grants. Many scholarships aren’t need based. There’s academic scholarships, sports scholarships. Most merit scholarships are given out for academic performance, for example the one I got was due to a combination of testing scores and high school performance. Some schools offer more than others.

      Reply
    • Hi Josh,
      You’re right, we’re not expecting any scholarships based upon financial need, but our eldest was offered a merit based scholarship we didn’t even apply for! We’re hoping our other two might get merit based scholarships as well, but certainly not counting on it.

      Reply
  5. Congratulations to all your success after years of hard work. My question to you is what are you going to do with all the pretax money, as you seem to have at least $5 M, does RMD push your tax bracket to the max? Thank you

    Reply
    • Hi Hospitalist,
      Thanks! Our RMD will be moderated because I plan to roll over the 401K Roth assets into a Roth IRA which has no required distributions. I don’t have to take distributions until next year (age72), so have one year to accomplish this. This roll over will remove $1-2M from the 401K. Also, because my wife is a lot younger than I am, our RMD divisor is larger, so our RMD will only be about $100K/yr. If I had to use the divisor base upon my age, and if I couldn’t roll over assets into a Roth IRA, then the RMD would be more than double this amount.

      Reply
  6. Your tax savings strategies (sell of cryto, rolling your whole life into annuity, sell of your practice) is quite impressive. Did you work with a tax strategist, accountant or other? any recommendations on how to seek out such a person?

    Reply
    • Hi B,
      Getting consistent good advice isn’t easy, and doesn’t come from one source. For example, I got the suggestion to roll over the whole life into a low fee annuity by a financial advisor that I did a one time consultation with. In that consultation, I realized he wasn’t the appropriate person for me, but that one suggestion was quite helpful. I have had the same CPA for 30 years, and he has been helpful with specific taxation questions, but has been less helpful with long term planning (though he did refer me 30 years ago to a pension plan administrator to establish a Defined Benefit Pension Plan, which about 5 years ago was converted to a 401K. In my opinion, being the Trustee of your own pension plan is extremely important to accomplish if you have your own medical corporation, as it gives you the ultimate flexibility to invest (as you do with your after tax funds), yet with none of the tax considerations. I also have an entrepreneur relative who’s “hobby” is designing tax strategies, and he suggested selling most of my practice as “goodwill”. I’ve looked my entire career for one individual or team that could design/maintain for me an overall strategy, but never found such a person, and don’t think such a person exists unless you are willing to hire a portfolio manager that charges 1-2% annually to manage the entire portfolio, and I wasn’t willing to either spend that much, nor to give up control to that extent. Finally, my fellow members in the Private Investment Club often discuss tax strategies, and this has often been helpful. Joining that club (it’s free, though you are vetted before you can join) was the most important and impactful economic decision I ever made, and really replaced any need for a hired portfolio manager.

      Reply
      • You stated that joining the PIC is free but when I tried to join it states there’s a $10,000 annual fee? Do they invest the 10k for you or is that just the membership fee?
        From the website below:
        I understand the annual fee to join as an Elite PIC Member is $10,000 with some special bonuses.

        Reply

Leave a Comment