FIRE Crossroads 020: Should They Pay Off the Mortgage?

As a former military doctor, he’s familiar with hard work. And now, despite having a generous pension and enough assets to potentially retire any day, he seems to be working harder than ever.

Their expenses are low, income is high, and he’s wondering if it’s time to eliminate the mortgage on their $1.3 Million home.

After reviewing their complete picture, you may be inclined to advise on other situations, and in these interviews, unsolicited advice is fair game. I’ve shared some of mine with our guest, whose story follows.

If you’re interested in participating in one of three interview series, please download the most appropriate form for your life situation: FIRE Starter, FIRE Crossroads, or Post-FI Notes. To see other posts in the series, visit our Q&A archive.

 

 

Getting to Know You

 

Where are you on your financial independence journey? Have you crossed the halfway point in terms of net worth and/or passive income?

Although I’ve more than crossed the halfway point to my goal of $3 Million in investment assets and a paid off house, I feel that we’re at a crossroads in deciding whether we should pay off our house or not.

 

Tell us about your household. How many people and at what ages? Are you supporting anyone outside of your home? Where do you live?

My wife and I are both 52. Although my wife is a trained physical therapist, she’s stayed at home with our 2 boys until recently. Our oldest, 20, lives at home and goes to a state school nearby. Our youngest, 18, is now a freshman in college, also at a state school here in WA.

My wife is now in her 2nd year of an accredited brewing and distilling program at a local community college (I’m pretty jealous about this, but she totally deserves it). [PoF: This is awesome.]

I also retired from the Army 5 years ago, and am fortunate enough to receive a pension plus disability.

Although we’ve lived all over the country while I was in the Army, for the past decade we’ve lived in the Pacific Northwest, despite the fact that my wife and I are both recovering New Yorkers.

 

In what field are you working? How is your career going? What do you like best and least about your chosen profession?

I’m a full-time pediatric subspecialist concentrating on inpatient ICU work and inpatient hospitalist work. I work as a full-time contractor at one local hospital, and I have 2 different part-time W2 gigs at some other nearby hospitals, primarily covering in-house nightshifts.

I enjoy taking care of children and being able to reassure parents during their darkest hour.

 

Do you feel you’ve come to a crossroads of sorts? If so, tell us about it. What options are you contemplating?

I currently feel at a crossroads because I COULD pay off our mortgage using our taxable investment account, and I COULD potentially retire (or give up one or more of my current jobs), but I’m not sure if I have enough of a safety net to do so. Our monthly expenses (without the mortgage), are roughly $5,000 per month.

I hesitate to pay off the mortgage, primarily from a liquidity standpoint- I definitely wouldn’t have enough to put a down payment on another home if we were to decide to move without making a contingency for selling our current home, or much liquidity for any major expenses as they arise.

My current positions: Full time contractor at a military facility: The pay is much better than when I was active duty military, but as a 1099 position, the contract can be discontinued any given year. To offset this risk, I work a 0.6 FTE as a pediatric hospitalist, primarily seeing healthy newborns as well as newborns requiring care in a level 2 nursery.  It’s less fulfilling than PICU work, but only requires four 24hr shifts per month.

This position also provides short and long term disability insurance, as well as additional life insurance for my wife and me. I also have another W2 position where I provide weekend phone coverage for a subspecialty service.

In total, I gross a little over $650,000 per year, with roughly $360,000 from the contracting position, $130,000 from the 0.6 hospitalist position, and about $100,000 from retirement plus 100% disability from the VA.

In the 5 years since I retired from the Army, we’ve been able to max out a solo 401(k) using employer (me) contributions, as well as a regular W2 401(k). Prior to that, we never made it a priority to maximize retirement accounts, a mistake that I’m paying for by working like crazy now.

 

Investing

 

How is your nest egg invested? Approximately what percentage is allocated to stocks, bonds, real estate, and alternatives?

 

The majority of our holdings are in retirement accounts, spread across a solo 401(k) ($400,000), a regular 401(k) ($400,000), Roth IRAs, ($250,000), and regular IRAs ($300,000). All of these retirement accounts are primarily in index funds, with a ratio of roughly 90% stocks and 10% bonds.

We keep an emergency fund of $75,000 in savings and have an taxable investment account of $400,000 that is invested primarily in individual stocks, although no single stock is more than 5% of that account.

Finally, the only real estate is our primary residence, with a $350,000 mortgage, currently at 2.25% 15yr. Our property is currently valued at $1.3 Million.

 

Are your investments primarily in tax-deferred, Roth, or “taxable” post-tax accounts?

The bulk of our net worth is tied up in tax-deferred accounts. In hindsight, I wish I’d concentrated more on accumulating more wealth in our taxable investment account, since I can’t access our retirement accounts for another 7 years. [PoF: That’s not entirely true…]

 

Do you have investments in an HSA? How about 529 Plans?

We don’t have an HSA or any 529 plans. I’ve been fortunate enough that we can pay for all of our healthcare needs through the military pretty inexpensively, and my VA benefits pay for the majority of undergrad studies for both of my boys.

 

What has been your best investment?

One short-term win was with bitcoin. I doubled my money within a few months, allowing me to buy a nice car with cash.

Otherwise, I’d say my best financial choice was re-training as a pediatric intensivist (I was already a peds pulmonologist for some time prior to that). That decision opened up many more doors in the region for me to work at other positions.

 

Your worst investment?

I unfortunately bought Variable Universal Life Insurance accounts for both my wife and I early in my career. We luckily figured out not much later that this was a bad idea, and sold out of them.

 

Into the FIRE

 

Numerically, what is your FI goal?

My goal is fairly conservative- $3M total in investment assets and a paid off house.  Although on paper, I could pay off the house and retire now, living solely on retirement and disability, I’d prefer to have more breathing room and an ability to pay off any large surprise expenses as needed.

 

When do you suspect you will achieve financial independence? Will you retire from your career once you’re comfortably FI?

I believe that working at my current pace, I should hit all of  my goals in the next 5 years.

Once I reach my goal, I’d like to fully clinically retire. I might keep a part-time position for awhile just to stay active and see how a trial run of semi-retirement goes.

 

What are your post-FI plans? How will your life change? What do you look forward to the most?

I most look forward to reliably sleeping in my own bed. I also look forward to traveling- we’ve been fairly frugal as far as vacations go the past few years.

I’m also looking forward to some small-scale distilling first as a hobby, and then potentially as a small business with my wife.

 

Have you made any major changes in your lifestyle or investments to accelerate your FI path?

I’ve tried to accelerate my FI path by taking on extra gigs, although juggling multiple calendars can get pretty painful. I’ve also done a fair bit of per diem work in Montana, but the travel wasn’t very efficient, and being away from home for a week at a time wasn’t really worth it.

As far as investments go, I took control of our after-tax investments early this year and have been actively accumulating long positions in primarily US stocks.  Although I’ve been doing ok, it’s probably not worth the stress of trying to actively manage this account.

 

Incrowd_surveys

Answer quick MicroSurveys for cash. Designed with convenience and timeliness in mind, 70% of surveys are answered on a mobile device in just a few minutes.

Physicians, Pharmacists, and other healthcare professionals are invited to join Incrowd today!

 

Are you facing any unique challenges making FI or RE more difficult?

Some days it feels as though I’m one day further away from retirement and one day closer to death (morbid outlook, I know).  I certainly feel the clock ticking on how much longer I can sustain working the hours that I currently work.

The primary reason for me pushing myself the way I do is that I don’t know how much time I’ll have in retirement due to several medical conditions. I also know that when I do die, my wife will only have 55% of my pension and none of my disability, so I’d like to have a better financial cushion.

 

What advice do you have for others who are seeking financial independence?

I wish I had a time machine to go back and knock some sense into younger me- put whatever you possibly can afford into retirement accounts as soon as you can, and get your children to open up Roth accounts the minute they can push a lawnmower or babysit.

Other advice- rather than taking out loans for med school, the military CAN be a good option for some, especially for Surgery, Radiology, Emergency Medicine, and Anesthesia. The newer blended retirement plan the military has is less attractive than the old 50% of base pay after 20 years model.

Unfortunately, the pay is typically not competitive with the civilian world (especially for traditionally higher-paying specialties), so the smart move is likely to rejoin the civilian workforce as soon as you’ve paid back your training commitment.

I was fortunate enough to have the Army sponsor my training at some fantastic civilian institutions, but I’ve also had to deploy to Iraq for a year at a time twice in my career to work with Soldiers in combat units.

 

Finally, is there anything under the sun that you’d like some help with? The hive mind would be happy to weigh in.

I’d be interested to hear folks thoughts on whether they would pay off their primary residence in this situation, and then work to build up the after tax account all over again.

I’m leaning towards a mixed approach of simply throwing extra money at the mortgage whenever I can, and then pay off the house once I have double the amount of the mortgage in hand.

 

 

PoF: Catch all the future interviews from those just getting started, at a crossroads, or at the end of their FI journey with a free subscription to Physician on FIRE.

 

 

I've got my 2 acres of non-leveraged, crop-producing, cashflowing farmland via AcreTrader. Get yours.


 

I thank today’s interviewee for sharing their story, and I’ve shared my feedback privately with them. I wouldn’t want my opinions to influence yours. Please give your take and answer any questions they have had in the space below!

Again, if you’d like to partake in a future Q&A, please download a FIRE Starter, FIRE Crossroads, or Post-FI Notes interview form.

 

15 thoughts on “FIRE Crossroads 020: Should They Pay Off the Mortgage?”

  1. Subscribe to get more great content like this, an awesome spreadsheet, and more!
  2. Use index funds in the taxable account to make your investing life 10x easier.

    Doesn’t sound like you need them, but SEPP will allow you access to your retirement accounts early without penalty.

    Cut back at work and stop burning the candle at both ends. You don’t “need” the money from work and you are trading a lot of your time for money. At a certain point you should stop trading your time for money and spend more time on the things that are truly important to you.

    I’m about 2 decades behind you, but my wife is already cutting back at work to find a better balance that leads to more happiness now. Nothing is guaranteed, and we are trying to find the most overlap in our Venn diagram between “Our Life as It Is” and “Our Life as We Want It to Be.”

    Congratulations on your success and thanks for your service!

    Reply
  3. My advice is pretty simple.

    1) Start working less…..this is insane. You’re too wealthy to not sleep.
    2) Hold off on paying off the mortgage….keep the liquidity. You need less stress
    in your life.
    3) Try to shift from individual stocks to indexes…..less stress and standard
    deviation.

    You are in much better position than you think. Need to remember that life is
    a marathon, not a sprint.

    Reply
  4. Start receiving paid survey opportunities in your area of expertise to your email inbox by joining the Curizon community of Physicians and Healthcare Professionals.

    Use our link to Join and you'll also be entered into a drawing for an additional $250 to be awarded to one new registrant referred by Physician on FIRE this month.

  5. Chiming in from the perspective of a spouse to someone with a medical condition that will likely limit our time together later in life: time spent together, making memories, is far more valuable than another $1 million.

    Reply
  6. I would only pay off a sizable mortgage if there is a probability of living in the home during retirement. I retired from the Navy after 24 years; downshifted by working as a clinical professor three days a week for three years, followed by one day a week for four years. I completely retired at age 57 during the Great Resignation. Current passive income is $280K/year. mostly from net rental income and pension. Our burn rate $120-150K/year is due to a moderate budget for travel. Net worth including primary home is $8M. We are not withdrawing from our TSP, index funds, Roth and 401K ($1.5M). Our home is located one block from the beach is San Diego, making for a great retirement location. We paid off the home four years ago because: 1.) Mortgage debt should be extinguished in retirement. 2.) As long as you have a mortgage you are not a homeowner, but a co-owner with the lender. 3.) Mortgage interest deduction is not significant during last 1/3 of the life a mortgage loan. 4.) Extra cash flow can be used for travel or investments. Based on your numbers, you are following the same track as me, ready to fully retire in five years. The big question now is do you want to retire in your current home or move to a warm climate like Hawaii or Florida?

    Reply
  7. (Retired military doc here, also. Recently FIREd).
    It sounds like you are a workaholic: a 1099 job, two W2’s, and some per diem work. Financially, the numbers suggest you are fine with achieving all of your goals, if not there already then really soon.
    If you are not a workaholic, then I suggest that you give up one, or 2, of your job positions that are least pleasing in order to have more personal/family time. This will also prevent burnout or worse.
    With your military/disability pension, you can afford to lower your emergency funds or have none at all since this is guaranteed regular paycheck for the rest of your life.
    Paying off your mortgage can go either way, but if it is not your forever home then I say don’t. Some people do feel tremendous relief after paying off, so if it is really weighing on you heavily, you can still consider paying off at any time.
    You’re doing great financially. A 20-year military medical career with retirement affords you to practice medicine the way you want – or not at all. Bravo Zulu!

    Reply
  8. My thought is that you are in much better shape (financially) than you are giving yourself credit for. Go lookup the value of your pension (calculators on the internet). I think you’ll find that number plus your existing investments put you well past your $3M goal. You could easily give up 2 of your 3 jobs, continue to pay down the mortgage, tap your retirement accounts using the SEPP(72T) rules when ready, and enjoy all the newfound free time you have.

    Reply
  9. Wow- I am amazed how much we have in common!

    I too am a 5 years out retired military physician working in the civilian world. We also have been trying to decide about paying off our mortgage (2.5% 30year fixed, $400k balance) or not. We have ultimately have decided not to do so for now as we want to maintain flexibility and not lock more funds into our mortgage. The home we currently live in is probably not our retirement home and so we would like to be able to perhaps buy something else that is smaller, more energy efficient, with less maintenance without having to sell our current home first. We may also keep our current home as rental as it is in a prime public school district and should have strong demand and value. For now, we are just paying extra on the mortgage while I am working and will decide in the future. I prefer to just keep our options open.

    We too could also retire today with our current pension and VA benefits and I struggle often with the “why am I doing this when I don’t have to” question. I am currently going through a significant (not life threatening but quality of life threatening) health crisis that is changing my view on this: life is short and it is not guaranteed nor is a high quality of life. Based on this experience, I am tentatively accelerating my retirement plans.

    Thanks again for sharing story and wish you the best in your decisions!

    Reply

Leave a Comment