I was an MD at a big bank, and I retired early. Real happiness came from buying my freedom, not an expensive watch.

• Eric Sim, 55, left banking in 2017 after working in the industry for over 20 years.
• Sim said he was able to retire early because he didn’t spend like the average investment banker.
• Use your money to buy freedom, not to impress others, Sim told Business Insider.

This as-told-to essay is based on a conversation with Eric Sim, 55, a former UBS banker who became a published author, professional speaker, and career coach. The following has been edited for length and clarity. Business Insider has verified his education and employment history.

When I was a young banker, I often heard my more successful colleagues bragging about their new watches or latest sports cars.

It wasn’t long before I gave in to peer pressure. I bought myself a $3,000 watch. That made me happy for a few weeks, but the novelty of owning the watch faded quickly.

I stopped wearing that watch and switched back to my Timex, which I continue to wear.

Needless to say, I didn’t follow my colleagues’ footsteps when they bought their exotic cars or private wine cellars. I began to realize that instead of spending my money frivolously to impress others, I was better off using it to buy my freedom.

When you are working a regular job, you have to work with people that you may or may not like. You don’t get a choice. But if you have freedom, then you can choose what you want to work on and who you want to work with. That’s far more invaluable.

That mindset allowed me to retire eight years ago, when I was still in my forties.

You shouldn’t be spending to impress

Holding back on big-ticket expenditures during my banking days has given me the freedom to pursue my passions and interests in retirement.

In 2005, I contemplated going back to school to get an architecture degree. I eventually dropped the idea once I factored in the costs of switching careers. Besides having to study for five years, I would also miss out on the income and bonuses I could have gotten if I had just stayed in banking. It just didn’t make sense to do it.

Now, in retirement, I could finally scratch that itch without any financial worries. I recently attended a six-week summer course on landscape architecture at UC Berkeley.

Of course, attending summer school at UC Berkeley isn’t cheap. The fees, flights, and accommodation add up to around $20,000. That’s on top of the six-week duration, which would be hard to make time for if I were still working a full-time job.

Going to UC Berkeley was only possible because I wasn’t tied to a job. Most of my peers in banking still have to work because they stretched themselves financially with their expensive purchases.

By not splurging on sports cars or wine cellars, I had the flexibility to pursue the experience of being a student again. That’s priceless.

We often spend a lot of time and energy worrying if people will look down on us based on the type of house we live in or the car we own.

Since leaving banking, I have been driving the smallest car I have ever owned. At first, I wondered how people would perceive that. Would they think I was no longer successful after quitting banking?

Surprisingly, no one cared. I have given people lifts in my car, and no one has ever treated me differently because of my car. These worries are usually all in our heads. Most people are more worried about their own image and reputation than about other people’s.

Use your time and money on the right things

There are three stages in your life.

In stage one, when you are about to join the workforce, you have time and health, but no money. Then, in stage two, when you are in your prime, you have money and health, but no time because you have to work and take care of your family.

Finally, when you are in stage three and have retired, you might find that your health has deserted you. Yes, you have time and money, but you can’t really enjoy it.

I believe there’s a magical stage between stages two and three, where you have time, health, and money.

My goal has always been to prolong that period, which usually lasts less than five years for most people. Some may even jump from stage two to three without enjoying the magical stage.

If you can rein in your spending in stage two, you can stretch the magical stage to 10 and 15 years. That’s the real sweet spot we should all aim for.