How Stan Middleman's past informs Freedom Mortgage's future

November 14, 2024 5:28 AM

Freedom Mortgage CEO Stan Middleman has distinctive insight into housing finance that few can match, according to people in the mortgage business who know him and his organization.

Middleman is "a true industry leader" and particularly well-known in the City of Brotherly Love, said Kevin Kauffman, the Philadelphia-based head of client engagement at Freddie Mac.

"I get to interact with that organization on a day-to-day basis, and they're truly focused in a way I think some others are not," he said.

The Freedom executive's insights were the key to his successes, which among other things afforded him partial ownership of the Philadelphia Phillies baseball team. But it took a lot of time - and mistakes - to get to where he is now.

Freedom Mortgage CEO Stan Middleman speaks at his alma mater, Temple University — Freedom Mortgage CEO Stan Middleman and Chip Hunter, dean of Temple University's Fox School of Business and the School of Sport, Tourism and Hospitality Management

Middleman recently acknowledged early career struggles in an interview about his new biography, "Seeing Around Corners." He was in Denver to discuss it at the Mortgage Bankers Association's annual convention at the time, just prior to the election.

The memoir penned by NMN columnist and analyst Chris Whalen, extends from Middleman's younger days when he attended Philadelphia's Temple University and later ran a "disco deli" to how he utilized mortgage servicing rights to help build Freedom's business.

It's a tale that Whalen said reflects a more entrepreneurial time for the business.

"Stan is one of the last examples of a large firm that was grown organically by accumulating MSRs and without passive capital from Wall Street," said the author, who forecasts a pending nonbank capital rule will bar small firms from doing this in the future the way Middleman did.

Middleman is "an eyewitness to the rebirth of nonbank finance in America in the 1990s. And he survived a number of challenges and financial crises, a testimony to him and his veteran team," Whalen added.

When asked about some of the more dramatic moments in his life, such as when he dealt with a threat of violence at work or a failed business, Middleman said, "I don't think I would change a thing.

"We are all prisoners of our own experience, and we're all [an amalgamation] of the things that we live through. Everything that ever happens to you has a part to play in the person you become," Middleman said.

What follows is an interview with the Freedom Mortgage executive, condensed and edited for clarity.

On the concept of the book’s title “Seeing Around Corners”

Middleman: A lot of things that happen that are cyclical and that you could anticipate if you were paying attention. It's not so much seeing around the corners, but seeing history repeat itself. We're understanding how certain things work in certain ways.

For example, if unemployment is rising, one could expect that the Fed will cut interest rates. If the country is at full employment, one should be surprised that the Fed cut interest rates. If the country was in full employment and coming off a period of inflation, one would be even more surprised that the Fed cut interest rates. However, if that happened during an election cycle, one wouldn't be as surprised.

That's the kind of thing that I'm talking about. The different pieces that come together and how they fit sometimes can give you a window into the future that you might be able to make decisions on, or preparations for, that you otherwise might have missed.

How past is prologue for home values in the next few years

Middleman: When we look at office space today that's used at about 60%, it's very reminiscent of office space in the late 1980s that was utilized at about 60% like the Resolution Trust Corp. era when savings and loans were deregulated and created a lot of liquidity that fostered a lot of construction. When that collapse came, led by office, it went into other asset classes. One of those asset classes was residential homes, where property values fell by 30%, but that's because property values had been going up for years based on increase in demand.

The set of circumstances that were impacted at that time were really not that different than today, and we might see another correction led by [the office sector] that might create a similar situation across all asset classes that are also overvalued or could be headed toward single-family again.

Single-family has not been overvalued as much as it will be in the next two or three years, which would lead me to the conclusion that by 2028 there's a good possibility that we have a value correction. The last peak is the next trough.

The values that we reached in 2007 took us until late 2021 to correct, and they corrected down to the 1988 level. I would suggest that it's possible that values go up and then correct back down to the previous peak of 2007-2008, which I wouldn't find surprising at all, and then see other asset classes — led by office — take that same corrective path.

The correction always comes because the assets become overvalued because of the difficulty of knowing how much liquidity needs to be in the marketplace. The Fed is carrying this conversation about soft landings, but I think they're really difficult to achieve because we live in a complex society. That makes it challenging to manage outcomes, and that's really what the Fed is trying to do. They're trying to manage the outcome and the country's at full employment. It's a little bit of an unusual environment to see the Fed trying to get in front of that, and my guess is that they probably won't be as successful as they'd like to be.

Why Middleman expects deflation instead of inflation

Middleman: I think that there is a greater likelihood of deflation than inflation, subject to the caveat of the size of government spending. Government spending has become so outsized since COVID. I think that we're more likely to be in deflationary times than inflationary times.

The two caveats that would alter that would be government spending for things that don't generate jobs, income and revenue, a huge influx of immigration and/or tariffs. Both of those things would cause inflation. If you got too much [immigration] you would have not enough jobs. Unemployment would go up. If all goods and services got to be more expensive because of additional tariffs, that also could cause inflation.

There are inflationary ghosts hiding around the corners that could change the way things go. If those things didn't happen, I would tend to believe that we are in a more deflationary environment than inflationary environment, because the technology keeps getting better and things keep happening faster, cheaper, and better. The quality of the stuff that we get is better. The speed that it happens at is faster, the cost of doing it is less. All other things being equal, I would expect us to be in a deflationary environment, which is kind of interesting.

The biggest concern for government servicing? Geographic concentration.

What's the problem with government servicing? It's the advances, right? So what's the issue is the advances? Why are the advances an acute problem, generally and historically? It's generally because they don't have a wide enough disparity geographically. Most servicing portfolios are not limited geographically to have the adverse impact of an event that would stop them from supporting the advances. Geo-diversity is an important thing, and not everybody has learned that lesson.

When you become a servicer, and you start to retain servicing, that's fine if you've got good diversity in geography. It gives you stable cashflows. You can make money when [the origination] business is not great. The problem is, if delinquencies go way up, it can create an adverse situation. It can become an anchor around your neck if it's not geo-spread.

There are levels of complexity to this. For example, those advances continue until the property is foreclosed on in government loans. If your concentration is in New York and New Jersey, where the timeline is so long to foreclosure, it could potentially put you out of business if your delinquency in those states was exaggerated.

It's not that it's independent mortgage banks that are at risk. It's small, independent mortgage banks that are at risk, unless you had nuclear war and 15 cities were under siege and nobody paid their mortgage. Then it would be a problem for everybody, but it wouldn't be our biggest problem.

The fears [from public officials aiming to manage servicing risk] are legitimate fears, the [policymakers] interpretations [of what should be done] aren't, necessarily.

Why Ginnie Mae nonbank risk-based capital rule and the Basel III endgame bank reproposal don’t properly address servicing stressors

Middleman: I just don't think it speaks to the issue because I think the issue really is scale and geodiversity. What's the risk that would cause the problem? It's widespread unemployment. Even if it got as bad as it has gotten to be, you could still have enough cashflows to cover the advances and do your job if it's not geo-centric and going on for years and years and years in areas where foreclosure is the slowest. That situation would be bad, because you're advancing against those Ginnie Mae securities, right? You have to pay that coupon rate on an ongoing basis, and if you can't foreclose, and it's going to go on and on and on and on while you're paying and that's problematic. If you had all nonjudicial foreclosure states, it's not even a blip on the radar. That's why the geographic makeup of the portfolio is important.

You have the opportunity to really plan around that risk, and that's something that we do, because we think that the advances are a risk, and we try to mitigate those risks by managing what those exposures are.

How his broker-to-banker transition informs his worldview today

Middleman: In theory, the more responsibility you take on, the more revenue you create, the more control you have over the outcome; and the biggest difference is it gives you the opportunity to manage the interest rate risk more effectively. That is still true. Sometimes you might say that a broker has an advantage because they could lock with a variety of different lenders to manage their interest rate risk. The truth is that you have far more control as a banker over managing your interest rate risk to a certain degree, or geographic risk, because you tend to have more scale as a banker, and you solve your scale problems more readily. It's very difficult to scale a broker and also scale with diversity from a geographical. Those are big advantages to running a banker

Now, if you're a banker or a broker and the only place you do business is New Jersey, and you only have 50 people, and it's not much different. There's just not a material difference.

It's what you do with the change that gives you a broader opportunity for a wider variety of revenues and more scalable management of your expenses.

On his aspirations for Freedom

My base philosophy is you're either growing or dying, so you better be growing, and you better understand the risk of growth. I think there is a limit, but I think that limit is about a 30% market share before you deteriorate the credit quality, and I don't think we're in danger of hitting 30% market share at this point in our careers. So we've got a lot of room and opportunity to be more important in the marketplace.