A lot has been written about the housing crisis in San Francisco. A recent Business Insider article lists San Francisco as the most expensive city to rent in in the United States, ahead of New York City. A lot has been written about this issue, but generally the arguments I've seen fall into two camps. First there are the people who want to increase population density by building more high rise apartment units. This is a fairly sensible solution to the problem: rental prices are expensive because there aren't enough rental units for all of the people who want to live in San Francisco, so adding more rental units should lower prices. The other camp argues that San Francisco doesn't have the transportation infrastructure to support the estimated 100,000 to 200,000 additional people it would take to meet the current market demand, or that building this number of units would significantly and negatively impact the character of San Francisco. This is probably also true, but isn't constructive because it doesn't really provide any solutions.
There's another part of this problem that I think should be discussed more broadly: why is it that so many people want to live in San Francisco and not elsewhere in the Bay Area?
San Francisco is less than 50 square miles and supports a population of more than 800,000 people. The only other major city in the United States that has a higher population density is New York City. We can certainly try to fit more people in San Francisco, but it's probably better to try to accomodate more people elsewhere in the Bay Area. For instance, the other major metropolis in California is Los Angeles which is relatively affordable and has only half the population density of San Francisco. Across the bay Oakland is also relatively affordable and has a population density similar to Los Angeles. More people could afford to live and work in the Bay Area, including those who work in San Francisco, if the rest of the Bay Area was designed with a higher density of rental units.
The population distribution in the Bay Area is dismal. Take a look at this map which is available from the USGS:
You can easily see the three large communities that have highly dense populations: San Francisco, Oakland, and San Jose. Elsewhere most of the South Bay is suburban sprawl, particularly the communities where the major tech companies are that everyone loves to malign. Throughout the South Bay there are entire cities built around the idea that the people living there will live in large houses with big backyards. This is unsustainable, and the lack of housing in these areas causes problems in neighboring communities like San Francisco. More high density zoning in the South Bay would decrease rental prices in those cities and would incentivize more people to live there. There are already a large number of people living in San Francisco who commute to the South Bay, and lower rental prices in the South Bay would encourage many of those people to move closer to where they work. The zoning laws in these communities are decided by the communities themselves so changing the population density in these places will require the people who live there to demand it. Cities like Cupertino are known for having residents who actively oppose the construction of new apartment complexes, so this is something that we need to try to change by raising awareness of the issue and discussing it more publicly.
The other major problem is the poor transportation infrastructure in the Bay Area. There are two major commuter rail systems: BART and Caltrain. BART is way over capacity. During rush hour the trains are completely packed, and the layout of seats on the trains isn't really designed to have a lot of people standing anyway. Both systems have poor hours. The last trains on both systems out of San Francisco stop running shortly after midnight. The coverage of both systems is also not that great: many of the people who take BART or Caltrain have to drive a car to the station and pay for parking which is expensive and inconvenient. A lot more people who live in San Francisco would be willing to live in the East Bay or South Bay if the commuter rail options were faster, cheaper, more convenient, and had better hours.
Increasing the capacity of these commuter rail systems is unfortunately rather expensive. For instance, the Central Subway in San Francisco that connects the SF Muni system to Caltrain stretches 1.7 miles underground and is being built at the astronomical cost of more than $1.5 billion. The Oakland Airport BART extension connects BART to the Oakland Airport above ground for 3.2 miles and has a budget of almost $500 million. In the future if fleets of self-driving cars are successful they could drastically change how people travel and reduce the need for this type of public infrastructure, but in the meantime it's irresponsible to delay the construction of this type of infrastructure even if it is expensive.
These are policies that San Francisco doesn't directly have control over, so in the immediate future San Francisco will have to focus on policies it can exert more influence over. Unfortunately this likely means building more high rise apartment complexes. In the future, however, I hope that the people in the Bay Area will continue to try to vote to fund improvements to our transportation infrastructure and encourage more high density housing units to be built throughout the Bay Area.