The U.S. government does much more
to abridge freedom of association than freedom of exclusion.  But I
still think freedom of exclusion is worth defending.  Over at Cato
Unbound, my friend David Bernstein has presented a libertarian defense of “the application of antidiscrimination laws to private parties.” 
Or to be more precise, he criticizes some libertarians’ “blanket
opposition to such laws.”  The heart of Bernstein’s argument:

[T]o say the least, segregation and exclusion of African Americans in
public places in the South wasn’t entirely a voluntary choice of
business owners.  Jim Crow segregation involved the equivalent of a
white supremacist cartel.  The cartel was enforced not just by overt
government regulation like segregation laws, but also by the implicit
threat of private violence and extra-legal harassment of anyone who
challenged the racist status quo.  This violence and extra-legal
harassment was often undertaken with the approval of local officials;
the latter, in fact, were often the perpetrators.

As a result, both available policy options were unattractive in libertarian terms – and only one was viable:

To break the southern Jim Crow cartel there were two options: (1) a
federal law invalidating Jim Crow laws, along with a massive federal
takeover of local government to prevent violence and threats against,
and extralegal harassment of, those who chose to integrate; or (2) a
federal law banning discrimination by private parties, so that threats
of violence and harassment would generally be met with an appeal to the
potential victim’s obligation to obey federal law.  The former option
was arguably more appealing from a libertarian perspective, but it was
completely impractical.

I have two main complaints.

First,
Bernstein’s argument has much more sweeping implications that he
admits.  It justifies laws against discrimination in dating and
marriage.  After all, merely striking anti-miscegenation laws
from the law books hardly freed mixed-race couples from the “implicit
threat of private violence and extra-legal harassment.”  Why not
mandate race-blind dating to defuse the threat of the white supremacist
cartel? 

Indeed,
Bernstein’s argument justifies laws against racist expression.  Fear of
the white supremacist cartel could easily force people to pretend
to be racists.  Just declaring free speech isn’t enough; once again, it
seems like you’ve got to punish racist speech to pave the way for free
anti-racist speech.  I’m surprised that the author of You Can’t Say That!  The Growing Threat to Civil Liberties from Antidiscrimination Laws didn’t consider this implication. 

Of
course, you could point to history and conclude that laws against
marital discrimination and racist speech weren’t actually necessary to
protect the liberty of mixed-race couples and anti-racist speakers. 
But this highlights my second objection to Bernstein’s argument: If
simply striking pro-racist laws from the books worked for marriage and
speech, it would have worked for employment and public
accommodations, too.

In fact, simple deregulation probably would have worked better for business than for marriage or speech.  Both marriage and speech have a strong herding component
Most individuals don’t want to marry a member of a group that most people
don’t want to marry, and most individuals don’t want to say things that most
people don’t want to say.  Despite weak incentives to defy the cartel,
though, deregulation still worked wonders.  In for-profit business,
on the other hand, contrarian strategies often pay, big time.  The
first firm that hires qualified minorities or accepts minorities’
patronage cleans up.  That’s quite an incentive for defiance.

As my co-blogger David Henderson points out,
“The fact of the matter is that this country moved from segregation
required by law to segregation forbidden by law without trying freedom
of association for a millisecond.”  Bernstein tries to defend this
sudden switch by pointing to historical context.  But the real lesson
of history is that when we tried freedom of association, it worked. 
There may be cases where you’ve got to restrict liberty to protect
liberty, but this wasn’t one of them.