The actor Jason Patric went to court to “fight ’til I’m dead,” as he put it, to see his 4-year-old son. Patric and his girlfriend conceived the child through in vitro fertilization, and a judge earlier denied Patric any paternal rights. Since then, Patric has become a hero to frustrated fathers everywhere who are alienated from their ex-wives and girlfriends. Patric is part of a movement of vigilante fathers who challenge the assumption that mothers should have any more rights than they do. Skier Bode Miller famously challenged a girlfriend’s right to move to New York after she got pregnant with his child, and earlier this year a group of Utah fathers sued the state over a law allowing mothers to give up their babies for adoption without their consent. The courts, argued one of the Utah plaintiffs, treat men “like they’re scum—like they don’t have rights at all as far as having a relationship with their children.”

It used to be that women had to worry about men disappearing after they got pregnant or divorced. Now, some women have the opposite problem. A growing fathers’ rights movement is aggressively challenging what it sees as the courts’ assumption that the mother is the only real parent.  Men’s rights activists air their grievances about unfair child custody laws on sites such as A Voice for Men and on subreddits like Men’s Rights and The Red Pill. Last year, Ken Cuccinelli was tied to a men’s rights group advocating for divorced fathers. “Men are angry at losing their kids in the divorce court and taking their dream of raising them and reducing it to a child support payment and every other weekend,” writes one men’s rights blogger quoted in Michael Kimmel’s 2013 book Angry White Men. And that view is shared by the broader public. One recent study showed that people are generally in favor of joint custody, but they believe that divorce courts are seriously slanted toward mothers.

But is this actually true? “There’s a real perception—even women share it—that courts are unfair to fathers,” says Ira Ellman, a custody expert at Arizona State University. But in fact the great revolution in family court over the past 40 years or so has been the movement away from the presumption that mothers should be the main, or even sole, caretakers for their children. Individual cases like Patric’s may raise novel legal issues, but on the whole, courts are fair to men, particularly men who can afford a decent lawyer.

Cases like Patric’s and Miller’s, which involve fathers who never married the mothers, are relatively new to the courts, but divorce courts have a long history of trying to keep up with changing gender dynamics. In the 1970s, family courts began to move away from assuming a model of breadwinner husband and dependent wife. Instead, courts assumed interdependence, meaning that husband and wife shared assets and domestic duties. Pretty rapidly, Naomi Cahn and June Carbone explain in their new book, Marriage Markets: How Inequality Is Remaking the American Family, the rules became more “gender neutral.” As the image of an abandoned, innocent wife faded, alimony declined. The maternal presumption that mothers should automatically get custody of children in the “tender years”—meaning younger than 7—also faded. And the vast majority of states moved toward an assumption of joint custody. In 2000, for example, a new law in Wisconsin directed courts to maximize the time children spent with both parents.

Men’s rights activists complain that despite the legal changes, mother preference still lingers, and studies have shown that through the 1980s sole mother custody still prevailed. But more recently judges have been catching up to the law. According to one of the most thorough surveys of child custody outcomes, which looked at Wisconsin between 1996 and 2007, the percentage of divorce cases in which the mother got sole custody dropped from 60.4 to 45.7 percent while the percentage of equal shared custody cases, in just that decade, doubled from 15.8 to 30.5. And a recent survey by the American Academy of Matrimonial Lawyers shows a rapid increase in mothers paying child support.

Berkeley law professor Mary Ann Mason tracked the changing priorities of divorce courts over three decades. The biggest recent change, she writes, is the courts’ preference for the “friendly parent,” meaning the one who can get along with the other parent. Mothers who get in the way of a father’s involvement can in fact be penalized by the courts. In their book, Cahn and Carbone tell the story of the Renauds, a divorcing Vermont couple whose case was resolved in 2004. Before the divorce, the couple shared child care. The mother took Fridays off to be with the children, and the father took them to and from day care and was an involved dad. The marriage ended when the father told the mother that he was having an affair with a colleague. In another era, the mother would have gotten sole custody of the children and alimony, but not much child support. Now, “the mother’s ability to retain custody depends on her willingness to support the father’s involvement,” Cahn and Carbone write. In this case, the mother accused the father of abuse and neglect. When the investigators could not confirm the charges, the court awarded the father 50 percent custody and made the mother’s custody contingent on her working to repair the relationship with the father.

The real inequality in family courts these days is not based on gender, but on income. Wealthy men have successfully fought against proposed reforms that would have forced them to pay more child support. With elite, college educated men, “it’s outrageous how little they can end up paying in child support in some cases,” says Ellman, the Arizona State professor. But poor men are in a different predicament. Welfare reform in the 1990s included an effort to track down fathers who weren’t paying child support. As the economy sank, those fathers fell behind on their payments and often wound up in jail or permanent debt, as Elaine Sorensen of the Urban Institute has documented.

A father who never married the mother of his child has a much shakier legal status. Petitioning the courts for paternal rights as a father who had a child out of wedlock is complicated to do and much less likely to be successful. A legal system that has evolved to recognize equal, interdependent parents doesn’t really apply. As Cahn and Carbone have written, in this social class the women are generally better off and choose not to marry the fathers, precisely because they want to avoid future legal disputes over children. If that father is Jason Patric or Bode Miller, he can probably afford a lawyer and get sympathetic publicity. But if he’s not, the best a poor father can hope for if he wants to impact his child is to be a steady paycheck.