The government in Allentown, Pennsylvania, was desperate to find ways to fund its pension plan for city employees, which was about $160 million short. Officials decided in 2012 that selling off the city’s public water system was the best solution. They gave serious consideration to contracting with Suez North America, a private water company with headquarters in Paramus, New Jersey.

With such a momentous decision looming, Mary Grant, of the advocacy group Food & Water Watch, said she thought residents should have some say in the deal. She tried to get information from the city about the offers from bidders before a contract was approved.

But then she hit a wall – one erected by Suez and the city itself. The company had intervened and urged the city to turn down the group’s request for financial information related to the deal, Grant said.

State secrets

The State Secrets project is a joint effort by Reveal from The Center for Investigative Reporting and the Milwaukee Journal Sentinel.

It took the state’s Office of Open Records to force the city to come up with some of the financial information that Grant was seeking. But it was six months after the city approved the deal on April 13, 2013, and a year after Grant’s initial request, rendering the documents essentially useless.

“The process was so delayed that we ended up not getting the documents until after a contract was signed. We had wanted the information to help inform the public conversation before the decision was made over the future of the water system,” Grant said.

While her request was pending, the city made a different deal: After much debate, the city signed a 50-year lease with another water system in nearby Lehigh Valley – one that is publicly run.

The fight in Allentown highlights a problem common across the United States: Many public contracts are shrouded in secrecy. Government contracting, which involves billions of dollars in public funds each year, has become one of the least transparent systems that state and local governments maintain, according to an investigation by Reveal from The Center for Investigative Reporting. Under the guise of protecting “trade secrets,” state and local governments are withholding critical information about public spending. That allows private entities to claim that some information is proprietary and its disclosure would harm their business.

And it is far from the only way governments and companies keep the public in the dark about government spending.

In many communities, residents, such as those in Allentown, who want to evaluate the terms of a deal while it’s being negotiated are also stymied by state regulations and laws that ban disclosure until a contract has been signed. That makes it impossible for the public – and in some cases, elected officials – to dig into such matters as contract termination fees, profit margins and vendor investment in infrastructure.

It’s also increasingly difficult to find out who private contractors are hiring, whether their employees are properly qualified or might have criminal records, what extra fees they are charging the government, how the contractor is performing based on internal reports, who is bidding on a contract and their qualifications, and what types of internal documents they are creating for the government agency.

And perhaps the most difficult information for the public to glean: Is the contractor actually saving the public money? Is there sufficient accountability to guard against favoritism and corruption?

Across the country, even the smallest town is buying services from outside contractors: school bus drivers, trash collectors, parking system managers, court record managers, and police video camera and Taser vendors, among many others. But when members of the public – community activists, journalists, businesses and anyone else – knock at the town hall door seeking details they often are told: We can’t make that information public.

A San Diego police officer wears Taser’s Axon body camera. There are fierce debates about whether such video footage should be public and who actually controls the footage. In most cases, Axon, a Taser company, has control. Credit: Frank Duenzl/picture-alliance/dpa/AP Images Credit: Frank Duenzl/picture-alliance/dpa/AP Images

In Charleston, West Virginia, when residents wanted to find out whether the local water company was prepared to deal with deadly chemical spills – after a poisonous chemical in 2014 fouled the water supply in the Elk River – they, too, encountered resistance.

West Virginia American Water, a private company regulated by the state and performing an important public function, refused to supply the information to a local activist group, Advocates for a Safe Water System. In one document the company eventually handed over, it blacked out the names of public officials and the public utilities commission’s toll-free 800 number, labeling that “Highly Sensitive Confidential Information.”

In Los Angeles, Canada-based New Flyer of America made a crucial promise as it was bidding for a coveted $500 million bus contract with the Los Angeles transit authority. The company would create 250 new jobs in the United States. With that commitment, the company was able to seal the deal in 2013.

But as of this year, three years after the contract was signed, taxpayers still don’t know if New Flyer has kept its promise.

When Jobs to Move America, a labor-affiliated watchdog group, tried to learn in February just how many of those jobs actually had been created, what types of jobs they were and what workers were being paid, they bumped into an impenetrable wall. Even though the deal with New Flyer was paid for with public funds, the Los Angeles County Metropolitan Transportation Authority and the company provided few details to the watchdog group, claiming they were trade secrets. The organization has gone to court to challenge the government-sanctioned secrecy.

The trade secret exemption “absolutely eviscerates the public records law,” said Madeline Janis, executive director of Jobs to Move America. “Companies claim that everything is a trade secret, everything is proprietary. If communities and workers can’t get the information, how do the public and the public agencies know if the contractors are keeping their promises?”

The answer: Often they don’t. And governments aren’t highly motivated to push back too hard against private firms – which usually decide what should be blacked out – especially if the goods or services the company is providing are delivered on time and at the agreed upon price.

In 2013, Code for America, Omidyar Network and the Sunlight Foundation, in a survey of 28 cities and counties, found that it was nearly impossible for the general public or prospective bidders to get the most basic details about the procurement process, such as selection criteria  the text of contracts and information about subcontractors. That not only limits the public’s ability to scrutinize the deals, it may make it impossible for other vendors to offer a better bid.

Since that study, Code for America and Sunlight Foundation have been working with state and local governments, suggesting ways to make bids easier to understand, and recommending bids be broken into smaller contracts to offer opportunities for smaller vendors. They also are pushing for more detailed disclosure of contracts.

But when it comes to dealing with contractors, state and local governments encounter a complex legal landscape because of the trade secret exemption, and frequently rely on the contractor itself to define just what is proprietary and should be withheld from the public. Many governments, especially smaller ones with no in-house general counsel, find themselves unable to muster the legal firepower to push back.

“They don’t want contractors not to do business with them. They don’t challenge contractors. The agencies effectively do what contractors say about trade secrets, but they have no basis to know if it is true,” said Scott Cummings, a law professor at UCLA who has conducted research for Janis’s group. Many governments, he said, are simply “risk averse.”

More outsourcing, less scrutiny

As governments have stopped doing tasks they once did – running the water supply, hiring the school bus drivers and operating schools, conducting audits, running ambulance services, to name a few – the outsourced contracts are built into their annual budgets, often appearing as part of line items that also include other spending.

And even when the contracts are audited, the audits don’t zero in on details, such as determining if invoices reflect actual work, or are just a pile of numbers. That lack of scrutiny makes it difficult, and in some cases impossible, for members of the public, or even government agencies, to act as watchdogs over spending.

“The major problem is that governments don’t always know how well they are doing before they contract out,” said Daniel E. Bromberg, who teaches public administration and political science at the University of New Hampshire. “If you don’t know how much you are spending, how do you know you are going to save money?”

State and local governments spend a lot of time talking about transparency, and touting local efforts to put more and more data online. But when they post the data, such as the names of the vendors who secure contracts, they often do only that, and don’t provide the more detailed information the public may be looking for: Why did this vendor win, and how much money is that company saving the government? The usefulness of this data varies tremendously because the data often are just that – a bunch of numbers that don’t provide a rationale for the award.

And because many companies have been able to use the trade secret exemption to keep their lists of employees and subcontractors secret – even though they are getting paid with taxpayer funds – it can be nearly impossible to know whether, for example, the relatives of a public official are on a contractor’s payroll.

Changes in 2016 to the federal Freedom of Information Act have opened up federal contractors to greater scrutiny. The new provisions make it tougher for federal agencies to withhold contracting details. But with a few exceptions, there has been no similar movement among the thousands of cities, towns, boards, commissions and counties across the country. In September, California enacted a new law that would allow government agencies to withhold from the public any codes or numbers they use to identify contractors they hire.

To Bromberg, public contracting has created a “black box” where the important details are hidden.

“You could download every contract you would want, but you still won’t know what’s in the black box,” said Bromberg. “It’s impossible to know whether or not there is corruption.”

Secrecy has been on the rise in many sectors of the government for several years.

The National Freedom of Information Coalition has found a pattern of growing secrecy at the state and local levels. The group, which works closely with journalists and open government advocates, found between 2009 and 2015 a similar pattern of growing state and local secrecy in studies in 2009, 2011, 2013 and 2015.

And a labor-affiliated group, In the Public Interest, which pushes for greater transparency in government spending, conducted studies of the nexus between growing secrecy and outside contracting. In 2012, and again in 2015, the organization found that privatization of government functions has put formerly public information out of public reach in state and local governments – and created less accountability in spending.

“While taxpayers might still have access to basic spending information that is held by the state – such as the total dollar value paid to the company – they lose access to public information that is held by the company. Without details about contractors’ expenditures – such as executives’ salaries and the services provided by subcontractors – the public cannot prevent irresponsible spending,” the organization’s 2015 report said.

Media shrink, secrecy grows

National Freedom of Information Coalition also found that the decline of local media has played a key role in enabling state and local government secrecy; that there are fewer news organizations holding state and local institutions accountable; and that in many parts of the country, media watchdogs that remain are less aggressive than in the past.

“The traditional media, particularly newspapers, have always led the open government charges if the school board is closing a meeting illegally or the city is denying records or a judge is kicking a reporter out,” said Jeffrey Hunt, a prominent media lawyer in Utah. “I just see the media leaving the field in terms of fighting these battles.”

Two studies were particularly blunt about the problem. In its 2013 survey, the National Freedom of Information Coalition – made up of media and open government groups, and the Media Law Resource Center, a group of lawyers – found “a greater inclination among government officials for gaming the system than complying with existing disclosure and accountability laws.” A follow-up survey in 2015 by the two groups, joined by Investigative Reporters and Editors, reiterated those concerns.

The city government salary scandal that began unfurling in 2010 in Bell, California, illustrates the notion that officials are abusing the public records system. When a resident made a public records request for city council member salaries, he was given bogus records that masked the truth: City administrator Robert Rizzo was taking a salary of more than $400,000, and several city council members were padding their paychecks. The scandal included allegations of voter and contracting fraud; many officials went to jail.

Lack of transparency in procurement and budgeting takes many forms.

Some government agencies fail to provide budget information that can be understood by someone other than an accountant, or they name the contractor but don’t explain why it was the winning vendor.

Others charge excessive fees for information. The Massachusetts state police asked for $2.7 million to search and produce a 2012 database for a lawyer who was questioning the reliability of drunken driving tests. Though the data were available electronically, the state police said they would have to print out all 2.5 million pages at 50 cents a page and pay workers by the hour to review each page. Wisconsin only asked for $75 for similar data.

Sometimes public institutions hire outside companies to collate the data and then sell it back to the public, the rightful owner of the data to begin with, while others will simply provide the data for a nominal fee – or for free.

Nervous state and local government officials often are inclined to avoid disclosure, either for fear of accidentally releasing an employee’s personal or medical information, or because they can’t find the information. It’s often easier for government officials to simply say they are barred from disclosing the information or to ask for delays, which can make the eventual release of material useless to those seeking it.

And if members of the public want to press hard for disclosure, they face another hurdle: the appeals process. Individuals – often residents seeking to influence a government decision – struggle in many states with a legal system that doesn’t offer them much help. Many states have no meaningful internal appeal system, forcing anyone challenging a decision to hold back information to go to court. For many people, that ends the battle, because they can’t afford to sue.

How open is your state? An interactive database

Pushback against privatization

In New York, the Rockland County government has found itself on the side of public disclosure, pushing for more transparency from the state Public Service Commission about Suez North America, the same private water company that Allentown, Pennsylvania, residents were pursuing.

Suez has been trying to recoup $54 million it says it spent on a desalination plant that the state initially told the company to look into but then ordered it to drop the effort. At first the state commission told Suez it could not be reimbursed by ratepayers – individuals and businesses who rely on the water system – because, among other reasons, it had not started construction.

But in February 2016, the utilities commission changed its position after Suez provided nearly 10,000 pages that it said documented its rationale for getting reimbursed by the public users.

The documents were short on details, says a lawsuit from the Rockland County government, whose residents would be the ones footing the bill if Suez were to start adding a surcharge to water bills. The documents included “heavily redacted invoices for legal and non-legal expenditures,” the county’s lawsuit says.

Did the company really spend that money? The Public Service Commission, the Rockland County government asserts, didn’t verify the invoices, or look for backup documentation. So calculating Suez’s actual financial outlay is next to impossible, the lawsuit says.

In Chicago, the 2008 privatization of the city’s 36,000 parking meters – which earned the city $1.15 billion from an entity created by banking giant Morgan Stanley – sparked extensive controversy, beginning with the city’s aldermen complaining that they were given only two days to review the contract.

And the public was left out completely. The city did not provide any information about cost-benefit analysis or details about how the contract would be structured, said a report by In the Public Interest, which analyzed the contract.

“In effect, the public had no information about the deal or any opportunity to provide stakeholder input, even though the contract would greatly affect parking across the city,” the report said. “Consequently, the contract was rife with clauses that directly opposed the public interest.”

Among the terms that sparked controversy: The operator would be able to increase meter rates, and the city would have to compensate the company if its revenues declined – such as when the city closed streets for public events or snow. The deal also gave the metering company influence over when the city might build new garages – potential competitors to on-street parking meters.

The city’s own inspector general found – after the deal was inked – that the city may have sold off its system for about $1 million less than it was worth.

Charter schools resist transparency

The charter school industry is another government contractor that has resisted transparency in several states, even though its schools are publicly funded.

There is no central place for the public to find out how much tax money is going to charter schools, said Lisa Graves of the Center for Media and Democracy, which is examining charter school spending.

And whether the schools themselves, or their governing bodies, are subject to public disclosure can vary by state, she said.

In Arizona, Florida and Louisiana, for instance, most information about charter school spending, including teacher salaries and educational backgrounds, can be kept from the public because of an authorizing system that set up private governing bodies for the schools.

“Are they public or private? Do accounting rules apply?” Graves asked.

The result is that the public has little information about how much federal money is being passed to individual charter schools and how those schools are being managed locally. Public schools, by contrast, are required to be much more transparent about their spending, publishing budgets that are reviewed in public by school boards, and city or county councils.

Graves also wanted to examine applications for charter school funding, much as the Allentown residents wanted to see proposals for privatizing the water system before a contract was awarded. But when she asked the federal Department of Education for the information, she was turned down.

Similar complaints about a lack of transparency in charter schools have occurred across the country.

New York’s comptroller audited Brooklyn Excelsior Charter School in 2012 and raised questions about its spending, including paying almost $800,000 above market value to rent a building from its parent company. The audit also said that the parent organization, National Heritage Academies, did not give the auditors details on how the school spent $1.6 million in public funds, claiming the information was proprietary.

In Massachusetts, an advocacy group supporting public schools sought information in 2015 from charter schools related to the possibility that they had improperly used public funds when they asked parents to lobby on behalf of charter schools. The group, Jobs for Justice, was given a bill of nearly $100,000 for document processing from several of the charter schools before the schools would even begin to dig out the information.

Weak oversight, low standards

A 2014 study of contracting in New Jersey provided a grim picture of that state’s capacity to provide information about contracts, citing weak oversight as the problem.

“No agency within the state appears to have the capacity and competence to provide effective contract oversight,” said the report by Janice Fine and Patrice Mareschal of Rutgers University. “New Jersey lacks an adequate number of trained staff to provide oversight.”

And when they do find problems, the standards aren’t high, the report said. “Contracts typically include weak performance requirements and standards, and they do not include adequate penalties to hold contractors accountable for poor performance.”

And because government data collection was scattered, the study said that even when information is public, it isn’t useful. “Contract data for human service contracts is not kept in any systematic way. As a result, it is nearly impossible to gather information on these contracts,” the report said.

Maryland Comptroller Peter Franchot has made procurement reform a key part of his political agenda. Even though he is a government official who sits on a state panel overseeing contracting, getting the details can be difficult. Franchot is part of a three-person board that also includes the governor and the state treasurer who meet weekly to examine contracts larger than $200,000.

Franchot believes Maryland’s procurement system is broken. “We have to take the agency’s word for it that there wasn’t any competition and this is a good deal,” he said.

Nicole Neditch, formerly senior director of government practices for Code for America, and now director of community engagement, said Franchot’s complaint is echoed by many government officials. Neditch said she often hears concerns that they are often hamstrung because vendors have been able to take command of the system and there is often not enough time to look for alternatives.

This story was edited by Robert Salladay and Amy Pyle and copy edited by Stephanie Rice and Nadia Wynter.

Miranda S. Spivack, a former Washington Post editor and reporter, is the Pulliam Distinguished Visiting Professor in Journalism at DePauw University. This story is funded by the Ravitch Fiscal Reporting Program at the City University of New York.

Contact Miranda S. Spivack at or tweet to her @mirandareporter.

Republish our articles for free, online or in print, under a Creative Commons license.