All U.S. companies with 100 or more employees are required to submit an EEO-1 report each year with demographic breakdowns of their workforce to the federal Equal Employment Opportunity Commission.

Reveal from The Center for Investigative Reporting sought these government-mandated reports from 211 of the biggest technology companies based in the San Francisco Bay Area to look at how they were faring on employee diversity.

As of Oct. 19, 2017, we have been able to obtain EEO-1 reports from 23 of those companies. One of those 23, Clover Health, now says its report might be inaccurate.

What’s the big deal about EEO-1s?

The EEO-1 form includes more than 130 data points and breaks down the demographics of a company’s workforce by race, gender and job category.

Unlike many other kinds of diversity reports that companies release, which may include only percentages, EEO-1 reports include the actual number of employees broken down by category. By using these reports, we can assess whether percentage calculations of race and gender are reliable or misleading. For example, a smaller company needs only a few women or minorities to substantially change its percentages.

Managers at many tech firms were critical of EEO-1 forms, saying they contain outdated categories not suited to their fields. The “professionals” job category includes both tech workers such as software engineers and non-tech employees such as lawyers, accountants and human resources specialists.

Some critics fault the forms for what they don’t include. The multiracial option, for example, doesn’t specify races. The reports don’t include age distributions, gender identities, sexual orientation or disability data. They are only snapshots of demographics at a point in time.

An additional layer of potential error lies in the fact that employers are guessing the race of employees who don’t provide that information. But despite its flaws, the EEO-1 is the only standardized report that can be used to compare companies with each other.

Many companies that have not publicly released their EEO-1 reports, such as PayPal, Pandora Media and Netflix, instead create their own diversity reports with customized categories. Reveal found these reports often don’t divulge key information, including the overall size of the companies and gender data broken down by race. Companies may have their own classification systems or may not include employees who choose not to identify themselves. These factors make it unreliable to compare one company with another using their alternative reports.

For this reason, we’re including in our analysis only companies that have released their EEO-1 reports publicly or exclusively to Reveal. We’re also calling on other large technology companies to publicly release their EEO-1 data.

What definitions did we use while analyzing our data?

We have used the terms “people of color,” “women of color” and “underrepresented minorities” in our analysis.

“People of color” is made up of people belonging to all race groups, except those who are white. “Women of color” includes all women, excluding white women. “Underrepresented minority groups” includes groups such as black, Latino, Native Hawaiian or Pacific Islander, American Indian or Alaskan Native, and two or more races; it excludes those who are white or Asian. “Underrepresented minority women” includes all women, except white and Asian women.

EEO-1 reports also include a category called “two races or more,” which does not specify which races are included. Those employees are included in our “underrepresented minorities” category.

On EEO-1 forms, all employees must be classified as either female or male, thus obscuring any information about employees whose gender identity does not fit into those two categories.

Which job categories did we focus on?

We are comparing many types of tech companies to each other. Those companies do vastly different things, and their workforce compositions vary.

Even within the same industry, companies vary in size, maturity and structure. Some companies, such as Apple, have a large sales team, while others have insurance agents, health care workers or customer service representatives. This could alter their overall workforce demographics, making apples-to-apples comparisons difficult. For this reason, we focused on specific job categories instead of total employees.

There are several types of job categories in EEO-1 reports. We decided to focus on professionals, because they make up the bulk of most tech companies’ workforces and because they include tech workers such as software engineers and web developers. We also focused on managers and executives because most major decisions are made at those levels.

Managers are classified as “first/mid officials and managers” in EEO-1 reports. Executives in our analysis are called “executive/senior officials and managers” in EEO-1 reports.

We also analyzed managers and executives together because we found that some companies had a relatively large number of executives and almost the same number of managers, while others had fewer executives and more managers.

A major drawback to using EEO-1 forms is there is no way to figure out which employees have technical jobs and which have nontechnical jobs. Professionals may include lawyers, engineers and financial analysts.

Here are the Equal Employment Opportunity Commission’s definitions of jobs included in the EEO-1 categories.

How did we decide which are the largest tech companies in the Bay Area?

For this project, we wanted to focus on Silicon Valley. Therefore, we considered tech companies headquartered in the San Francisco Bay Area only, which includes Alameda, Contra Costa, Santa Clara, San Mateo and San Francisco counties. This means we did not include companies such as Microsoft, Amazon and Dell, which may have a presence in the Bay Area but are headquartered elsewhere.

We created a custom list of the biggest Bay Area tech companies by combining data from multiple sources.

We started with this list compiled by The Mercury News in San Jose of the 150 largest publicly traded companies in 2016. We added privately held companies valued at $1 billion – so-called “unicorns” – from lists curated by Crunchbase and CB Insights. Reveal’s list was current as of Aug. 3, 2017.

We’ve excluded unicorns that have been acquired by other companies. AppDynamics and Jasper Technologies are examples of $1 billion private companies that recently were acquired by Cisco Systems – they are not included in our analysis.

We decided to include LinkedIn as a part of our analysis. It was acquired by Microsoft in late 2016, but was a publicly traded Silicon Valley tech giant that had been releasing its EEO-1 reports publicly.

The Mercury News’ SV150 list of publicly traded companies includes Alphabet Inc., the parent holding company for Google. On its website, Alphabet links to Google’s diversity data and does not appear to release separate diversity data. We decided to include Google as a part of our analysis instead of Alphabet.

If a company on our list had not publicly released their 2016 EEO-1 report, we contacted them multiple times to request it. Three companies, SciClone Pharmaceuticals, Deem and Quanergy Systems, reported that they had fewer than 100 employees in 2016 and thus were not required to file EEO-1 reports. They are not included in our analysis.

How did we assemble the data?

We collected EEO-1 reports in various file formats such as .pdfs and .jpgs. We used optical character recognition software to digitize what we could and then manually entered the rest of the data into a spreadsheet. A second reporter who had not entered the data verified each entry.

Sinduja Rangarajan

Sinduja Rangarajan is a data reporter at Reveal, focusing on academic collaborations around workplace issues. She is the organizer of Mind to Mind, a symposium that brings academics and journalists together to foster conversation and partnerships. She is a former Google News Lab fellow. She is based in Reveal's Emeryville, California, office.

Will Evans is a senior reporter and producer for Reveal, covering labor and tech. His reporting has prompted government investigations, legislation, reforms and prosecutions. A series on working conditions at Amazon warehouses was a finalist for a Pulitzer Prize and won a Gerald Loeb Award. His work has also won multiple Investigative Reporters and Editors Awards, including for a series on safety problems at Tesla. Other investigations have exposed secret spying at Uber, illegal discrimination in the temp industry and rampant fraud in California's drug rehab system for the poor. Prior to joining The Center for Investigative Reporting in 2005, Evans was a reporter at The Sacramento Bee. He is based in Reveal's Emeryville, California, office.