Making Data Matter in Administrative Systems

Read more articles by Steve Goldsmith

By Stephen Goldsmith • August 4, 2014

This is the third installment of an article series based on Stephen Goldsmith's paper "Digital Transformations: Wiring the Responsive City." Click here to read the report in full.

For decades, well-managed cities have used data to enhance performance. Police Commissioner William Bratton launched the New York Police Department’s widely copied CompStat program in 1994. Through CompStat, Bratton and former mayor Rudolph Giuliani proved data’s effectiveness by gathering hot-spot data, which they connected to operations and strategy to dramatically drive down crime. This program inspired former Baltimore mayor Martin O’Malley’s CitiStat (launched in 2001), which, after O’Malley’s election to the governorship, was expanded to create Maryland’s StateStat in 2007. Yet these early examples stood out as exceptional and difficult. Analysts had no access to real-time information and even less insight into data from other agencies. The process of gathering data from multiple points and making sense of it in a paper-based analog world defied imagination, let alone execution.

Today, city officials sit atop loads of data that are expanding exponentially. In the 1990s, and even shortly after the turn of the century, the thought that every machine, vehicle, and clerical worker would incessantly generate important information remained a pipe dream. Even now, some ready sources of data fail to reach the attention of city leaders, a consequence of continual technology breakthroughs in every area of city services. In this chapter, I will discuss how the seemingly boring world of basic administrative systems can be, and is being, transformed to produce breathtaking advancements for municipal leaders and their residents. Nothing better proves this example than the work of former New York City finance commissioner David Frankel, who used data to produce much more with less.

Even those who oppose higher taxes generally do not object to increases in revenue derived from ensuring better compliance—more people paying what they owe. Incomplete enforcement favors those who abuse the system, shifting more of the burden to those who follow the rules. One of my first conversations with Frankel, when I was deputy mayor in New York, was about how to increase corporate taxpayer compliance. Simply increasing the total number of audits conducted seemed both unrealistic in terms of manpower and unfair to those taxpayers randomly targeted for an expensive audit, since more auditors meant more audits of those companies that had paid their taxes in full. Frankel and his auditing team, led by his deputy commissioner for audit, Michael Hyman, had a better idea: use analytics to increase the productivity of auditors reviewing companies thought to be underpaying their taxes. Using sophisticated data analytics, the commissioner instructed his department to look for patterns—identifying individuals who had businesses similar to others but who stood out as outliers on taxes paid. In so doing, Frankel and his team reduced the portion of audit cases closing without change: from 37 percent to 22 percent over three years. This represents a 40 percent increase in productivity for the department and a 100 percent reduction of government intrusion for the thousands of companies that would have been catapulted into the audit process, with an end result of no change on their returns.

To accomplish this task, the Department of Finance created the Data Intelligence Group (DIG), with the goal of identifying the most audit-worthy candidates. DIG creates models that suggest audit priorities. In their first three years, these models produced estimated assessments of $292 million, including $27 million from non-filers. The analytics encompass all business and individual income and excise taxes. Experts in the group access a broad variety of data, including city, state, and IRS tax information, as well as nontax information from other governmental agencies. These efforts include comparison of reported income and expense data across multiple sources, leading to the identification of under-reporters as well as non-filers. According to Frankel, “DIG’s modeling techniques, sophisticated statistical approaches, such as predictive modeling, and a multistep non-filer protocol are designed to avoid flagging businesses that are not required to file.”

In this way, the Finance Department more effectively captures underpaid taxes by focusing its finite audit resources on the targets most likely to have underpaid taxes. The Frankel lesson reveals the path to operational excellence—taking advantage of existing data, mining information from other sources, and assigning a well-trained team that asks the right questions and creatively applies algorithms.

For a century, the ideas of renowned mechanical engineer Frederick Winslow Taylor shaped efforts to improve organizational productivity. Taylor broke down assembly-line operations to identify the most efficient ways to perform a wide range of tasks. Workers received bonuses for reaching certain productivity milestones. Every “record” created a new standard for workers. But modern management requires more than performing a series of simple, specific tasks. The best managers develop detailed understandings of their employees’ unique skills—and they work with those employees to develop those skills over time.

American government has seen various basic human resource regimes. Under old political machines, winners gave out jobs and contracts to supporters. If patronage workers performed well, defenders of the system said, voters would reward them with reelection. If they failed, voters would turn them out of office. To wipe out the abuses of this system, civil services set detailed, legalistic rewards and punishments for government workers and contractors. Progressivism expanded this system by setting rewards and punishments for businesses. The welfare state did the same for recipients of government largesse.

But issues that require collaboration, negotiation, problem solving, and creativity are a different matter. Those issues require a broader skill set—and the autonomy to get the job done. Hiring results-oriented public employees and granting them a level of discretion previously thought to be unwise is now feasible; those employees, however, need tools and processes that allow them to find the best way to achieve their goals and to help others reach their goals as well. Those tools begin with data and new administrative systems, allowing data to be translated into results.

Human Resources

No area better defines the clash between new and old than public-sector human resources departments, where administrators apply outdated approaches to public administration—while beginning to use data to manage employee costs. The standard city HR department concentrates on job classification and testing regimes to attract individuals capable of performing carefully prescribed job descriptions. But this is occurring at a time when more complexity is confronting workers and thus, they need tools available to allow them to do more.

Managing Overtime and Scheduling

Many cities have created value by converting paper processes to online ones, including processes such as “onboarding” new employees, automated time and attendance tracking, and data-driven scheduling. The same issues now exist in HR as in other areas—for example, officials are increasingly sitting on huge amounts of underused data. Mining, harvesting, and analyzing this data will help mayors increase productivity by hiring better, promoting smarter, and managing scheduling and overtime more effectively.

The best workforce-management systems start with straightforward applications of automated systems. Houston, for example, uses automation to ensure flawless connection between payroll, time, and attendance—ensuring accuracy for time worked, tracking of medical leave, and unexcused absences. The Mississippi Department of Corrections uses its system to provide automated alerts to supervisors before overtime is worked. These digitally based systems dramatically increase accountability and set the stage for substantial productivity improvements. Accountability results from such newly available applications, including:

  • Staffing decisions that optimize open job or overtime assignments with individuals who have the right skills and training
  • Automated staffing based on business rules and employee preferences
  • Automated pay that uses analytics to connect contractual and overtime rules for pay calculations and to identify exceptions, making it easier for supervisors to pay attention to outliers
  • Simplifying management of vacation, sick FMLA, and flexible time attendance

Don Pagel, a data-driven, former deputy director in Houston, oversaw the implementation of a sophisticated time and attendance system that, coupled with analytics, will not only facilitate the move away from paper-based systems to track attendance, but will also eventually save $10 million annually in compensatory and unauthorized time. According to Pagel, not only did the city save personnel costs with an automated system; it was able to introduce more accountability. He was, as well, able to redeploy people who had previously entered data from paper.

Moving from Accountability to Productivity

The state-administered highway system in Arkansas totals 16,416 miles, which ranks 12th in the country. The State Highway and Transportation Department utilizes about 6,000 pieces of equipment, including its vehicle fleet of about 2,000 and a workforce totaling more than 3,000 full-time employees. Nearly a decade ago, the department developed and implemented an inventory system to manage its fleet, equipment, and machine parts. Yet the time and activities tracking for staff was still a slow, manual process completed by data clerks, which caused even more expense when merged (manually) into the fleet and inventory systems.

When mechanics at the department started to work on a piece of equipment, they used the fleet-management and inventory systems to log their activities online, and an HR manual system to log time on paper, which was then processed by data stewards into the HR system. Once the stewards painstakingly entered the data, others had to merge the data with cost information about parts and supplies utilized (e.g., oil change for tractor, versus oil change for a dump truck, versus an engine overhaul). The department sat on top of data in two areas: work-order management; and time and attendance, which produced true value when analyzed together.

After showing promising savings through paper-based systems, department managers made a clear case for why their time and activities system needed to be updated to a technological platform that would more efficiently tap the benefits of analytics. As a result, the department’s leadership looked for a system that would automate the workforce-management process, accurately track employee time, streamline payroll, and give employees access to their regular and leave-time information.

Under the department’s new time and activities system, management can now better plan activities and align budgets with real costs, while also addressing staffing, maintenance, equipment, and other needs in real time, instead of after the fact. Further, Arkansas not only retired clerks who no longer needed paper but redeployed others, who are now focused more on managing knowledge than on mind-numbing data processing and time entry. Other benefits include:

  • Improved efficiencies through strategic deployment of resources (decreasing redundant maintenance trips to the same area)
  • Consistency and standardization of data collection to improve accuracy and better analysis of payroll calculations
  • More discipline and better activity tracking for grant management
  • Better tracking of repairs based on actual activities and the ability to use maintenance history to predict future maintenance needs
  • Ability to identify trends and flag problems

As governments continue to invest in more advanced IT tools, they will show savings from effective workforce-management processes. One can anticipate an empowered workforce compensated not for activities, like oil changed and tires and fan belts replaced, but instead for keeping the fleet operable, where data is used to retire the obsolete and extend the life of equipment with preventive maintenance.

Employee Empowerment

Data produces results when effectively organized and delivered to the people who do the actual work. As important as it might be for managers to use data to enhance performance and reduce overtime, the true value results when those who do the public’s work avail themselves of this new treasure trove of information. Powered by digital tools, street-level workers now can pierce hierarchical and jurisdictional barriers to find answers to public service requests. In these chapters, we advocate a new approach to public employment that allows public employees to control more of their processes and accept more responsibility for outcomes.

This movement will take time. In the interim, officials can rely more on digital means to extract innovative suggestions from within the bureaucracy. In the mechanically driven work-order systems of the past, government workers would be dispatched to fill a pothole. If, on the way, they passed another pothole on the same block (or worse, if the homeowner pointed out another pothole to the workers), they would decline to change their instructions. After all, following the rules, not following common sense, drove work. And, of course, city officials did not trust their workers to make these decisions—out of fear, perhaps, that some inappropriate factor would influence them. Now, however, city workers in Boston more effectively manage work orders via mobile devices, which allow them to open new cases themselves. The mobile app enables before-and-after pictures of work, which are useful for management, promote transparency, and engage neighborhood residents who can see the work and, in some circumstances, the picture of the person doing the work.

Smarter Management

Analytics allow managers insights into labor productivity that were previously impossible. For example, the U.S. Office of Personnel Management (OPM) is taking steps to use data to change the way it manages hiring, training, and vacancy planning. OPM aspires to use data to facilitate better-informed hiring decisions by looking at mission-critical occupational areas, accession planning, retirement bubbles, and the effect of alternative incentives for early and voluntary retirements or retention, and more.

In HR, as in other areas, data capabilities no longer should, or need to, reside in some remote IT workshop. The management insights from data are most useful when available to a broad array of managers. Business intelligence tools provide such broad access, and OPM has encouraged chief human capital officers to avail themselves of the tools. As emphasized by OPM, a data warehouse accompanied by business intelligence tools allows ad hoc queries “on a number of subject areas, including (but not limited to): age, agency, employee, payroll, performance appraisal, personnel action, position, and retirement.” Knowing how to manage future vacancies will help government perform better. However, discoveries about other matters that affect hiring, promotion, and performance will not take root if trapped inside an archaic system that neither allows true discretion, nor rewards excellence, while also failing to hold employees accountable for poor performance.


In this chapter, I discuss how city leaders can learn from easily available data. Such insights often come from employees discovering lessons from the data. Yet these lessons also come when citizens can see data not previously available. Municipal budgeting has long been an arcane sport—one designed to control departments and impress taxpayers and rating agencies, but with little direct relationship to actual management, and even less to do with transparency and participation. Many municipalities have taken the step of “opening the budget” by putting it online. This transparency effort encourages civic participation, as well as more accountable government, but only when city hall provides usable information and presents it in an easily consumable way.

Mayors express policy through the budget, which, in turn, drives government by defining priorities. Budget processes across the country tend to be fairly complicated, political, and difficult to communicate. Citizens gain confidence and force better results when open government allows them to be innovators as well as watchdogs. In recent years, several city governments increased the level of transparency in their public budget process, opening their checkbooks and building budget portals to share data online.

Palo Alto, utilizing software from a Silicon Valley start-up, OpenGov, launched an easy-to-use portal with virtualization tools that provide citizens with more information and flexibility. For example, residents may search and download detailed information about government budgets, contracts, spending, subsidies, and tax expenditures across most, if not all, government functions. These portals offer a range of search-and-sort functions that allow residents to navigate complex expenditure data with a single click of the mouse, gaining access to checkbook-level spending information by agency, recipient, category, or purchasing office.

With only a few more clicks, taxpayers can also view expense and revenue items by department, type of account, and project; digging slightly deeper, users can find data details relating to their neighborhood’s police, library, or other city service. Users can find out how much each function costs, down to budgeted technology, workforce, or “bricks and mortar” expenses. The portal also includes budget history, allowing for historical benchmarks and comparisons of budgeted accounts for the current year against actual expenses from previous years.

In New York, four city council members launched a pilot that allows their constituents to democratically allocate $6 million in discretionary capital funds to community projects. More than 8,000 people participated in these four districts, funding everything from public school bathroom renovations to pedestrian walking-path repairs. The program is now entering its third year, and nine council members have endorsed participatory budgeting in their districts. Between September 2013 and April 2014, residents will directly decide how to spend $12 million in capital funds. The Participatory Budgeting Project, a national advocacy group, predicts even more involvement during the NYC 2014–15 cycle, with as many as 21 of the 51 council members committing their districts, and $25 million in capital funds at stake.

But participatory budgeting has yet to make the digital leap to include online deliberation and voting. The current process is a time-intensive commitment that involves a long series of face-to-face meetings and committee sessions. Project proposals tend to be low-tech, with little presence in online or social media platforms. Votes are counted on paper ballots. Albeit on a much smaller scale, the mayor of Westfield, Indiana, managed the digital transition by inviting his community to the polling and nominating process through Facebook to gather advice on priority projects for a new capital fund. Easy-to-use open-data budget portals that focus on citizen engagement, transparency, and accountability can save cities money through more efficient government operations, better communication, and citizen engagement programs, as well as more competitive contracting and lower risk of fraud and abuse within agencies.

Managing Performance

The CompStat and CitiStat approaches now serve as the foundation for transformative breakthroughs, when coupled with the examination of cross-agency data. For example, Louisville’s LouieStat uses data analytics to move the stat programs one step further. Led by a Chief of Performance Improvement, who combines a major open-data effort with a data-driven approach to identify the root causes of problems, Louisville can now better tackle those problems, rather than simply manage the efficiency of reacting after problems arise. Before LouieStat, for example, more than 300 inaccurate inmate fingerprints were returned each month. Agency staff initially thought that the problem stemmed from the hardware and software used in processing the fingerprints. But after looking at the shifts with the highest return rates, the corrections department realized that much of its staff had not received formal training in fingerprinting. Now every shift has at least one trained technician, the city is training more, and the return rate has dropped from more than 300 a month, to fewer than ten.

The Elements of Success

There for the taking: huge increases in operational excellence through the use of readily available data. We are on the cusp of a breathtaking set of government reforms that will unleash far more productive governance by combining the talents of an empowered public worker with a more engaged citizen. New digital tools flatten bureaucracies while enabling managers to know much more, in real time, about the productivity, effectiveness, and fairness of their employees. The reformers of the last century had to dramatically restrict discretion to produce accountability. Now we are on the verge of a new definition of accountability that furthers responsiveness.

The elements of success will include not just empowerment, but also the use of predictive analytics to answer new and serious questions about the root causes of problems. These breakthroughs involve more than technology: they also involve building a structure that encourages interagency capacity. Applying these fundamentals will result in much success, thereby moving government’s use of data away from simple, open-source transparency through participation, and toward integrated analytics.