What Works Cities, an initiative of Bloomberg Philanthropies, pairs midsized cities with expert partners – the Government Performance Lab at Harvard Kennedy School, the Behavioral Insights Team, the Center for Government Excellence at Johns Hopkins University (GovEx), Results for America and the Sunlight Foundation – for technical assistance in better using data and evidence. After two years of work with 77 cities across the United States, the appropriately named program identified the key characteristics of a city devoted to using data to comprehensively identify, well, what works. Driven by the desire to share that knowledge more broadly, What Works designed its newly-announced Certification program to recognize high-performing cities across these criteria, to create an objective standard of success, and to help cities at any point in the data journey understand how they can improve their practices. As Jenn Park, Associate Director for What Works Cities, said, “We want to be able to show the world what the best cities are doing. The Certification program is made to be able to do just that – publicly validate, recognize, and celebrate cities that are doing this at the highest level.”
Certification measures a city’s work across criteria in the domains of open data, data governance, performance analytics, low-cost evaluations, results-driven contracting, and repurposing for results. The What Works Cities Standard – commit, measure, take stock, and act – has guided the What Works Cities initiative from the beginning, and the Certification criteria are divided into those four areas. The Standard represents phases of a city’s work to use data and evidence effectively, beginning with a mayor’s public commitment and concluding with using a deep understanding of city data to inform major policy and program decisions. Simone Brody, Executive Director of What Works Cities, described the Standard as “the North Star of what this work should look like.” She noted that based on demand from cities for a tactical guide to improving practices, Certification takes the theoretical Standard and translates it into concrete indicators.
Any city with a population over 30,000 is eligible to apply for Certification, and after a robust evaluation of their efforts, high-achieving cities will be recognized with silver, gold, or platinum certification. Applicants will be able to benchmark themselves against their peers and get a clear sense of where their individual practices are and in what areas they can move forward. Although What Works technical assistance is limited to midsized cities, participating in Certification allows far more cities to access resources and participate in a growing community of cities.
What Works Cities Certification fits into the existing landscape of initiatives in this space by recognizing governments that have developed a broad, citywide capacity for using data and evidence, rather than awarding specific successful initiatives. By measuring aspects such as establishing a person or team responsible for data standards and protocols, developing a process for releasing open data, and measuring the outcomes of key procurements, Certification focuses on the fundamentals of data-driven government in a way that other recognition programs do not. Elevating the day-to-day city work and processes that results in dramatic successes is an important contribution to the field.
Brody emphasized that even a city just starting out can benefit from the process, noting that the keys to success are accessible to any city: “What we’ve found is most important to being effective at this is a real commitment from senior leaders in cities, a real belief this is going to improve outcomes, and then giving folks in city government the space to be innovative and try new tools and practices. Any city can do that if they want to, and we’ve seen dramatic progress in cities just starting out, even in a few months.”
CASE STUDIES OF PLATINUM PRACTICES
What Works Cities Certification evaluates applicant cities on 50 criteria within open data, data governance, performance analytics, low-cost evaluations, results-driven contracting, and repurposing for results. The criteria are focused on the people, programs and policies necessary to improve the effectiveness of government. Below, we highlight examples of the cities in the What Works community that are already achieving selected criteria at the highest levels, both to provide inspiration and to illustrate what a top example of each criterion looks like.
Setting Strategic Goals in South Bend
Commit: Has your local government defined and made publicly available time-bound, measurable local government-wide strategic goals (e.g., reduce homicide by 20% in three years)?
The City of South Bend’s notable ability to set strategic goals has improved city government in a major way: helping Mayor Pete Buttigieg deliver on critical priorities and driving structural changes in the way the city addresses problems and services. By setting clear goals that drive work throughout the city, and reporting on those goals to residents, Buttigieg has created a high-performing government that is accountable for results. One outstanding example is the publicly-stated strategic goal of addressing 1,000 vacant or abandoned properties in 1,000 days, which started in early 2013. Mayor Buttigieg wanted to tackle the issue of blight, which residents told him was a priority during his campaign, in a visible way that allowed citizens to track the city’s progress.
The city’s commitment to addressing the vacant properties was measurable and available on the city’s website. Even when the process had issues, the public value was clear. Local media picked up on a bug in the progress tracking system that erroneously showed 100 pending properties as already addressed. The city’s Chief Innovation Officer Santiago Garces said this media revelation led to structural changes in the way that the city was tracking its progress with code enforcement. These changes, which included simplifying inspector checklists, requiring inspectors to take pictures of the properties, and assigning a central data analyst to do quality assurance, allowed the city to “improve the speed at which we were addressing the properties and we actually exceeded the goal that we had set,” Garces said.
Other strategic goals laid out by the South Bend city government include ensuring transparency and equity in policing, enhancing physical and technological infrastructure, and addressing mobility. In addition to addressing public concerns and creating action-driven strategic goals, the city consulted with the Drucker Institute to help with the framing of goals as well as the Center for Priority Based Budgeting, What Works Cities, and GovEx. Garces said working with outside groups was critical to build the “operational capacity and framework” and making tangible goals that address public concerns has been critical in building trust with residents.
A key thread running through the strategic goals is the emphasis on reporting progress and critical information to the public. The city is working to create transparency-oriented microsites on the open data portal that will report data and contextual information about specific goals to the public. The first such site, which is set to be released in Spring 2017, will focus on the strategic goal of “making sure the city has a 21st-century police department.” Garces added that these microsites will help the city better tell the story of what the city is trying to achieve and how it is progressing toward that goal.
Seattle's Open Data Policy
Commit: Does your local government have a codified open data policy?
The City of Seattle, Washington was one of the first U.S. cities to pursue open data, creating the first iteration of its open data portal in 2010 under Mayor Michael McGinn, a prominent proponent of government transparency. Since then, Seattle has established itself as a leader in open data, consistently increasing the volume and accessibility of available information. In 2015, Seattle was named a What Works City, and according to Seattle’s Chief Technology Officer (CTO) Michael Mattmiller, it came at the perfect time for building the city’s open data capabilities. “We were already thinking about how to re-engage the city and expand the open data portal’s use,” Mattmiller said. “One of the areas that we identified to focus on was this notion of creating an open data policy.” The city began work with What Works Cities partner the Sunlight Foundation to develop this policy. “It was very helpful for us to have the model policy language, to have specific policy objectives that we could work towards,” said Mattmiller. “But, we also realized that we had some unique aspects of Seattle that we had to mediate.”
For Seattle, it was important to develop a policy that fit the needs of the community, particularly in the realm of privacy. Mattmiller explained, “Before the What Works Cities engagement started, we had several missteps in our community about how we collected and used residents’ data…When we thought about opening more datasets, we felt the tension that our community was going to have between seeing this as a win for transparency and economic development and concern about what that data might do in terms of causing privacy harms.” The partnership with What Works Cities provided an impetus to mold an open data policy that worked for Seattle residents. The city partnered with the University of Washington and received a grant to develop a municipal privacy program, creating an action committee to establish a set of privacy principles to include in the open data policy. The city then reached out to the Seattle community for feedback on the policy, making a number of changes to the types of data to be opened. Seattle also established a network of open data champions in the city government to examine data before publication to monitor for invasive personal info and potential mosaic effects—combinations of datasets that together provide private information. To implement its privacy and open data policies, Seattle has partnered with the Future of Privacy Forum to identify and help mitigate risk present in its open data program, and contributed to research led by the Berkman-Klein Center at Harvard to share its policies with other cities.
However, publishing and protecting data was only the beginning for Seattle; the city then had to encourage city employees and citizens to use that data. Seattle began convening the open data champions and other city employees each month in an “Open Data Breakfast of Champions,” bringing in guest speakers to talk about applications of data. “Through these meetings we keep people enthusiastic and give them best practices,” said Mattmiller. Seattle also held a Data Camp, in which the city took employees off the job for three days for training on how to use the open data portal in addition to other data skills. Moreover, to promote resident engagement with data, the city created the position of Civic Technology Advocate, a data leader that goes into communities and hosts meetings, hackathons, and design labs to spread the goals of the open data program and empower residents to use this data. As a result, the city has seen the development of a number of useful tools and applications. For example, thanks to a Park Hackathon, developers created a tool using Parks Department trail data that helps users navigate Seattle’s parks.
By creating an open data policy that engaged users but fit the needs of Seattle employees and residents, the city was able to invigorate and institutionalize its open data program.
Kansas City's Data Inventory
Measure: Does your local government maintain a comprehensive data inventory?
Kansas City, Missouri’s comprehensive data inventory shows the importance of clear internal structures and processes to a successful, sustainable open data program. When Chief Data Officer Eric Roche realized how much time he was spending updating out-of-date, non-automated open data in the city’s portal, he embarked on a project to understand and inventory the data in all departments to develop a more systematic approach to open data publishing in the city.
Roche took a methodical approach to building the inventory: he drew on the relationships he had built through the performance management program, asked for organizational charts, and talked to individual departmental representatives. Through What Works Cities, the Sunlight Foundation and GovEx provided guidance on the inventory process. Roche acknowledged that not all department officials were data systems experts, but nonetheless, he and his team were able to find the answers they needed: “We asked what kind of work the departments do, how they track that work, where they store it and then backed our way into the more technical questions.”
This process has proved fruitful; Roche has been able to identify people who “speak data” in several city departments and this has led to positive relationships that yield results beyond the inventory. This peer-to-peer work in the city government has been the key ingredient in the building of a comprehensive data inventory for Kansas City. Roche said that the biggest lesson he learned was to “start small”— the city originally planned to complete an inventory of seven departments in 60 days, but adapted the plan to incorporate departments in an ongoing way that also builds capacity for data in other city departments.
Roche said the development of a citywide data inventory has given the city a thorough, well-documented resource that facilitates a more effective open data strategy. It allows the city to prioritize data releases based on key priorities and what can be automated, instead of just the “low-hanging fruit.” Technical difficulties are a common barrier to publishing certain city datasets, Roche said, as data systems are not always compatible with publishing online, but the inventory has served as a critical resource for the city in navigating such challenges. “The inventory gives me the ability to move on to the next thing,” Roche said. “It gives me the sense that there’s more out there, there’s a lot more valuable data to grab at any given moment.”
Results-Driven Policy in Boston
Measure: Does your local government measure outcomes, impacts, and/or cost-effectiveness of key procurements, contracts, and/or grants? (i.e., monitor performance data in real-time and troubleshoot with contractors to achieve the goals of the contract and/or grant)
Boston, Massachusetts has a robust open data, performance, and analytics program, so when the opportunity to engage with What Works Cities experts arose, government leaders looked to apply the power of data to their contracts through results-driven contracting strategies. They focused on the Department of Public Works Construction Management Division, which manages numerous contracts each year, with the help of Elijah de la Campa, a Fellow from the Government Performance Lab (GPL) at the Harvard Kennedy School.
Each year, Boston spends nearly $8 million on an asphalt resurfacing program for its 800 miles of streets. To ensure an equitable distribution of repairs, the city divides this work into three geographical regions and accepts bids for each. While the prior contracts included technical standards related to the quality of asphalt resurfacing, there were few mechanisms in place to enforce or incentivize vendors to adhere to the standards. Modifying the contracts for this program offered the city a chance to increase the overall quality of repaving efforts, to improve communication and transparency with vendors, and to enhance the articulation and measurement of outcomes crucial to the asphalt resurfacing process.
The Department of Public Works and GPL began assessing the existing procurement process by gathering information from stakeholders. De la Campa emphasized the qualitative and human-centric nature of this work, as he spent considerable time meeting with city engineers and vendors to understand concerns with the program, how they could be best addressed, and the viability of different types of performance payment.
In its new asphalt resurfacing contracts, the city has defined outcomes of interest related to pavement quality, the speed and progression of paving operations throughout the city, parking management, and environmental management, among others. Because the data to rigorously measure these aspects of performance did not yet exist, the city set up new processes for its engineers to track data. The contracts are now written with a clear set of outcome metrics, which are incentivized with a new performance-based payment structure. In addition to offering performance payments for meeting pre-specified progression of work benchmarks, the city will grade each contractor’s performance three-quarters of the way through the paving season, and then award additional in-season work for the final quarter according to vendor performance. The vendors benefit from the clear information about the city’s expectations and the incentives for high-quality performance.
Boston has now hired its pavers for 2017 using the new contracts, and will implement the performance-based payment structure for the first time this paving season. The new approach of results-driven contracting has many more applications throughout the city’s operations to help the city deliver better services to its residents.
Louisville's Performance Management Program
Take Stock: Does your local government convene a performance management program (i.e. Stat meetings)?
The City of Louisville, Kentucky’s performance management system, LouieStat, sets the bar for city government performance improvement. Mayor Greg Fischer united lessons from his business background with existing government stat models and unveiled LouieStat in 2012 to focus on two areas: planning and operations. “We needed to figure out how to plan, and we created consistent guidelines and language and a single coordinated strategic planning process that would help us measure the strategic areas of focus,” said Daro Mott, Chief of Performance Improvement in Louisville. “We also needed something that was more operational, which would have us measure the critical business processes—the processes that deliver the core of citizen services. We really needed to create a program that could answer the question of how Louisville could continuously improve on service delivery.”
Mott said that breaking the work into distinct strategic and operational categories was critical for the success of the system. “Operations should flow from the strategy of the city…If you start with data that you already have, you may not develop the right performance measures. You need to ask, ‘What are we planning to do and what data will help us understand how well we’re doing the work?’” This way, a city’s performance management efforts will center around its strategic priorities, rather than boosting performance on arbitrary metrics.
As a part of the planning process, Mayor Fischer developed a six-year plan with 21 city goals and asked each agency to develop its own goals and plans to achieve them. The Mayor’s senior leadership meets with senior staff from 18 of 20 departments four times a year and with other staff members between these forums. In these meetings, attendees discuss progress, look at metrics for the department and identify areas of weakness, evaluate the impact of city programs, and make data-driven decisions about where and how to best allocate resources. The Mayor attends many of these forums himself, and also meets with Mott on a regular basis to analyze Louisville’s performance on a citywide level. Mayor Fischer said what he calls a “weakness orientation” is key to making these meetings productive instead of punitive: “Bad stat programs are human- and people-focused and create more of a blaming culture. Ours is a celebration culture, focused on identifying broken processes or bad data and then fixing that and celebrating the people who do the work.”
In order to promote buy-in from so many departments, LouieStat from the beginning sought to demonstrate its utility to agencies. According to Mott, “What really got us more buy-in was facilitating process discovery workshops with departments by which we documented the critical business processes of each department and talked about measures linked to these processes.” In doing so, the Mayor’s Office introduced departments to performance management—and showed how performance management could help identify and track metrics to improve service delivery.
The performance management culture has become increasingly embedded in Louisville’s agencies. Mayor Fischer points to this as a critical aspect of developing a culture of performance; he said, “We provided training for people to understand how to solve problems, which has given them a sense of not just empowerment, but fulfillment and hopefully joy in their work, where now they feel they are in control of making things better.” The Office of Performance Improvement has trained at least one staff member in each agency to lead the LouieStat process and analyze that department’s data. Most data analysis now happens at the departmental level, and agencies have come to embrace a performance-based approach, learning to adapt LouieStat to their various needs.
Washington D.C.'s Low-Cost Policy Evaluations
Act: In the last 12 months, has your local government used the results from low-cost or randomized evaluations to improve local government programs or processes?
In the past year, Washington, D.C. has dramatically increased its efforts to use low-cost evaluations in policymaking. Boosted by commitments to data-driven decision making from the Mayor and City Administrator, the city launched The Lab @ DC last year. The Lab brings diverse scientific skillsets in house to enable the city to use low-cost interventions and other research methods throughout its operations.
By basing the team in the Office of the City Administrator, The Lab builds on existing relationships, processes, and data infrastructure. Chief Performance Officer Jenny Reed noted that the connection to performance management surfaces ideas and also ensures that the work is tied to the city’s priorities. Lab Director David Yokum said that in order to identify opportunities for low-cost evaluations, “Having scientists inside government is a strength. You really need to know a lot about the agencies, what they are capable of doing, what their budgetary constraints are, what their IT looks like – you need all those pieces to make the scientific judgment of what the opportunities are.”
The Lab is already embarking on a variety of efforts, including testing redesigned paperwork for applications to Temporary Assistance for Needy Families (TANF). For a project with the Metropolitan Police Department (MPD), The Lab designed a randomized controlled trial for the rollout of body-worn cameras. Because the MPD was already planning to distribute the cameras and already collected relevant administrative data, adding in the randomized distribution had an extremely low marginal cost. The randomized trial will allow the city to compare the outcomes for officers with cameras to those without cameras to answer important questions about the technology’s effectiveness. Support from the Behavioral Insights Team through What Works Cities this year will facilitate additional projects.
Yokum said that even doing a small concrete project, such as testing two subject lines for an email, can inspire departments to come back with more ambitious ideas for ways to apply the same methods to other areas. The goal of the team is to support talented employees in all departments and raise the city’s collective capacity to use evidence to drive policy. Although it is still relatively new, The Lab @ DC demonstrates promise to scale to an evidence-driven District government.
Using Data to Repurpose Funds in Jackson
Act: In the last 12 months, has your local government shifted funds away from a practice, program or policy that, through rigorous data analysis and evaluation, has consistently failed to achieve desired outcomes toward a more effective and efficient practice, program or policy?
Last year, Jackson, Mississippi faced a challenge familiar to many cities: a need to make significant budget cuts while trying to preserve jobs and maintain service delivery. Because of Mayor Tony Yarber’s commitment to data and the city’s prior achievements in developing an open data portal and launching a performance management program, the solution was obvious: turn to data to restructure and repurpose funds.
Beginning in May 2016, four months before the budget season began, Jackson began a comprehensive effort to analyze its spending, programs, and results to identify opportunities for efficiencies. The team responsible consisted of directors, deputy directors, executive staff, fiscal officers, and on-the-ground support workers. This team worked with the Johns Hopkins Center for Government Excellence (GovEx) through What Works Cities for technical assistance in data analysis.
With the help of GovEx, the city analyzed its budget in comparison to comparable cities and identified areas of disproportionate spending. GovEx also trained every departmental data coordinator and fiscal officer in how to analyze and visualize their own data with Tableau to enable ongoing data use. The city then used this work to analyze each department’s programs more granularly, which leveraged the city’s JackStats performance management framework to identify which were producing results aligned with the city’s priorities.
The city made changes in many departments based on the analysis. The Human & Cultural Services Department merged low-performing senior and child care centers with higher-performing ones. In other departments, staff positions related to underperforming programs were repurposed to higher-impact areas in order to avoid layoffs. The city also looked at departmental structures to identify existing functions that would be more efficient under the purview of another department, such as moving tree and limb removal from Parks and Recreation to Public Works, which owned the necessary equipment.
Mayor Yarber and his budget team saw record turnout at community outreach events related to the budget. Basing decisions on the data helped the Mayor and his senior staff have difficult conversations with departments and with the community. The dashboards and visualizations that the budget team used are available to the public and all departmental employees, ensuring that the data behind the decisions is transparent.
Justin Bruce, Director of Innovation and Performance, emphasized the progression of work with data that developed the capacity that made this effort possible. He said, “Open data allowed us to take data, clean it, work with it, and actually track progress and performance. JackStats helped us look at the data at a more granular level to show us why and how we are meeting goals.” The latest effort, he said, “allowed us to take performance management to a different level, not just looking at what we are doing but how efficiently and effectively we are accomplishing our goals.”
Jackson was able to cut its budget by 7.6% with this process. The city went from a $14 million deficit in 2014 to a $6 million surplus this fiscal year, in addition to creating a new reserve fund. The city plans to continue its priority-based budgeting and repurposing work in the same way going forward. Bruce said, “Every time we touch an issue that has data to guide the situation, we’re always going to do an analysis of that data and always going to determine what’s most effective and efficient and ultimately, what’s best for our citizens.”
The stories above illustrate some of the many ways that leading cities are leveraging the power of data and evidence. By objectively recognizing successes like these and providing a roadmap for critical data practices, What Works Cities Certification will enable even more cities to improve the way they work and deliver better services to their residents. For the first time, an objective organization has produced comprehensive and reliable criteria for high performance that will produce a roadmap to operational excellence for mayors aspiring to improve the quality of life in their cities.
This brief was written in conjunction with Harvard Kennedy School’s Katherine Hillenbrand, Project Manager; Eric Bosco, Research Assistant/Writer; and Chris Bousquet, Research Assistant/Writer.