This article originally appeared as part of a paper on What Works Cities’ Certification program.
With help from What Works Cities, the City of Scottsdale has pursued behaviorally-informed interventions in a great variety of areas. The city has leveraged insights from behavioral psychology to design low-cost evaluations—A/B tests that compare the effectiveness of a number of interventions on a representative sample, also called randomized control trials (RCTs). Thus far, the city has developed low-cost evaluations encouraging employees to complete health risk assessments and set aside money for retirement, nudging residents to donate to local charities and pay water bills online, and recruiting a diverse group of new members to the police force and fire department.
According to Cindi Eberhardt, Scottsdale’s City Volunteer Program Manager and behavioral insights team leader, the choice of initiatives was strategic on the city’s part. “After we decided to partner with What Works Cities, we went to our executive team and asked who would be interested in sponsoring a trial,” she explained. Doing so ensured that the departments conducting trials had a genuine interest in and would follow through on the evaluations. The city’s police and fire department, for example, immediately spoke up about their interest in help recruiting a diverse cohort of new employees.
The other consideration was to identify policy areas that both addressed critical city goals and possessed the characteristics required to conduct low-cost evaluations. “Our approach was to look at Council priorities and identify all the places we could use data from RCTs,” said Brent Stockwell, Assistant City Manager. Conducting RCTs essentially requires two things: sufficient data and a human behavior to influence. For example, encouraging employees to fill out health risk assessments is a good subject for an RCT because there exists good data on whether or not employees complete the assessments and the desired behavior is obvious. On the other hand, a proposition like getting city employees to eat more healthily is not a good subject, because there is no way to track employee eating habits and the desired behavioral change is not clearly defined—should they eat more vegetables and drink less soda, or eat more protein and fewer carbs, or should this depend on the employee?
In an effort to embed this behavioral work into the city’s day-to-day operations, Scottsdale has created an internal team to work on behaviorally-informed interventions. While still in its early stages, the team has drafted a charter and brought in department leaders to discuss potential new areas for low-cost evaluations. Eberhardt described these meetings as focusing on three goals: “One, we wanted to educate department leaders on what the city has done. Two, we wanted to talk about potential areas in which we could provide value to their departments. And three, we wanted to discuss some of the challenges involved in finding data and determining outcomes to track.” The team is currently putting its structure and training in place, and Eberhardt says that they hope to finish four new projects by the end of the fiscal year next June.
However, as Eberhardt and Stockwell would admit, making behavioral interventions a regular feature in city policy has had its challenges. “With some of the projects we’ve done, the departments have gone back to business as usual,” said Stockwell. Part of the challenge is overcoming inertia in city departments, many which have completed the same tasks in the same way for many years.
According to Stockwell, facilitating strong organizational health is one means of overcoming this inertia. “You need an organization where people don’t think they have the answers to everything. You don’t test interventions if you know you’re always right. It’s critical to have that humility and curiosity,” said Stockwell. “Organizational health was part of what made the interventions in the police department easier and successful,” he continued.
Improving communications with departments can also help encourage continued commitment to behavioral interventions. On one hand, Eberhardt stressed the need to manage expectations upfront, communicating to departments that behavioral policies are not a silver bullet to all their problems. At the same time, “It’s important to have additional conversations with sponsors and individuals to ensure you emphasize the value of the work,” she said. Eberhardt pointed to the Behavioral Insights Team’s recent article on Medium “Eight Things Cities Can Do Today to Generate Evidence and Outcomes” as an example of the type of work that cities should produce and disseminate. The designers of a behavioral intervention should stress that such policies can produce significant results, but ensure that departments do not become disenchanted if interventions do not lead to radical change.
The other key lesson learned in Scottsdale was the need to create mechanisms for following up on initial evaluations. Eberhardt emphasized the value of management agreements that establish when a department will implement a behavioral policy or duplicate a project in another area. “You need capacity in place to follow up and make sure changes are actually implemented,” she explained. Doing so can ensure a city produces not only valuable insights from trials, but also meaningful behavioral change on a large scale.