Data-informed, Equity-focused, Expert-driven


More than two years since the start of the COVID-19 pandemic, it is sobering to account for all we’ve experienced as a nation. At the time of writing, the United States alone is approaching 80 million total cases and 1 million total deaths. The damage done to the social fabric of our communities and the nation writ-large has been harrowing. The cost to the broader economy has not yet been fully quantified. The inequitable impacts to communities that have been historically marginalized, and families of color have been stark and laid bare for all to see.

Today, we’re all struggling with the lingering impacts of the pandemic on our families, communities, and local economies, as well as the unintended consequences of the nation’s policy responses. As we attempt to course correct, we are also wrestling with the wicked challenge of building a better and more equitable economy and nation. It has been and continues to be, to say the least, a challenging time for the American people and the local leaders that fought this pandemic on the front lines.

Yet despite our challenges, it is important to take a moment and reflect on the incredible innovations in local government that have come out of this crisis. In the face of this pandemic, local government leaders have leveraged the limited tools and resources at their disposal to take innovative and data-driven actions in response to local challenges and to protect the most vulnerable. A leading example at the forefront of this battle comes from Milwaukee County who has served as an example for local governments across the nation for how to execute an equitable public health response.

Prioritizing Equity in Pandemic Response

In June 2020, we featured Milwaukee county’s COVID-19 surveillance dashboard, praising the county for serving as a test case for how to prioritize racial equity in public health response. Having launched the dashboard in March 2020, Milwaukee was one of the first local governments in the US to publicly share COVID cases and deaths by race and ethnicity. In launching the dashboard, County Chief Health Policy Advisor Dr. Ben Weston’s goal was to be able to identify communities at highest risk of adverse health impacts in order to target public health interventions and resources through an equity and social vulnerability lens.

After significant positive feedback and interest from the Data-Smart community, we followed up with Dr. Weston to better understand how the dashboard has evolved throughout the last two years and the impact it continues to have on their pandemic response.

Dashboard Evolution

The first iteration of the dashboard monitored hospital bed and ICU capacity, and highlighted total cases, hospitalizations, and deaths by race, ethnicity, gender, age, and location. The original purpose of the dashboard metrics was to track high level trends to inform the County’s Unified Emergency Operations Center (UEOC). With medical direction by Dr. Weston, the Center brought together public health, government, academia, business, education, community organizations, and first responders under one body for regional decision making, knowledge sharing, and strategy development. As conditions changed throughout the pandemic, the dashboard similarly evolved.

In April 2020, Dr. Weston and the UEOC incorporated more robust hospital data, including a breakdown of ICU bed availability and ventilator utilization to monitor health system capacity. This was coupled with more extensive testing data to gather a more complete picture of disease burden in local communities. According to Weston, “the dashboard allowed us to see not just how many tests were being conducted in our communities but also testing and positivity rates, and also tracked how quickly the lab was processing them.”

While having case, testing, and positivity rates on one dashboard gave the UEOC a better understanding of pandemic conditions, it was still challenging to act with this data without a clear understanding of how these data points intersect at a hyper local level. Dr. Weston noted that it can be challenging to interpret case rates without understanding what's going on with testing and positivity.

“If nobody's getting a test, you're going to have zero cases. And if everybody's getting tested, you will have a lot more cases. So, then the positivity rate becomes a lot more relevant. However, if you have two communities with the same positivity rates but one community has very low testing rates and the other has very high testing rates, there's a different picture of what's going on in those communities. If we don’t tie these data points together, we can’t truly understand what’s happening.”

To overcome this challenge, Dr. Weston and his team developed the Strategic Allocation of Fundamental Epidemic Resources (SAFER) model (see below), which compares testing and positivity rates by community.

In the SAFER model, green areas indicate high testing rates and low positivity; blue indicates low testing and low positivity rates; yellow indicates low testing and high positivity rates; red indicates high testing rates and high positivity. In essence, green areas are of low concern, blue has unknown disease activity, yellow is an at-risk area in need of greater testing capacity, and red is an area in need of significant public health resources. The county leveraged the SAFER model to prioritize resource allocation and inform public health messaging as the virus spread across communities. You can explore the SAFER model here.

Screenshot of the SAFER model site, showing comparisons between current and previous allocation of resources

Then, as vaccine roll out began, Dr. Weston sought new ways to monitor and track vaccine delivery and visualize vulnerability to determine which communities to target for vaccination sites and messaging. “When vaccination started, it just didn't hit vulnerable communities because the initial supply was for health care workers and first responders,” said Dr. Weston, “As we prepared for more supply, we tried to think of how we could visualize which communities to target.”

The team first tried to map vaccination rates by community. While this was helpful initially, “there are multiple reasons why people don't get vaccinated,” said Dr. Weston, “And if you go into a population that is largely against the vaccine, you can knock on all the doors you want but you're not going to make much progress. However, if you reach into a population that has different barriers to vaccination, you knock on those doors, you might make a lot of progress. And if that community is facing significant barriers, they are also likely more susceptible to poor outcomes.” This realization led to the development of the EVE model.

EVE stands for “Evaluating Vulnerability and Equity” and the model combines vaccination rate and social vulnerability (using the CDC’s Social Vulnerability Index) into a four quadrant chart designed to identify communities that are high vulnerability, low vaccination (see below). From the beginning of the pandemic, Milwaukee County identified social vulnerability as a clear risk factor for getting sick from, hospitalized with, and dying from COVID. “And now, clearly, it's a risk factor for not getting vaccinated and not having access to the vaccine as well,” said Weston.

On the map below, orange communities are high vulnerability, low vaccination. Purple communities are low vulnerability, high vaccination. Yellow communities are low vulnerability, low vaccination. Purple communities are low vulnerability, high vaccination. Like the SAFER model, the EVE model has been and continues to be used extensively by the county’s leaders and other stakeholders to target communications and messaging, interventions, and strategy. You can explore the interactive dashboard here.

Screenshot of the EVE Model dashboard showing different levels of vulnerability v. vaccinations

In the early days of the vaccine roll out, almost every census tract in the county fit into orange or purple. County leadership believed that this was largely due to poor communications around eligibility requirements. To simplify strategy and communications, Dr. Weston leveraged the EVE model in collaboration with county and state leadership to identify the 10 orange zip codes with the highest vulnerability and lowest vaccination rates. Together, the state and county worked together to expand vaccination eligibility to anyone over the age of 18 in those communities and “suddenly we saw this huge shift where thousands and thousands of people came in and rates in those communities shot up,” said Dr. Weston.

The county has built off of this early success, using the EVE model to set up targeted vaccination sites and identify homebound individuals for in-home vaccination appointments. To track progress, the dashboard tracks vaccination rates by vulnerability level over time. As shown below, in December 2021, only 10% of first vaccine doses were going to the highest vulnerability groups on a daily basis. Today, more than 60% of first doses go each day to the highest vulnerability groups. “We credit data transparency and the EVE model for our success,” said Dr. Weston.

Graph showing COVID vaccinations with high vulnerability populations increasing vaccination over time

As cases and deaths have declined, the county continues to use the dashboard and the EVE model to take a measured and cautious approach to reopening, with a focus on ensuring the most vulnerable residents are at the forefront of the decision criteria. They are also considering reintroducing the SAFER model to help make these decisions. “I think it's important to be guided by objective metrics that take vulnerability and equity into account,” said Dr. Weston, “The EVE and SAFER models have been key in holding ourselves accountable and helped to prevent us from moving the goalposts, ensuring every decision is made through an equity lens.”

While pandemic conditions are improving in Milwaukee and the rest of the country, Dr. Weston and county leadership continue to keep a close eye on the dashboard. “We just saw a tidal wave pass and we’re on the tail end of it,” said Dr. Weston, “If there’s anything we’ve learned from this pandemic, it is that we shouldn’t wipe our hands and say we’re ready to get back to normal just yet.” He hopes there will soon come a day where he doesn’t have to look at the dashboard daily and that it becomes more of a true bio-monitoring system with automated alerts that watch for irregular spikes in COVID-related activity. However, he acknowledged that until there is global vaccine equity and access, new variants are likely to emerge. Until he sees what he calls a true “steady-state” for at least six months where cases continue to decline and level off, the dashboard will remain front-and-center in Dr. Weston’s daily work.

Barriers Persist in Closing Equity Gaps

Despite the tremendous efforts and great progress made in the county, there are still significant disparities in vaccination rates. At the time of writing, 64 percent of Milwaukee County’s white population are vaccinated, while just 46 percent of the African American and 47 percent of the Hispanic population are vaccinated. Nonetheless, Dr. Weston and his team are working tirelessly to chip away at these gaps to get as many county residents vaccinated as possible.

Dr. Weston noted two key barriers; “I don't think this is unique to us, but when I think of folks who are unvaccinated, they generally fall into two groups. One is people who are against the vaccine and will not get vaccinated. Then there's also a portion that are not getting vaccinated because of systemic barriers and misinformation. And these two overlap a lot. The second group tends to fall into the higher vulnerability category.” He continued to note that the county is deploying resources directly into vulnerable neighborhoods, going door to door to reach those hardest to serve. The EVE model continues to play a leading role in this effort, helping to prioritize areas with the greatest need.

A Data-informed, Equity-focused, and Expert-driven Response

When asked to provide advice for other local leaders looking to replicate the county’s work, Dr. Weston emphasized the need to be “data-informed, equity-focused, and expert-driven,” not just for pandemic response, but all things relevant to the work of local government.

In Milwaukee County, there is not one central health department, but 11 decentralized health departments serving different regions. In building the COVID-19 surveillance dashboard, all 11 municipal Health Officers were able to operate in a unified and collaborative way using one centralized source of truth for data.

Additionally, across these health departments, there is widespread agreement that race and socioeconomic status are preconditions for vulnerability. Long before COVID-19 was part of our vocabulary, leadership across the county and city of Milwaukee, as well as the State of Wisconsin (including the Governor), put racial equity at the forefront of public policy decision making. As noted in our previous article on the county’s efforts, racism was declared a public health crisis in Milwaukee in 2019. This placed the needs of underserved communities of color at the top of the consideration criteria when making decisions on behalf of the public, and the COVID-19 surveillance dashboard was just an extension of those initial efforts. “We’ve been lucky to have widespread agreement among our leaders throughout the pandemic. Had we had different leaders, I am not sure we would have been as successful,” said Dr. Weston.

Lastly, county and city leadership understood the importance of having the right experts to inform decision making throughout the pandemic. “It's not to say that we should just let the health experts make all the decisions,” said Dr. Weston “It's to say that our elected leaders are the ones who are going to make the final decisions. But hopefully, they'll make them with the best information they can get from the best people in whatever the critical field is, whether it's public health, whether it's economics, or something else."

Keys to Data-informed Success

In reflecting on his team’s data-informed efforts over the last two years, Dr. Weston shared the county’s keys to success:

  1. Data transparency and embracing discomfort: In April 2020, the Washington Post published an article highlighting nine local governments across the US that were publicly sharing COVID-19-related race and ethnicity data. Of the nine localities, Milwaukee County had the worst racial disparities. For Dr. Weston, it has always been incredibly important to be transparent about these disparities and to embrace the discomfort that accompanies it. “From the very beginning, we were able to use this data to articulate an incredibly powerful message to all our stakeholders that equity is going to be front and center in our approach. We didn’t try to hide or shy away from these realities, instead, we used data to act and that enabled us to gain the buy-in we needed to drive our initiatives forward.”
  2. Place-based approach: From the very beginning of the pandemic, the dashboard’s landing page has always featured a map of the county. According to Dr. Weston, this has helped to personalize the data for end-users, including the general public. “A lot of people will look at this dashboard for the first time who may not fully understand what all the data means and may get overwhelmed, but almost everybody knows where they live on this map. And the first thing they will do is look at where they live, and the map is an accessible, interactive, and engaging way to ground people in the reality of what’s happening in their community.”
  3. Automating data collection and analysis: In the early days of the dashboard’s creation, the county’s data team spent hours a day manually entering data and maintaining the dashboard. This was incredibly resource and labor-intensive and made it difficult for the data team to focus on anything else. Since then, the team has incrementally built in automation to streamline data entry and analysis. Dr. Weston proudly noted that roughly 90 percent of the dashboard is now automated, which enabled county leadership to have a near-real-time view of disease spread and has also freed up the data team to conduct higher value tasks.

Important Implications for Public Health Response

Looking beyond the pandemic, Dr. Weston hopes to take the lessons learned from his experience with the dashboard to inform other public health response efforts. In fact, in early March 2022, the county released an opioid surveillance dashboard in partnership with the Medical College of Wisconsin that borrows heavily from the COVID dashboard, evaluating geographic trends and disparities by race and ethnicity. For Milwaukee and other local governments around the world, Dr. Weston wonders, “Can we do something similar for gun violence? Can we do similar for motor vehicle crashes? Can we do something similar for diabetes management?" There are significant implications from Milwaukee's work in addressing some of our nation's deepest and longest-standing inequities.

For all the success and lessons learned by Milwaukee County's leadership, the sky's the limit as far as how local leaders can apply methods like the SAFER and EVE model to break down systemic barriers and injustices in our public health system. The Data-Smart Community would be well served to replicate this work to address public health challenges in their cities.

About the Author

Matthew Leger

Matt Leger is a Research Assistant for the Innovations in Government Program at the Ash Center. He has a diversity of experiences in research across the public and private sectors, as well as in academia with a primary focus on understanding how technology can be used to help address some of society’s greatest challenges. Matt has worked with the Smart Cities Strategies team at the International Data Corporation (IDC); the NYCx team in the NYC Mayor’s Office of the Chief Technology Officer; and at the research institute CTG-UAlbany. He holds a Bachelor of Arts in Public Policy and a Master’s Degree in Public Administration both from the Nelson A. Rockefeller College of Public Affairs and Policy at the University at Albany in Albany, NY.