The Rise of the Data Scientists

By Chris Sommerfeld • January 26, 2015

As the volume, velocity, and variety of data that organizations have access to continues to increase, so too does the need for workers who are fluent in analyzing, interpreting, and visualizing this data. Cue the rise of data scientists and the growth in programs to train them.

While data fluency has become a skill of increased importance in government and policy, truly taking advantage of the power of big data often requires workers with specialized skills. Increased government use of data has the potential to deliver services more efficiently and improve the quality of life in communities. However, demand continues to outpace supply, leaving many cities with a severe shortage of qualified workers. A few years ago a McKinsey report predicted that by 2018 the US would face a shortage of 140,000 to 190,000 workers with deep analytical skills and of 1.5 million managers and analysts with big data skills.

Although dozens of universities are offering degrees in data science, demand is too great. “There is a shortage of big data experts,” said Michael Rappa, director of advanced analytics at North Carolina State University in an interview with Federal Computer Week. “I don't see the gap narrowing. Universities aren't producing enough.” The talent shortage has spawned the growth of several nontraditional or “boot camp” data science training programs. While many of these programs are specifically designed to help participants land plum jobs in the tech industry (with average starting salaries well over $100,000), as the programs continue to grow and expand likely so too will the variety of industries participants seek employment in.

There are over a dozen full-time boot camp programs graduating data scientists. These include programs such as Zipfian Academy, where participants spend 12 weeks building a portfolio of data science projects focused areas such as statistical inference, data engineering and visualization. Many of the boot camps are more recent upstarts, but some are also run by large education companies, such as Metis Data Science Bootcamp, which is owned by Kaplan. In addition to attending programs like these, more and more of the data-inclined are beefing up their data science credentials with online courses from companies such as Udacity and Datamonkey.

University of Chicago’s Data Science for Social Good is one of the few data science programs geared specifically towards the public sector. Its summer program, designed as a paid fellowship, trains data scientists for work in nonprofits, local governments, and federal agencies. Program fellows work in teams with front-line decision-makers on high-impact data analysis problems and are paired with a full-time academic or industry mentor who serves as a technical advisor and project lead. The program is organized by the interdisciplinary Center for Data Science and Public Policy, a collaboration between the university’s Computation Institute and the Harris School of Public Policy. Rayid Ghani, former Chief Data Scientist of the 2012 Obama campaign, leads the fellowship and serves as the center’s director.

The increased interest in data science has prompted a few schools to offer degree programs focused specifically on data science. Northwestern University has a Master of Science in Analytics. UC Berkeley offers a Master of Information and Data Science. Others universities have developed cross-departmental degree programs between computer science and other departments, such as Columbia University’s dual degree in Journalism and Computer Science and University of Chicago’s Master’s degree in Computational Analysis & Public Policy. University of Chicago’s innovative program (launched this year) aims to graduate students who will work as leaders in the intersection of data and public policy. Anne Rogers, associate professor in the Department of Computer Science, stated in a press release, “A lot of people who come through this program are going to become chief information officers or chief data officers.”

While data scientists are increasingly looked at to work as analysts, engineers, designers, and systems administrators, it is important to remember that data scientists rarely establish expertise in all of these domains. Accenture published an article with the Wall Street Journal declaring, “it takes teams to solve the data scientist shortage.” Given that data science programs still aren’t producing enough talent, organizations can begin to realize the potential of data science now by creating teams of people “who individually lack the full skillset of a data scientist, but as a group possesses them all.” Organizations that can’t find the talent they need may find this approach easier (if not more expensive).

Bringing deeply technical employees into the public sector remains a real challenge that goes beyond just increasing the talent pipeline. As a report put out last year by the Ford and the MacArthur Foundation suggests, “deep questions remain about the ability for many areas of government and civil society to identify, cultivate, and retain individuals with the necessary skills for success in a world increasingly driven by information technology.” Many government organizations will have to rethink how they recruit and retain this talent. This will likely require organizations to develop a more open and collaborative culture and to rethink compensation models and current training programs.

But cities don’t have to go it alone. For many, working with partners such as Code for America, a non-profit committed to the growth of civic technology, can be a great first step in transforming into a more technology and data-focused organization. In addition to incubating civic startups and creating networks of government peers and public volunteers, the organization also runs a fellowship program (similar in some ways to federal government’s Presidential Innovation Fellows). Bringing in tech- and data-focused talent on short-term fellowships continues to be a great way for organizations to test new models and ideas with minimal upfront investment.

About the Author

Chris Sommerfeld

Chris Sommerfeld is dual degree Master in Public Administration and Master in Business Administration candidate at the Harvard Kennedy School of Government and MIT Sloan School of Management. Before beginning his graduate studies, Chris worked as business strategist for Deloitte Consulting's innovation practice, Doblin. Previously he was a New York City Urban Fellow in the Mayor's Office of Operations and a second grade teacher through the Teach For America program. He recently interned at Amazon. Current research interests include technology-driven public service delivery, public and private sector data integration, and government innovation offices.