From Hieroglyphs to Hadoop

By Katherine Hillenbrand • April 9, 2013

Governments have collected civic data since the earliest civilizations. The ancient Egyptians counted cattle and measured oil usage to levy taxes. Over time, these papyrus records evolved to reams of paper in filing cabinets comprising everything from birth records to maintenance reports. Prior to the advent of affordable and ubiquitous digital systems, this data was collected and filed with little attempt to use it to improve governance or to unlock insights into government operations.

SURVEY DETAILS CIVIC TECHNOLOGY SURVEY The National League of Cities, in partnership with the Public Technology Institute, issued a national survey of municipal technology and civic participation practices in early 2013. The brief survey, issued between the 29th of January and 24th of February, was completed by approximately 60 people. The overwhelming majority were Chief Technology Officers, IT Managers or the equivalent. Participation included 44 cities and 10 counties, plus several non-local governments (who are not included in the analysis), of a diversity of sizes: Population Count 50,000 or less 14 50,000 to 300,000 21 Greater than 300,000 19 NLC PTI Logo

Today, with more and more government operations moving online, the era of digital civic data has arrived. Cities across the U.S., both large and small, have taken advantage of new technology, creating apps, opening data sets, and restructuring government offices to allow greater room for innovation. Digitized civic data opens new realms of possibility; while few insights could be gleaned from the tedious process of leafing through paper records looking for trends, digital data analysis is now nearly instantaneous. These new technologies also enable agencies to share records with one another, exponentially increasing the abilities of government to operate efficiently and effectively.

According to a recent survey (see sidebar for details), 89% of municipal governments have accomplished or are interested in open data initiatives. Governments at the city, state, and federal level have all begun opening their data sets to the public in standardized formats. The U.S. government’s portal,, features data sets as varied as the work schedules of railroad employees, historical earthquakes, and detailed information on the country’s tomato crop yield and pricing. On the municipal level, data sets located at include rodent sightings, 311 calls, and building permits issued.

Opening data sets is the first step toward creating enormous public value with technology. To date, government, the private sector, and the public have all made use of these data sets in far more innovative ways than have ever been possible before. In the aforementioned survey, 96% of municipal governments have accomplished or are interested in building mobile apps. Such technologies are accessible to governments of all sizes and budgets, particularly with the assistance of new civic tech-oriented groups that have emerged in this space. For example, the nonprofit Code for America provides technology fellows that serve in city governments for a year. Fellows have produced apps like DiscoverBPS, a web app for Boston parents to be better informed while selecting their children’s schools; BlightStatus, an app for New Orleanians to learn more about vacant properties in their neighborhoods; and, which informs Austin residents of their risk during wildfires. All of these projects provide significant value for citizens that could have never been imaginable in the days of paper-only records.


Private companies have also incorporated government data into their products, adding another layer of public value to opening data. Google Maps’ public transit directions rely on open, standardized government transit data. Yelp recently announced a partnership with San Francisco to include official restaurant inspection data in its review pages. Real estate company Trulia uses open data on crime and commute times to help homebuyers make better decisions.

Finally, cities have begun institutionalizing this cultural and technical shift with the creation of new positions like Chief Data Officers or Chief Innovation Officers, signifying that technology and innovation are mayoral priorities. Staffing changes are more costly than other kinds of civic data initiatives; consequently, as shown in the graph below, the majority of these positions exist in larger cities.


The preponderance of city apps and open data projects signals a new era in the use of civic data. No longer will valuable public data be relegated to locked file cabinets. As analytic technology increases in scale and drops in price, governments everywhere will be using their data to discover hidden correlations, to become more efficient, and to better serve the citizenry.

About the Author

Katherine Hillenbrand

Katherine Hillenbrand is the Assistant Director for Digital Platforms and Practitioner Content at the Ash Center's Innovations in Government program. She has been working on the Data-Smart City Solutions project since 2012 in different roles. She holds a Bachelor of Arts degree from Amherst College and a Master in Public Administration degree from Harvard Kennedy School.