Ag Data Services: Turning data sets into data assets

The cows at Homestead Dairy in Plymouth, Indiana, are blissfully unaware that every minute of every day they’re generating mountains of data for Luiz Brito, assistant professor of animal sciences. Sensors are everywhere on the commercial farm where the cows are videotaped day and night. Automated feeders record each calf’s milk intake and eating habits, while milking robots collect comprehensive production, activity and behavior data.

Brito, whose research goal is to improve the cows’ well-being and productive efficiency through genetic selection, needed to integrate this automatic flow of data with more than 70,000 genomic markers for each animal obtained from hair samples taken at birth. He also wanted to add in 10-plus years of historical data to increase his statistical power to ask increasingly detailed questions. And he wanted to be able to share this large and integrated data set so he could partner with colleagues in other disciplines.

To help him with these tasks, Brito called on Ag Data Services (ADS), a new team supporting researchers through the important steps between collecting data and applying it to discovery.

Catching up with data collection

“Across the college, there have been huge advances in the many technologies that we use to collect research data, but they’ve been outstripping our ability to store, organize and use it,” explains Aaron Walz, director of ADS.

ADS was established about a year ago as a partnership between Agriculture Research and Graduate Education and Agriculture Information Technology  (AgIT). ADS works with researchers in the College of Agriculture and their collaborators in the colleges of Science and Engineering, Purdue Polytechnic Institute and other universities.

ADS is a distinct team within AgIT and unique in the variety of disciplines it pulls together, Walz says. Team members bring skills in data science, statistics, data management, programming and databases, data visualization and storytelling, and geographic information systems (GIS). They share one purpose, however, Walz adds: “Ag data is our full-time job.”

Never-ending plant data

Where researchers once traveled to measure and assess their research plots by hand, that information today flows back to campus automatically from cameras and sensors on tractors, robots, drones, gantries, and other innovations that are driving a data-powered revolution in agriculture.

Walz cites, for example, high volumes of data streaming in from Purdue’s Ag Alumni Seed Controlled Environment Phenotyping Facility (AAPF), Indiana Corn and Soybean Innovation Center (ICSC), and the Agronomy Center for Research and Education (ACRE).

“We’re creating tons of data,” agrees Yang Yang, director of phenomics. “Big data” refers not just to the amount of data, but also to the many different types of it, he notes.

“We have wonderful facilities that collect and deliver the data pretty awesomely,” he says. “The better you’re doing your job in the front end, the more you need an efficient way to extract the information you need.”

Yang has worked with ADS to establish a data management system that strengthens researchers’ abilities to search, compare, mine, and model data. “I count on Aaron’s team to provide the IT infrastructure,” Yang says.

He wants to make sure researchers using the AAPF and ICSC can leverage their data to benefit farming. The data that adds value for Yang and these plant science researchers are ultimately the ones that help them help growers reduce farm inputs while maintaining or enhancing output.

Back to basics

Although ADS deals with big data, it approaches each project the old-fashioned way – by first meeting with faculty members to learn about their research needs and goals. “We take a step back,” Walz says, “and ask about their research: what kinds of data they’re collecting, what they’re doing with it, and what challenges they have.’”

The faculty in turn ask questions: “What’s the best way to organize and store all this data given the many options and platforms available? How can I automate some of the manual steps in collecting and processing my data? How can I leverage advances in artificial intelligence and machine learning to gain insights from my data, or visualize data to share with stakeholders?”

 “We work with people who are designing and building sensors, and others who just want to be able to use them,” Walz adds. “We help them navigate the journey, no matter where they are.”

A lot of data comes from Purdue’s research farms and relatively small plots, challenging the researchers to scale it up. “One of the things people in ag talk about is making big data from small data,” Walz explains. “What we ultimately want to do is take the data we’re collecting in different places and different experiments and combine it to make a larger data set to draw broader conclusions.”

Along the way, ADS team members are also collecting best practices to solve common problems across the college using common tools.

Walz encourages College of Agriculture researchers to explore how his team can help with data pipelines, data stewardship, and data analytics. “When people, especially new faculty, are thinking about research data, we encourage them to reach out to ADS,” he says. “We can connect you with the resources you need.”