Statistician tackles challenges as big data gets bigger
Massive amounts of data can be produced and stored today and at a much cheaper cost than ever before. The challenge for statisticians like Dr Tui Nolan is to develop effective methods for accurate analysis as “big data” grows ever bigger.
“Valid statistical analysis for big data is becoming increasingly important,” says Dr Tui Nolan, a proud Gudjal man, a scholar in mathematics, statistics and machine learning, and the recipient of the 2020 Chancellor’s Postdoctoral Indigenous Research Fellowship.
Put very simply, the problem is that the size and complexity of some data sets today exceeds the computational storage capacity of modern computer software. One response is to work on simple random samples of data, but Dr Nolan explains that this can lead to misleading results because scarce but nevertheless important information may be lost in the sampling process.
“Effective statistical procedures for big data applications need to handle large sample sizes by balancing statistical accuracy with computational efficiency,” he says.
Dr Nolan’s fellowship project will combine ideas from statistics, computer science and mathematics, and tap into the disciplines of physics, engineering and social sciences. The aim is to make significant advances in the statistical approach to data analysis, with his major goal being to develop a fast, efficient and accurate sampling scheme that can be used in modern machine learning algorithms.
Ultimately, real-world application could come through a new suite of software to be used in areas such as astronomy, robotics, economics and public health.
“To be able to process large data sets quickly, without losing accuracy, means governments can change policies in a timely manner to get better outcomes,” Dr Nolan says of the impact of some of these applications. He hopes his work will have an impact for Indigenous Australians, for example, with the proposed methodology directly applicable to more granular analysis of the factors behind health and educational outcomes.
“The challenges of big data provide new opportunities for statistics and modern society,” Dr Nolan says. “We require a shift in paradigm, where new statistical thinking and computational methods advance our understanding of modern [big data] applications.”
Dr Nolan is currently on a Fulbright Future Scholarship that has taken him to Cornell University in the United States. This followed his visit to the Alan Turing Institute in London—one of the world-leading centres in data science—as an inaugural recipient of the Australian Academy of Science’s Aboriginal and Torres Strait Islander Scientist Travelling Research Award.
Tui was recently selected to attend a 12 week pilot educational program at NASA’s Jet Propulsion Lab. For Tui, the NASA program allows him to work alongside the world’s best in statistics and mathematics, both propelling his career to the next level, and bringing knowledge and research back to Australia to inspire the next generation of students into STEM studies.