Humungous Data

Aug 22, 2012

by

In:Data

No comments

ieee_ethernet_future_demand1If you thought “Big Data” was already quite unmanageable, IEEE predicts a 1500% (x15) growth in data by 2015. That is 3 years from now.

On a similar scale, IEEE also suggests that terabit networks should be implemented soon to cater for demand in network traffic by 2015. This is up by x40-1000 times from today’s gigabit networks.

This probably also suggests that demand for data processing and delivery will need to increase by a similar scale. To some 10-40 times.

What products and skills will power the delivery of services for “Humungous Data”?

  • New Data systems – like GFS, BigTables, Hadoop, Hive, MapReduce
  • New Data patterns – No-SQL
  • Cloud computing – A must for elastic computing vs BYO data centres
  • Open data systems skills – unless you plan to pay for expensive database licenses.
  • Web Services – to tie it all together
  • Agile Architecture – often under-rated, but is increasingly important to focus corporate development.
  • Agile Security – also under-rated, but is increasingly important.

With corporations already struggling to manage data growth and demand, will this mean a growth of x15 in data staffing, or will a data specialist have to be x15 times more productive. I believe its a combination of both. New tools will make the data professional more effective. At the same time because of the lack of training and skills transfer, there will always be a need for the human bridge.

 

 

The future is indeed exciting.