In data analytics, alot of decisions frequently need to be made. These can be data issues, business issues, technical issues, to people issues. In larger data projects, sizable data team performing multiple functions need to be managed. This is where delegation becomes important. Michael Hyatt discusses the 5 levels of delegation that a manager would encounter.

Level 1: Do exactly what I have asked you to do. Don’t deviate from my instructions. I have already researched the options and determined what I want you to do. Level 2: Research the topic and report back. We will discuss it, and then I will make the decision and tell you what I want you to do. Level 3: Research the topic, outline the options, and make a recommendation. Give me the pros and cons of each option, but tell me what you think we should do. If I agree with your decision, I will authorize you to move forward. Level 4: Make a decision and then tell me what you did. I trust you to do the research, make the best decision you can, and then keep me in the loop. I don’t want to be surprised by someone else. Level 5: Make whatever decision you think is best. No need to report back. I trust you completely. I know you will follow through. You have my full support.

Gartner is hosting the Business Intelligence & Information Management (BIIM) Summit. A must attend event for BI/IM folks who are looking into the evolution of BI/IM to provide value to their organistions. What is interesting are topics like “the last mile for BI” and the move to “predictive and prescriptive” data.

Two thoughts come to mind when I look at the details of this conference.

For alot of organisations, they are very very far from “the last mile”. Without methodical and data-centric approach to BI, many organisations are stuck in no-man’s land. The chaos of semi-defined masterdata, a plethora of link tables/translation tables, poor definition of transactional and analytical datasets, half-fuel performance at all levels of the BI stack, and probably significantly under-performing BI. Naturally business owners want to get value from their BI/IM investments. Traditionally, this has been considered an important and justifiable cost to the organisation especially for decision making, but its increasingly important to extend that capability into future scenarios. Future scenarios involve the unknown. Unknown datasets, unknown parameters, unknown visualisations, unknown BI/IM capability. These are the topics that I would like to see from this conference.

Anyway, I’ll be attending the conference. Email me if you want to meet.

Conference Tracks

Trends and Futures Information Innovation Performance Management Social and Big Data Virtual Tracks

Informatica (INFA) issues warning on weakness in Q3 results, reporting guidance of USD189+M, a drop from USD200+M (5% drop). The stock market has further punished the company with stock prices dropping from USD45 to USD 30 (33% drop) over the last 6 months.

Informatica blames the weakness on Europe, but could it be that the value of its core business of “data transfer” is being eroded by open source equivalents like Talend. This is definitely a lower cost alternative for open source oriented European companies.

The future of Git is bright. It displaces probably every Code-Versioning System (CVS) system out there, and its open sourced (GPL2)

So what is being displaced by Git? Mercurial svn cvs

So what is great about Git? It’s flexibility, and ability to manage code in any code-management workflow that you can think of. Some examples are:

Local development (for Individuals) Hub-Spoke (for Teams) MegaHub-Hub-Spoke ( for multiple Teams) Spoke-Spoke (for peer-to-peer development) Spoke-Spoke-Hub-Spoke-Spoke (for peer-to-peer and Teams) and the combinations go on…

So what unique concepts drives the unique distributed development capability of Git? These are:

Efficient key-value file storage ( Efficient and precise history / log tracking Strong performance even on large repositories

But of the key strengths of Git is the adoption by the linux community. Git is the brainchild of Linus Tovalds (the creator of Linux).

SAP re-launches the <a href=””>HANA</a> (<strong>H</strong>igh-performance <strong>AN</strong>alytic <strong>A</strong>ppliance) platform in 2012 and looks to this as the “game changing” technology for BI/DW/analytics. But is it?

Driven by the corporate demand for real time analytics, the HANA platform seeks to put data into memory and dramatically improve performance. This will help address the demand for big data, predictive capabilities, and text-mining capabilities.

But doesn’t this sounds like the typical rhetoric from computing vendors that previously addressed technology issues by recommending the addition of more CPU, or RAM, or disk space. SAP HANA is delivered as a software appliance focused on the underlying infrastructure for SAP Business Objects. This <a href=”″>white paper</a> suggests alot of scoping is centred around hardware and infrastructure design.

HANA makes incredulous claims that traditional BI/DW folks would falter to whisper. The one that stands out is the “Combination of OLAP and OLTP” into the one database. Ouch! Feel the wrath of the stakeholders of business operations. Another claim is running analytics in “mixed operations”. Double ouch!

It’s already challenging enough to get DW/BI solutions deployed without affecting operations. BI folks have constantly advocated separate infrastructure for analytics, with the ETL window  as the firewall between systems. The same ETL window has also created delays for realtime analytics. To advocate moving the BI/DW infrastructure back into operations is going to be a challenge. Yes, it facilitates “closer to real-time”, but its going to be a challenge to make it work politically.

For other BI/DW vendors, this solution would be unfeasible, but because SAP also happens to the largest ERP application platform on the planet, they definitely have a good shot at consolidating their ERP and HANA’s BI analytics. Google, Facebook and the large online behemoths already do it. So why not?!

This is indeed exciting, and its definitely time to take a closer look at SAP HANA.



If you thought “Big Data” was already quite unmanageable, IEEE predicts a 1500% (x15) growth in data by 2015. That is 3 years from now.

On a similar scale, IEEE also suggests that terabit networks should be implemented soon to cater for demand in network traffic by 2015. This is up by x40-1000 times from today’s gigabit networks.

This probably also suggests that demand for data processing and delivery will need to increase by a similar scale. To some 10-40 times.

What products and skills will power the delivery of services for “Humungous Data”?

New Data systems – like GFS, BigTables, Hadoop, Hive, MapReduce New Data patterns – No-SQL Cloud computing – A must for elastic computing vs BYO data centres Open data systems skills – unless you plan to pay for expensive database licenses. Web Services – to tie it all together Agile Architecture – often under-rated, but is increasingly important to focus corporate development. Agile Security – also under-rated, but is increasingly important.

With corporations already struggling to manage data growth and demand, will this mean a growth of x15 in data staffing, or will a data specialist have to be x15 times more productive. I believe its a combination of both. New tools will make the data professional more effective. At the same time because of the lack of training and skills transfer, there will always be a need for the human bridge.



The future is indeed exciting.

Kudos to Brittany Wenger from Lakewood Ranch, USA for winning Google’s Science Fair Grand prize.Using a 6-node Artificial Neural Network (see her slides), and alot of cloud computing power, Brittany has managed to train the neural network to detect maligned breast tumors with an accuracy of 99.11%

Now, what is notable is that this girl is 17 years old. I was talking to some parents recently about how the amount of new knowledge being generated today is in the exponential scale. What this means is that they next generation of kids will have to learn more and in less time. Now, I am sure neural networks have been implemented by geniuses far younger than 17 years.

The comparison I would like to make here is that I learnt neural networks at age 20 (and with minimal successful commercial application), and as Britanny has a successful implementation of a neural network at age 17, I would now say that:

My kids will probably be implementing neural networks at age 14-15 Artificial intelligence is going to be more commonplace in the future.

PHP 5.4

Jun 26, 2012


In:Development, News, Software

Comments Off on PHP 5.4

Here are some things you need to know about PHP 5.4.

1) Trait Support

Improvements to single inheritance for classes.

2) Array Improvements

Array manipulation has improved by introduction of array de-referencing. This removes the need to define temporary variables. Code looks neater. Now you can do:

$food = explode(",", "pizzahut,burgerking,kfc,mcdonalds"); $food[3]; // mcdonalds echo explode(",", "pizzahut,burgerking,kfc,mcdonalds")[3]; //mcdonalds 3) $this support in Closures 4) Build-in CLI Web Server

A command line web-server is included in the php interpreter. This is invoked by: $ cd ~/public_html $ php –S localhost:8000 5) <?= support

The short_open_tag setting in php.ini allows for the use of <?= by default in PHP 5.4. Code looks neater. Example:

Previously we used: <?php echo $variable ?>

Now we can use: <?= $variable ?>


Read more

The Open Web Application Security Project (OWASP) has opened a chapter in Canberra. Kicked off by Andrew Muller of Ionize, OWASP brings to Canberra expertise in web application security. It also brings the small community of security professionals to meet, discuss and engage in the crucial business of securing applications.

OWASP Canberra is committed to monthly meetings, and the occasional “special” meeting. See you there!

OWASP has a project called ‘The OWASP top ten project‘ which list the top 10 security threats for web-based applications.

OWASP Current Top Twelve Threats

Cross-Site Scripting (XSS) Malicious File execution Insecure Direct Object References Cross-site Request Forgery (spoofing) Information Leakage and Improper Error Handling (I’m guilty) Injections Flaws Broken authentication and session management Insecure cryptographic storage Transport Layer Protection (TLP) Failure to secure URL access (I’m guilty) Security Misconfiguration Unvalidated Redirects and Forwards

Ok, which ones are you guilty of?

Agile Methods has gone through a roller-coaster ride of adoption. The first thing a team member notices about agile are:

Regular scrums – usually daily Micro-Issue tracking Measurements

There are of course many other aspects to Agile methods and Evan Leybourn @ The Agile Director has alot of experience in implementing Agile methods in software development teams. He has a few courses running in Canberra and Sydney. Check them out:


Sydney – 2-3 April12 – Agile Methods Sydney – 4 April12 – Advance Agile Methods Canberra – 10-11 April12 – Agile Methods Canberra 12 April12 – Advance Agile Methods