Thank you to Fabien of Power Data Group for endorsing the ideas and vocalising the need of managed data in an increasing connect world with enterprises hungry for insights. You can have all the tools, and systems, but it takes careful planning, consultants and data to get insights and reports flowing.
Glitchdata welcomes Power Data Group as a distributor of our products.
Hi! It’s been a busy last couple of weeks, with trips to asia, a slew of conversation with friends and honing of concepts around Glitchdata. However, it is done and I am pleased to present “Glitchdata“.
Until now, Glitchdata has been a community of data architects focused on data integration. We all breath data, and over the course of work it is becoming increasingly apparent that the main problem faced by the data industry is data itself. As such, we are now focused on master data management and the distribution of managed datasets.
I recently ordered a copy of the book “Directing the Agile Organisation”. Since I know the author, I am obliged (disclosure) to say something brilliant about the concept and the book. Unfortunately, I haven’t read the book yet (its in the mail), but since I have a little time this morning, I’d thought I plug the book with some Agile commentary. Agile methods are evolutionary. In the scale of doing things faster, and in a flexible way, Agile breaks the rigidity of Waterfall, PRINCE2. Agile methods are well proven in the open source community, game development, and product development where the parameters of a project like cost, time, features, and quality tend to be established.
However in most enterprises, project parameters like cost, time, features and quality are fluid. And that is where the Agile project horror stories begin. Business executives who hear about the brilliance of Agile unwittingly adopt it for an environment where it will fail. Traditional project controls like project costing, feature management, stakeholder management etc… are not addressed adequately. So the Agile engine splutters. If we are able to address the business management of a project in an agile way, Agile methods will find a more relevant place at the core of the enterprise.
I know the author has plenty of experience and insight in transitioning Agile into the enterprise, and this is what I am keenly looking forward when I read this book. 🙂
You can purchase this book at: http://www.itgovernance.asia/p-958-directing-the-agile-organisation-a-lean-approach-to-business-management.aspxDISCOUNT: To get the 10% discount, enter ‘agile10’ (without the apostrophes) into the discount code box during the checkout process. This special will run until the 15th of September. About the Book
What is ‘Directing the Agile Organisation’ about ? Business systems do not always end up the way that we first plan them. Requirements can change to accommodate a new strategy, a new target or a new competitor. In these circumstances, conventional business management methods often struggle and a different approach is required.
Agile business management is a series of concepts and processes for the day-to-day management of an organisation. As an Agile manager, you need to understand, embody and encourage these concepts. By embracing and shaping change within your organisation you can take advantage of new opportunities and outperform your competition.
Using a combination of first-hand research and In-depth case studies, Directing the Agile organisation offers a fresh approach to business management, applying Agile processes pioneered In the IT and manufacturing industries. Agile business management is divided into four domains, which each changes the way your business operates.
In data analytics, alot of decisions frequently need to be made. These can be data issues, business issues, technical issues, to people issues. In larger data projects, sizable data team performing multiple functions need to be managed. This is where delegation becomes important. Michael Hyatt discusses the 5 levels of delegation that a manager would encounter.Level 1: Do exactly what I have asked you to do. Don’t deviate from my instructions. I have already researched the options and determined what I want you to do. Level 2: Research the topic and report back. We will discuss it, and then I will make the decision and tell you what I want you to do. Level 3: Research the topic, outline the options, and make a recommendation. Give me the pros and cons of each option, but tell me what you think we should do. If I agree with your decision, I will authorize you to move forward. Level 4: Make a decision and then tell me what you did. I trust you to do the research, make the best decision you can, and then keep me in the loop. I don’t want to be surprised by someone else. Level 5: Make whatever decision you think is best. No need to report back. I trust you completely. I know you will follow through. You have my full support.
Gartner is hosting the Business Intelligence & Information Management (BIIM) Summit. A must attend event for BI/IM folks who are looking into the evolution of BI/IM to provide value to their organistions. What is interesting are topics like “the last mile for BI” and the move to “predictive and prescriptive” data.
Two thoughts come to mind when I look at the details of this conference.For alot of organisations, they are very very far from “the last mile”. Without methodical and data-centric approach to BI, many organisations are stuck in no-man’s land. The chaos of semi-defined masterdata, a plethora of link tables/translation tables, poor definition of transactional and analytical datasets, half-fuel performance at all levels of the BI stack, and probably significantly under-performing BI. Naturally business owners want to get value from their BI/IM investments. Traditionally, this has been considered an important and justifiable cost to the organisation especially for decision making, but its increasingly important to extend that capability into future scenarios. Future scenarios involve the unknown. Unknown datasets, unknown parameters, unknown visualisations, unknown BI/IM capability. These are the topics that I would like to see from this conference.
Anyway, I’ll be attending the conference. Email me if you want to meet.
Conference TracksTrends and Futures Information Innovation Performance Management Social and Big Data Virtual Tracks
Informatica (INFA) issues warning on weakness in Q3 results, reporting guidance of USD189+M, a drop from USD200+M (5% drop). The stock market has further punished the company with stock prices dropping from USD45 to USD 30 (33% drop) over the last 6 months.
Informatica blames the weakness on Europe, but could it be that the value of its core business of “data transfer” is being eroded by open source equivalents like Talend. This is definitely a lower cost alternative for open source oriented European companies.
So what is great about Git? It’s flexibility, and ability to manage code in any code-management workflow that you can think of. Some examples are:Local development (for Individuals) Hub-Spoke (for Teams) MegaHub-Hub-Spoke ( for multiple Teams) Spoke-Spoke (for peer-to-peer development) Spoke-Spoke-Hub-Spoke-Spoke (for peer-to-peer and Teams) and the combinations go on…
So what unique concepts drives the unique distributed development capability of Git? These are:Efficient key-value file storage (http://git-scm.com/book/en/Git-Internals-Git-Objects) Efficient and precise history / log tracking Strong performance even on large repositories
But of the key strengths of Git is the adoption by the linux community. Git is the brainchild of Linus Tovalds (the creator of Linux).
SAP re-launches the <a href=”http://www.sap.com/solutions/technology/in-memory-computing-platform/hana/overview/index.epx”>HANA</a> (<strong>H</strong>igh-performance <strong>AN</strong>alytic <strong>A</strong>ppliance) platform in 2012 and looks to this as the “game changing” technology for BI/DW/analytics. But is it?
Driven by the corporate demand for real time analytics, the HANA platform seeks to put data into memory and dramatically improve performance. This will help address the demand for big data, predictive capabilities, and text-mining capabilities.
But doesn’t this sounds like the typical rhetoric from computing vendors that previously addressed technology issues by recommending the addition of more CPU, or RAM, or disk space. SAP HANA is delivered as a software appliance focused on the underlying infrastructure for SAP Business Objects. This <a href=”http://download.sap.com/download.epd?context=B576F8D167129B337CD171865DFF8973EBDC14E3C34A18AF1CF17ED596163658ABE46C2191175A1415B54F1837F5F0A13487B903339C6F98″>white paper</a> suggests alot of scoping is centred around hardware and infrastructure design.
HANA makes incredulous claims that traditional BI/DW folks would falter to whisper. The one that stands out is the “Combination of OLAP and OLTP” into the one database. Ouch! Feel the wrath of the stakeholders of business operations. Another claim is running analytics in “mixed operations”. Double ouch!
It’s already challenging enough to get DW/BI solutions deployed without affecting operations. BI folks have constantly advocated separate infrastructure for analytics, with the ETL window as the firewall between systems. The same ETL window has also created delays for realtime analytics. To advocate moving the BI/DW infrastructure back into operations is going to be a challenge. Yes, it facilitates “closer to real-time”, but its going to be a challenge to make it work politically.
For other BI/DW vendors, this solution would be unfeasible, but because SAP also happens to the largest ERP application platform on the planet, they definitely have a good shot at consolidating their ERP and HANA’s BI analytics. Google, Facebook and the large online behemoths already do it. So why not?!
This is indeed exciting, and its definitely time to take a closer look at SAP HANA.
If you thought “Big Data” was already quite unmanageable, IEEE predicts a 1500% (x15) growth in data by 2015. That is 3 years from now.
On a similar scale, IEEE also suggests that terabit networks should be implemented soon to cater for demand in network traffic by 2015. This is up by x40-1000 times from today’s gigabit networks.
This probably also suggests that demand for data processing and delivery will need to increase by a similar scale. To some 10-40 times.
What products and skills will power the delivery of services for “Humungous Data”?New Data systems – like GFS, BigTables, Hadoop, Hive, MapReduce New Data patterns – No-SQL Cloud computing – A must for elastic computing vs BYO data centres Open data systems skills – unless you plan to pay for expensive database licenses. Web Services – to tie it all together Agile Architecture – often under-rated, but is increasingly important to focus corporate development. Agile Security – also under-rated, but is increasingly important.
With corporations already struggling to manage data growth and demand, will this mean a growth of x15 in data staffing, or will a data specialist have to be x15 times more productive. I believe its a combination of both. New tools will make the data professional more effective. At the same time because of the lack of training and skills transfer, there will always be a need for the human bridge.
The future is indeed exciting.
Kudos to Brittany Wenger from Lakewood Ranch, USA for winning Google’s Science Fair Grand prize.Using a 6-node Artificial Neural Network (see her slides), and alot of cloud computing power, Brittany has managed to train the neural network to detect maligned breast tumors with an accuracy of 99.11%
Now, what is notable is that this girl is 17 years old. I was talking to some parents recently about how the amount of new knowledge being generated today is in the exponential scale. What this means is that they next generation of kids will have to learn more and in less time. Now, I am sure neural networks have been implemented by geniuses far younger than 17 years.
The comparison I would like to make here is that I learnt neural networks at age 20 (and with minimal successful commercial application), and as Britanny has a successful implementation of a neural network at age 17, I would now say that:My kids will probably be implementing neural networks at age 14-15 Artificial intelligence is going to be more commonplace in the future.