Freely subscribe to our NEWSLETTER

We all know the challenge of retaining and maintaining the vast volumes of data required for business and regulatory requirements, is one that faces every modern IT organisation. Increasingly, this puts pressure on IT budgets and data centre infrastructure and is compounded by rising operational and energy costs.

So in the current economic and environmental climate, should we really be buying and consuming more storage? Somewhat controversially, but perhaps timely, I will discuss a way to buy less storage - or do more with your existing infrastructure.

By way of example, let’s consider a 100 Terabytes (TB) of structured data with varying requirements such as complex analysis, long term data retention and archiving. These may be new requirements or extra load on an existing data warehouse. Either way, you’re faced with a number of questions and decisions and you will no doubt be bombarded by vendors and consultants offering best advice - or their version of best advice – which isn’t usually cheap! Either way you’re looking at an expensive system with some expensive consulting. Or are you?

Very few people tend to look at these requirements from the perspective of the data or consider a data-centric architecture, as we are so used to deploying application- centric architectures. In simplistic terms you’re looking at a large pool of data. And you’re not entirely sure what it’s going to be used for except that you have to store, manage and retrieve it. In essence, this large pool of data is a corporate information memory – a primary repository of enterprise data which will enable your company to face the challenges mentioned earlier.

There are a number of approaches to deploying a corporate information memory architecture but in essence it comprises a mix of software, solutions and techniques which, when combined, enables companies to deliver significant commercial, competitive and environmental advantage.

By way of example, by combining deduplication, compression and indexing techniques, this 100 TB of data can be held in under 2 TB. That’s right - 2 TB – or a 98% reduction. Not only that, but it is optimised for fast querying and retrieval, requiring little administration and can work with Oracle, SQL Server, DB2, SAS, Business Objects and so on.

To demonstrate the benefits of this approach, consider an organisation that retains and analyses huge volumes of retail data in an ever changing, very demanding and thin margin industry. They are constantly striving to introduce operational improvements across the business. As a result of deploying a corporate information memory architecture, they reduced their storage infrastructure for their analysts by a factor of thirty which resulted in:

• Lower administration costs enabling IT staff to creatively support the business instead of constantly fire fighting
• Improved operational efficiency to over deliver on service levels for operations and querying
• Increased productivity of their end user analysts by a factor of three

And it fits the current green agenda as well.

For years the IT industry has been obsessed with size – who has the largest data centres, largest data warehouses, largest severs and so on. Isn’t it about time we got clever and applied innovative solutions to problems and brag out who has the smallest data centre footprint and shortest batch windows?

By the way - what keeps me awake at night? It’s the ducks that live at the end of my garden performing duck karaoke well into the night.