Big Data – The New Natural Resource


Answering a Big Data Question: Providing 360° View of Customer Behavior

Please join Datameer and IBM for a Webinar on Big Data. 


Date: March 15, 2012 | 10am PT/1pm ET

Sign up on the Link below:


And the Award goes to…

The IBM Big Data Ecosystem of Business Partners! 

But wait, before I get ahead of myself and start fielding acceptance speeches from our Business Partners, I wanted to provide some backdrop. 

We started our Big Data journey about a year ago in the sense of defining to the world how we viewed Big Data.  Extracting insight from an immense volume, variety and velocity of data, in context, beyond what was previously possible. In addition, we released our first version of InfoSphere BigInsights, our Hadoop based product. It joined our existing streaming analytics product InfoSphere Streams. Today our Big Data Platform encompasses Streaming Computing, Hadoop, Information Integration, and Data Warehousing.

As the eco-system got built out in 2011, we finished the year as an industry leader in terms of the number of partners in our rich eco-system for Big Data. Although, I would like to cover each and every business partner in detail here, we do have a microsite that provides a more indepth look (see the link at the end of this post).  I will however profile a couple of partners to give you a flavor of the quality of our ecosystem. 

Revolution Analytics

Revolution Analytics delivers advanced analytics software by building on open source R— very powerful statistics software—with innovations in big data analysis, integration and user experience, Revolution Analytics meets the demands and requirements of modern data-driven businesses.  Revolution Analytics not only supports InfoSphere BigInsights, but Netezza as well.


Datameer utilizes and runs on IBM’s Hadoop based platform (InfoSphere BigInsights), which provides a dependable, enterprise ready implementation of Apache Hadoop. Datameer is leveraged in leading enterprises in the financial services, telecommunications, internet security, gaming and retail industries as well as government agencies to analyze structured and unstructured data. 

Datameer enables users to rapidly integrate, analyze and visualize massive amounts of data.  Datameer users take advantage of wizard-based data integration to instantly access and load virtually all data sources into Hadoop without the requirement of building time-consuming data models. Once complete, the results and insights are then visualized in reports, charts, maps and dashboards in all manner of use cases including customer behavior analytics, fraud detection, web analytics, IT infrastructure analysis and text analytics.  


Combined with IBM’s Big Data Platform, ClickFox’s cross-channel analytic engine will analyze structured and unstructured data from warehouses to build a visual mapping of product and customer experiences across the enterprise. The experience analytics solution will yield powerful insight into consumer behavior, channel conflict, marketing and sales effectiveness, retention strategies and the impact of product and service issues on customers.

By revealing hidden bottom-line connections between products, touch points, consumers and business outcomes, organizations can reduce costs, increase satisfaction and loyalty and drive revenue and competitive advantage. Organizations benefit from a multi-dimensional view of their business from several lenses, including: employee, customer, product, service, company and influence, and can focus analysis on the most critical areas.


TerraEchos builds ‘big data-in-motion’ analytic solutions. Employing IBM’s InfoSphere Streams, their systems handle very large Volumes of a wide Variety of data at high Velocity (V3), enabling clients to aggregate and analyze data from a variety of sources as it streams in real time. Applicable areas include:

  •      Protection of sensitive infrastructure (oil & gas, power grids, etc.)
  •      Monitoring and control of complex systems (power grids, manufacturing, medical)
  • Traffic flow control, including disaster evacuation route optimization, and air traffic control
  • Real-time predictive disaster avoidance (tsunamis and extreme weather)
  • Large-scale environmental monitoring and proactive eco-system protection (air and water temperature, flow and contaminant analysis)
  • Vetting and tracking of known criminals
  • Near-zero latency decision and trade execution on capital markets
  • Scalable architecture against volumes of unstructured or structured data
  • Computationally intensive calculations at unprecedented velocity
  • Dynamic correlation and fusion of a variety of data


Atigeo delivers an experience that benefits communication service providers, media content owners, healthcare organizations and others. With IBM Streams, information can be easily aggregated and correlated across large volumes of data flowing from all aspects of life. And by leveraging IBM Languageware to derive semantic understanding and Cognos Now! for real-time Business Intelligence and Analytics, and Atigeo xPatterns are able to gain useful insights from this vast amount of information that can then be used to create personal and portable profiles, while always preserving the highest degree of privacy.

Persistent Systems

Persistent Systems offers solutions based on IBM technologies. Persistent’s proven BigInsights implementation methodology for Big Data solutions leverages –

  • In-depth knowledge of BigInsights product and underlying Hadoop technologies
  • Specialized tools and frameworks that significantly reduce time to value and also provide ease of maintenance
  • A large Analytics practice that has been involved in all aspects of BigInsights product, all the way from installation, implementation, integration, quality assurance

The Persistent team also has deep knowledge of different components of the Hadoop ecosystem such as Hive, JAQL, PIG and other open source platforms where we have contributed IP and actively support these systems. See today’s announcement: 


Concord is an established IBM Premiere Business Partner with a proven track record delivering industry solutions based on IBM’s Information Management, and WebSphere product lines as well as Hadoop.

In addition, Concord has created ComplETE suite that complements and enhances the BigInsights platform by providing end-to-end business process visibility in mainframe & distributed environments as well as environments where establishing precise transaction relationships seems impossible. They offer true end-to-end correlation. The suite includes transaction monitoring, transaction trending, transaction analytics, event management and payload forensics. Concordwill also leverage IBM InfoSphere Streams to handle streaming analytics.

The suite couples the power of Hadoop with in-memory MOLAP cubes embedded in our RETE rules engine to deliver the fastest real-time analytics & simulation platform on the market.

For more details and information on IBM big data business partners please visit:

In the end, it is you, the customer who will decide who gets the Award for being the best IBM big data Business Partner.  In the mean time I want to make sure you have many high quality ones to choose from…  – Bruce Weed

“The secret of success is constancy of purpose.” – Benjamin Disraeli

IBM Big Data Success Stories – Client References and Big Data Business Partners

Click on the link below to access the Success Stories (PDF)


“Getting to the top is optional. Getting down, mandatory” – Ed Viesturs, Mountain Climber

See you at the IBM Information OnDemand (IOD) Conference in Las Vegas this week – And yes, Big Data will be a hot topic!

It is that time of year again; our IBM IOD conference is upon us.  I hope to see many of you there in attendance.  With over 10,000 people attending and some great sessions planned, it should be a valuable experience for everyone (Business Partners, Customers and IBMers).    

What is my predicted buzz at the conference this year?  What else, Big Data.  I mean what else could be bigger then Big Data, since it already has Big in its name. 

I will be kicking off the Big Data sessions with my talk on Sunday for Business Partners, from 2:15pm to 3:15pm.  Below I have included other key Big Data sessions you may want to attend while you are at IOD.

Big Data Sessions at IOD 

I would encourage Business Partners and Customers to raise your hands and ask questions about our Big Data Platform and strategy.  Talk to one another and get views on this exciting area. 

Safe travels and see you in Las Vegas!

BTW it is never too late to attend the conference it runs from October 23 – 27.  See the link below for more info.

“I am easily satisfied with the very best.” – Winston Churchill

IBM big data Business Partners

Many of the Business Partners you see here listed below will be out at our Information On Demand (IOD) Conference in Las Vegas starting on Oct. 23 – Oct. 27.

Please take the time to meet with them if you are out there.  Watch for my next post for more details on the conference.



“It is a mistake to look too far ahead. Only one link in the chain of destiny can be handled at a time.” – Winston Churchill


Published in: on September 30, 2011 at 6:55 pm  Leave a Comment  
Tags: , , ,

Learn Hadoop and Go to Las Vegas All Expenses Paid!

IBM is sponsoring 3 Big Data enthusiasts for an expenses paid trip to Las Vegas to attend the Information on Demand conference taking place during October 24-28, 2011. To qualify all you need to do is complete free Hadoop Fundamentals course online at Big Data University  and fill out the course evaluation form. Who knew learning Big Data could be so rewarding.

Learn Hadoop for Free and Go to Las Vegas All Expenses Paid:

“You give but little when you give of your possessions. It is when you give of yourself that you truly give.” – Kahlil Gibran
Published in: on September 14, 2011 at 4:27 pm  Leave a Comment  
Tags: , , ,

1 to 40 Trillion, those are large numbers!

Somewhere over a trillion gallons of water fell from Hurricane Irene.  I even heard numbers quoted as large as 40 trillion. So, how are we doing in the aftermath of Irene, and how did we do leading up to the storm?  Let’s start with SCM (supply chain management), since it was non-existent.  I went looking for D-cell batteries one night several days before the storm, none in stock. 

Two weeks before the storm hit, they knew it would hit somewhere on the east coast.  Battery companies at that point should have been sending stock to mid-eastern seaboard warehouses.  A week before the storm those batteries should have been shipped to retailers from NC to Maine. 

I would like to say we need a Big Data solution to solve this issue, but we really do not.  We need improved SCM and common sense.  It is very easy to figure out what people need before and after a storm, here is the short list:  Batteries (D-cell in particular), bottled water, first aid kits, generators and chain saws.   Secondary items: ice, MRE’s, sump pumps, wet vacs, inflatable rubber rafts, etc. 

I was speaking with Billy Bosworth the CEO of DataStax yesterday and we both agreed companies need to study the military to really understand provisioning/SCM.  Having lived through Hurricane Andrew back in 1992, where wind speed was recorded at 220 miles per hour at Turkey Point, I was able to see what the military can do and how fast.  Tent cities set up over night, roads cleared in a blink of an eye, looting stopped on a dime (army rangers with M-16s never had to say a word), food and water distributed to those in need.   

With the right IT solutions, common sense, and help from the National Guard/Army we should be able to do a much better job pre/post hurricanes in the future. 

Now, where does Big Data come in?  The storm itself, although we had good warning and it was tracked fairly well, there were two areas that fell short.  First, the areas they thought would get the most damage did not.  Many of the coastal areas had minimal damage, ditto with places like Manhattan (NYC). However many places in land got hammered,Vermont, rural NY, Connecticut, etc.  

The second area they missed was the wind tail of the storm.  The storm was supposedly gone from my area (CT) by late afternoon; however we had high winds on Sunday from 5pm to 11pm, even though the storm was “gone”.  These winds cause a lot of extra damage.  Big Data can help in predictive modeling, with more data points analyzed; maybe those two misses above would not have happened.  

In the mean time I am in the dark both at home and figuratively. Still no power at the house and still wondering why we can not handle these hurricanes in a more effective manner.   

“Everybody lives for something better to come.” – Anonymous