Tag Archives: data analytics

2016 Retrospective

by

Like many of you, the start of another calendar year made me think about what I accomplished in 2016 and what my goals are for 2017. Overall 2016 was a good year. I worked on some interesting projects, and was able to spend some time working on the business too. I’m going use this post to share 4 key observations around data analytics and business intelligence as it relates to the data integration projects I managed last year.

  1. Projects were cross data more than any time before – It used to be that a data integration project was very specific and limited in scope to single sets of data. Sometimes this happened as a result of trying to solve a very specific problem, or the specific team paying for the implementation. But this year, all my data integration projects were done at a higher level covering multiple sources. People and businesses are leveraging different data points/sources more than they ever have before.
  2. People want self service tools to cover all scenarios – Traditionally, organizations had specific roles or departments that handled data analytics. A big reason for this was the level of expertise required to mine data (databases, programming languages, etc). The increase and implication of self services business intelligence tools have enabled many more people to participate. Unfortunately there is still a level of expertise required to master these tools. We are starting to see the impact of this with users believing the single tool or skill they invested in will solve all their data analytic questions. But that’s not the case. Using the wrong tool for the job, or trying to get a single tool to cover all scenarios often results in frustration all the way around.
  3. There’s a lot still to learn about data quality – In every data integration project I have managed, there has been an epiphany moment with the customer where they realize the data isn’t as clean as they thought it was. This might be as simple as have gaps in data where you thought it existed, but it can also extend to data mistakes, duplication, missing relationships, etc. Nobody wants to hear that there are issues with the data having been used for years. However, projects where the stakeholders have an open mind and treat the project as an opportunity to remedy some of these issues are often more successful. Vendors and project teams need to work closely with the customer to ensure proper documentation and root causes are identified to the best of our abilities.
  4. Flexibility is key – We are still working in times of very tight purse strings, but needing to move very quickly to respond to current and future market signals. For businesses to succeed, the organization needs to be working at optimal performance and be able to flex with the client needs around product, services, payments, etc.

What were your key take-aways from your projects in 2016? Without reviewing where we came from in our projects and operations, how can make the next initiatives more successful than the last?

Semi-homemade is better than bespoke for data analytics

by

I read a product review this week where the company referred to themselves as a provider of “bespoke” data analytics. I had never heard that term used in the context of data analytics, or software specifically. However, when I googled the term, I found many companies using it in their marketing language, but no reference to it by the people who write about data analytics or software. This led me to start thinking about my experiences managing data integration software projects and how my customers view the solutions.

The projects that I’ve worked on in the last couple of years have primarily been data integration projects where we are combining multiple datasources into a single data warehouse and then leveraging that data to deliver data insights. The platform has some standard integration components that you can leverage, but there is also room for quite a bit of custom development. In every implementation, I have had conversations about what “standard” tools are available and what capabilities can be developed custom. On one hand, once these customers start reviewing the available tools, the first questions asked are usually about how we can customize those tools to their business. Each customer self-identifies as a unique even though most are within the same overall industry. There are always unique scenarios for each customer that needs to be accounted for.

bespoke-suit-pattern

http://www.giandecaro.com/img/background-bespoke.jpg

On the other hand, customization takes time and effort, regardless of whether the work is done in house or by external consultants. Where does that leave us if our customers want/need something specific to their business but don’t want or can’t invest the time and money to do so?

I think as integration partners, we are probably looking at the entire product management and implementation process incorrectly. Our customers need a balance of standard tools that they can quickly customize to their specific needs along with partners who will work with them to develop custom solutions for new or innovative work. This is similar to the idea of leveraging a template to develop your website, but then be able to customize your experience by changing colors or adding widgets that extend the template capabilities. We can think of these types of products as “semi-homemade.”

Semi-homemade is a term used heavily by Sandra Lee regarding her cooking style. She leverages pantry staples and other ingredients and creates amazing dishes. By not having everything made from scratch, Sandra Lee reduces the cooking & prep time but is still able to deliver tasty dishes people want to eat. If we apply the same principles to data analytics, I think we can definitely leverage some basic tools that we allow people to extend or meld, which result in delivering data insights without the pain of everything being a custom solution.

It’s time to shift our mindset away from solely developing out of the box solutions, or solely developing custom solutions. Product and services should be working together to build base tools that are easily extended to meet the changing needs of our customers. We won’t totally eliminate the need for custom solutions, or new products for that matter. But we will more quickly be able to meet the changing needs of our customers.