Like many of you, the start of another calendar year made me think about what I accomplished in 2016 and what my goals are for 2017. Overall 2016 was a good year. I worked on some interesting projects, and was able to spend some time working on the business too. I’m going use this post to share 4 key observations around data analytics and business intelligence as it relates to the data integration projects I managed last year.

1. Projects were cross data more than any time before – It used to be that a data integration project was very specific and limited in scope to single sets of data. Sometimes this happened as a result of trying to solve a very specific problem, or the specific team paying for the implementation. But this year, all my data integration projects were done at a higher level covering multiple sources. People and businesses are leveraging different data points/sources more than they ever have before.
2. People want self service tools to cover all scenarios – Traditionally, organizations had specific roles or departments that handled data analytics. A big reason for this was the level of expertise required to mine data (databases, programming languages, etc). The increase and implication of self services business intelligence tools have enabled many more people to participate. Unfortunately there is still a level of expertise required to master these tools. We are starting to see the impact of this with users believing the single tool or skill they invested in will solve all their data analytic questions. But that’s not the case. Using the wrong tool for the job, or trying to get a single tool to cover all scenarios often results in frustration all the way around.
3. There’s a lot still to learn about data quality – In every data integration project I have managed, there has been an epiphany moment with the customer where they realize the data isn’t as clean as they thought it was. This might be as simple as have gaps in data where you thought it existed, but it can also extend to data mistakes, duplication, missing relationships, etc. Nobody wants to hear that there are issues with the data having been used for years. However, projects where the stakeholders have an open mind and treat the project as an opportunity to remedy some of these issues are often more successful. Vendors and project teams need to work closely with the customer to ensure proper documentation and root causes are identified to the best of our abilities.
4. Flexibility is key – We are still working in times of very tight purse strings, but needing to move very quickly to respond to current and future market signals. For businesses to succeed, the organization needs to be working at optimal performance and be able to flex with the client needs around product, services, payments, etc.

What were your key take-aways from your projects in 2016? Without reviewing where we came from in our projects and operations, how can make the next initiatives more successful than the last?