Originally posted to LinkedIn April 29 2022

The right job for the tool, or something like that…

Why the data tool that isn't right for you is still worth learning more about

My role at Kinesso is focused towards highly scalable and repeatable data processes. We don’t run processes for one account, we run it for hundreds or thousands. It is a global company, we’ve got a lot of clients, you get the idea. It requires a certain way of thinking when evaluating tools. In the past I’ve been quick to discard those which aren’t fit for our purpose as being not fit for any purpose. Whilst this is much easier for me, it has been limiting in actually learning new things about different tools and technologies around the place.

Lately I’ve been trying to approach things in a different way. I’ll still cross things off as not being useful to me today - of course I’m not going to start using something not fit for our purpose just to give the team a challenge – but I’ll spend some time trying to work out what situation a tool would work for. Would it work for a small retail business to help them understand their week to week without adding to the workload? Would it benefit a medium sized org with maybe 1 or 2 data people? Is it something an enthusiastic learner could pick up, or does it require dedicated data engineering? Is it a low maintenance solution that a consultant could set up and then the business can just get value out of with little or no maintenance going forward? Okay, the last one is never going to happen.

For data visualisation I generally prefer Tableau. Would I recommend a small/medium sized business trying to improve their finance reports sets up a Tableau Server? Probably not. They likely already have Office 365 and staff that are comfortable enough in excel, so Power BI it is. Maybe it is a startup heavy on G Suite? Well, Data Studio or Looker is probably the answer. For a data warehouse, I like Snowflake a lot. I don’t have their logo tattooed on my chest, but if you were to ask me what database you should use my response is a bit like the WG Grace quote about winning the toss in cricket:

“If you win the toss, bat. If you are in doubt, think about it, then bat. If you have very big doubts, consult a colleague, then bat”

If you want to set up a database, I think you should probably go with Snowflake, although cricket captains sometimes do win the toss and bowl – maybe your org is entirely on prem, Snowflake isn’t going to help you there. Although I would probably suggest you start moving to the cloud and quit stressing about whether the air conditioner in your data center is going to fail over summer.