Journey into analytics

Introduction

Thanks for stopping by! I am blogging to share my experiences with you on how i got started into analytics and hope it will give those inspire others to reach out and help others.

How did i get into analytics?

4 years ago while working as an senior analyst within a large insurer,  I was tasked with cutting SAS code to answer questions that business partners had. Why is my revenue ahead or behind budget, could it be due to the volume or price or mix of new business?

The trouble with this was that it took considerable effort and days to get answers to simple questions. I thought that there must be a better way so i went online and discovered that there was called Tableau.  For those new to Tableau, it is best described as a pivot table on steroids! Download it and give it a go. You can simply load data into the tool and then it draws fantastic looking graphs almost immediately.

Don’t take my word for it, go download the product and see how it works for yourself!

Having used Tableau with SAS for a few years, I came upon another tool called Alteryx.  One night , I came across an presentation by a BCG consultant at a Alteryx Inspire 2014 conference and it sparked my interest on how i could be using it within my company to help business users get to the answers they need faster.

For those new to Alteryx, It can be best described as a graphical drag and drop workflow that can be used to combine and pharse numerous data sources together in an easy to use way such that anyone can use it!

Alteryx is really a 3 in 1 product supporting data preparation, spatial analytics and predictive analytics and works fantastically well with Tableau.

Don’t try and do everything yourself!  Buy world class products that can make your analytics lives easier!

Adopt a global perspective for analytics

The year I unfortunately couldn’t make it to Inspire 2017 in Las Vegas due to work commitments however I hope this will inspire you all to take analytics to the next level within your respective companies.

In particular, I really would like to dedicate this to all of you who are trying to challenge the status quo within your organisations and share some of the learnings that has worked for me.

1. Research how other customers have done it globally

As they say good artists copy, the best artists steal! Look for the things that have been done by people in the US and feel free to connect up with them and replicate what they have been successful at. Don’t be afraid to ask questions.Most importantly don’t be limited by the country or industry that you operate in. Learn and adopt global best practise from the start.

 Alteryx inspire is a clear example of this. It’s a global community of people who have overcome incredible organisational challenges to come out on tops delivering value to the business. Check out the links below. Feel free to copy and leverage successful deployments here.

https://www.alteryx.com/inspire-2017
2. Solve a business problem and ask for money$

So often I see people getting stuck into the technology but not focusing on the actual problem that the business has. Doing so often doesn’t end well, it becomes a debate of features and merits of product A vs product B. In the case of Alteryx just download the product, get the analytics done and show the insight to senior executives directly!

Once the insight is shown, remember to ask for money to proceed to phase 2.
3. Build solid partnerships with vendors

When choosing a vendor, make sure you have access to senior leaders within the organisation as well as engineering teams and their developers. Help them to help you. Often I see in major corporates, customers keep the vendors at a distance ( is: Classic supplier customer relationship) rather than partnering with them to improve their product offering. Such shortsightedness often means that a customer will not be able to get the best out of the product.
A recent discussion with some of my peers showed they wanted to get major cloud vendors to fit it out and pitch from a cost perspective. Vendors have to reach their revenue targets, it’s often much better to help them achieve this, being open and transparent and in so getting incredible support when you most need it.
4. Join the global community and be resilient

Finally join the global communities like the ones organised by Alteryx, tableau, Microsoft and AWS. Realise that you are part of something much larger and leverage others who have gone before you. Remember to also give back to others along the way.

https://community.alteryx.com

https://community.tableau.com/index.jspa

https://forums.aws.amazon.com/index.jspa

https://techcommunity.microsoft.com
Deploy a Data Warehouse on the AWS Cloud with New Quick Start

The democratisation of infrastructure and advanced analytics

The rise of different technologies such as AWS, Alteryx, Tableau, Databricks, Datarobot have democratised analytics for everyone.

Democratisation is about giving business users the power and capability to do advanced analytics without coding just in the same way as we use Windows instead of DOS.

What this democratisation is really about is people’s battle for relevance, the battle for existing people within companies who don’t wish to embrace change vs those that do. Often it comes from a fear that they are no longer useful, important or relevant to the organisation. 


AWS has democratised infrastructure enabling companies to focus on business outcomes

I particularly like the fact how companies and individuals can now modernise and consolidate to enable innovation for new revenues while improving customers engagement and experience.


Democratising and modernising analytics platforms

Anyone with a little knowledge can easily spin up a AWS EMR cluster and access nearly 100 cores in Spark to do processing without breaking a sweat. That type of processing power is incredible compared to what can be assessable on premise. In addition, the cost to run it is only $4 an hour. It’s almost crazy cheap! Certainly cheaper in a server at home.


One of the most recent advances in AWS is AWS Redshift spectrum. This is a really fast and easy way to query data that is stored in s3 instead of going through the traditional process of loading to a warehouse. This is definately a game changer as it enables users to directly access all their data , not just data stored within the databases be it Redshift or Aurora.

Big data is relative

it’s funny how everyone thinks that their data is ‘big’. After reading a few case studies on AWS like Netflix with 60 Petabytes in S3 and  who makes up nearly 1/3 of all Internet traffic in the US. they also land over 760 billion events into s3 daily. 

The weather company who produces 15 billion predictions a day or NTT Docomo who has over 68million customers. Most data in companies are really small by comparison. It’s really funny when you read about people using redshift for Petabyte scale queries. That’s unreal. Check out their video and  Slide share

A few interesting insights are

  • Data team is relatively large with over 100 staff out of 2000 total company staff
  • S3 is the main data lake where everything gets stored before downstream processing. The real benefit is infinite scaling and you can collect as granular as you want. I was lucky enough to use the technology at Amaysim and it’s fantastic as it enables one to use other database technologies easily in the future as you are not stuck to 1 tech.
  • hive metastore used to store information about files in s3.
  • Presto used for interactive queries
  • Redshift and Teradata used for data warehousing and Tableau for visualisation


Alteryx enables anyone to advanced analytics without coding

Alteryx has revolutionised the analytics market by making it easy for a business user to simply download the product, combine data from tons of data sources and then perform ETL, spatial, predictive and optimisation in 1 hit. 

This type of reach and power is completely unreal as it shift the power from the analyst directly into the business. It also enables anyone to become an analyst and ask and answer questions of their data. Alteryx turns every data worker into the discoverer of marginal profitability! 

To top it off Alteryx server is also available as an AMI on Amazon. just click start free trial.



Tableau the King of visualisation 

The ability to see and understand your data is clearly the mission of Tableau and they take it really seriously. Just look at the number of customer stories and you can tell it’s a successful company. It’s also fantastic that it’s also available on Amazon.

Collaborating across the organisation with the Alteryx


Yesterday I had the opportunity to help my organisation train a cross section of new users to Alteryx using the self serve enablement material. The session took 1.5 hours and had a cross section of the organisation of users in it.

In the session we had

  • Data scientists
  • Managers
  • Tableau specialists
  • SQL specialists
  • End to end BI analysts

What I learned by training others

1. Alteryx democratises data science and facilitates cross functional collaboration

Alteryx training is really about bringing different people within the organisation together leveraging each other’s skills and enabling them to have a common language of how data is acquired, wrangled and visualised. The most important take out for me is that people of any skill level can pick up the use of the tool really quickly which then forms the basis for collaboration across the organisation. 

2. Alteryx has a short learning curve

This is in stark contrast to other BI tools who require extensive training to pick up and get value from.The most important thing is that through the use of Alteryx, development times to get insight for business leaders is drastically reduced to hours instead of days.

3 Speed to insight matters

This in turn becomes a competitive advantage for the organisation.

I hope this blog inspires others within big corporations to take the first step and try it out.

Download the product here and give it a go

http://downloads.alteryx.com

Best of spatial analytics for alteryx

Intro to spatial

http://www.thedataschool.co.uk/ian-baldwin/spatial-data-alteryx/

https://community.alteryx.com/t5/Engine-Works-Blog/Spatial-Analytics-the-final-frontier/ba-p/17703

 

Polybuild

http://inviso.dk/2015/how-to-make-custom-tableau-maps

https://community.alteryx.com/t5/tkb/articleprintpage/tkb-id/knowledgebase/article-id/1251

 

Make grid vs Heatmap

http://www.thedataschool.co.uk/ben-davis/mapping-big-data-using-tableau-alteryx-part-1-heat-maps/

https://benjnmoss.wordpress.com/2016/04/12/reviz-city-of-crime/

 

Generalise polygon

Using Shape Files for Boundaries in Tableau

 

Spatial functions using summarize tool

https://community.alteryx.com/t5/Alteryx-Knowledge-Base/Combine-Spatial-Fields-Using-a-Summarize-Tool/ta-p/25685

https://community.alteryx.com/t5/Alteryx-Knowledge-Base/Spatial-Functions-with-the-Formula-Tool/ta-p/28409

Geocoders

http://www.findlatitudeandlongitude.com/batch-geocode/

https://www.programmableweb.com/news/7-free-geocoding-apis-google-bing-yahoo-and-mapquest/2012/06/21

 

Geographic analysis collection

https://community.alteryx.com/t5/Alteryx-Knowledge-Base/Spatial-Inputs-to-the-Market-Analysis-App-Geography-Analysis/ta-p/20196

 

Alteryx location analytics best practice

https://community.alteryx.com/t5/Alteryx-Knowledge-Base/Alteryx-Designer-Best-Practices-Spatial-Matching/ta-p/1176

 

Locality sensitive hashing with Spark

https://spark-summit.org/2016/speakers/alain-rodriguez/

 

Spark streaming

 

 

Tableau Fringe festival APAC

TCC fringe festival.PNG

I had the privilege of presenting at the Tableau fringe festival   #TFFAPAC on going from Struggle street to Easy street   – Using analytics to change the organisation.

Here’s the slide deck for everyone interested.

It was definitely a great experience hearing so many people share their stories, journey to analytics and a great learning experience for everyone. Best wishes and thanks go out to Fi and Emily for organizing the session. Will definitely post a full write-up when I get a moment..

Really enjoyed Craig Dewar’s hacks ,   Peter’s Orgill’s Street detective session,  Elinsa’s story  from dentistry to data and JJ Philips from alteryx.

Many thanks go out to Fiona (aka Vizchic) & Emily for organising!

 

How can you become an Alteryx ACE

I wanted to share the journey to becoming an Alteryx ACE and what it has meant for me personally. Alteryx is one of those amazing companies who has really changed the analytics landscape.

When I started my journey into Alteryx while working at amaysim, there were few Australian companies using the tool. I did some research online and strumbled on an amazing customer case study by BCG on their use of Alteryx.  I downloaded the product and built out my first workflow to acquire XML in a matter of hours. Such productivity was simply unheard of before.  A great place to start is  alteryx for excel users

 

When starting to use Alteryx, I highly recommending the use of containers (for business logic) and comment each step as to what you are trying to do. This is really important especially when you have to show your work to senior execs or show the challenges within the data. Also writing as you drag the icons allows you to focus on the business outcome instead of struggling with the syntax of the language

best practice alteryx.PNG

 

Often people ask me ‘Does this really work?’ My response is download the product and give it a spin then see that Alteryx can really get the data prep job done. Often this is the most time consuming task for an analyst.

The other often underestimated feature of alteryx is that you can also hand your work over to someone in the business to work on and then collaboratively switch between teams. In doing so you are no longer constrained by the headcount within your own analytics area.

In the realm of analytics, Alteryx really represents ‘power to the people’.  Not everyone can afford a data scientist, however everyone can afford Alteryx. I also particularly love that they focus on making predictive and prescriptive analytics more easily available to business analysts. Don’t spend time building the model, go visit the predictive district and download it!

power_to_the_people_by_bullmoose1912-d62p4pt

No longer are line of business analysts constrained to ‘putting in a request’ for IT to get a report or some advanced analysis. Managers demand insight and action yesterday!

I think my fellow ACE Jason Mack best summed it up – “This changes everything!”

Since then, customer stories of Alteryx have grown far and wide. Incredible use cases of Alteryx and Tableau occur in the most interesting of places. USDA using it to solve Avian flu was a great example of how Alteryx can actually be used to save lives!

Finally, I really encourage others to not hesitate, download the product and try it out. Feel free to reach out to myself and others within the community. Just as people in the Alteryx community have extended a helping hand when I most needed it, I’m happy to offer practical advice on how to get Alteryx into the organisation, overcoming ‘resistance’ and building  a centre of excellence empowering people of all skill levels to use Alteryx to drive change within their organisations.

engineeringtutorial.com_power-system-interview-question-and-answers_power-system-interview-question-and-answers

Another way to become an ACE is to help answer questions on the Alteryx community or help lead Alteryx user groups in your city.

Being part of the Alteryx and Tableau community has made me realize that I am no longer alone driving change within companies. I am now part of a wider global community who is incredibly helpful, encouraging and really sharing of experiences. This is really the basis of the ACE program. It’s really about sharing and supporting others in their journey and time of need.  I hope there will be more ACEs coming out of the Australia and APAC region and we can set an example for the rest of the world!

 

 

Leading the analytic revolution – The difference between a developer and an analyst

The world is becoming more connected than ever before. After returning from the alteryx analytic conference, it got me thinking on my own personal journey into using alteryx and tableau to drive organizational change.

  

Training is a sharing of life experiences beyond the textbook material

More recently I was in Melbourne training new analysts and business users  into how to use both alteryx and tableau together. What I saw amazed me was that so many people out there were still struggling with reporting, still stuck in using Microsoft excel and Access to get data from source systems and spending all that effort to meet reporting deadlines when they should providing analysis and recommendations to improve business performance.

  
Companies often looking for ways to drive cost and performance improvements

In all my years of working, every single company  is looking to drive improved operational and financial performance. Yet there remains a huge opportunity to see or invest in the right areas. Years ago, we went through an offshoring wave in analytics as companies were looking to lower their cost structures through the use of cheaper labour offshore. ( Cost out strategy) While this may help improve financial performance in the short term, the lack of closeness to the business ( aka where the action since) means that a lot of effort is actually wasted and lost in translation. It often also translates to an offshore model which is not truely in sync with the rapid changes in businesses.

  

The old way – Order taking – A waste of time and resource

  1. Business analysts takes business requirements from business stakeholders who not quite sure what they need or want
  2. PM organizes resources like developer, testers, database access
  3. Developer attempts to build from business requirements ( this column vs that column)and based on their own interpretation gets it wrong or does not foresee data quality issues or data interpretation or business rules.
  4. Business users wait 2-3 weeks for first sprint to complete and discover it’s not quite correct
  5. Repeat
  6. Finally after [x] months business users get a working report however by that time aesthetic the business has changed or another question needs to be asked

 
What if companies simply empowered their onshore analysts with the rights tools, training and thinking ?

Firstly this would drive transformational change through the organization. analysts would spend more time thinking of ways to drive operational improvements instead of struggling with data mingling. business leaders would be more empowered to change and act faster in their business environments.

      

    The new way – Collaborative analysis between business leaders and analysts ( Empowerment and automation)

    1. Download alteryx and tableau [10 minutes]    
    2. Watch YouTube training video [20 min]
    3. Business leaders asking questions on operational performance – What’s going on with my sales? Which customers are unprofitable? Why do you think this is going on.
    4. Business analyst co-locates with business leaders and builds analysis within a few hours, calls meeting with line managers and offers insight into their business and suggestions on how to drive improved performance on sales / profitability. Notice how different this is to build in just a dashboard that looks pretty but doesn’t serve business value.
    5. Business asks the next question, analysts builds on earlier analysis within hours and calls meeting a few hours again.
    6. Business analysts attends alteryx + tableau training to further upskill.

    The key difference is using the new way anyone within the business can get answers to their questions by combining data themselves and then quickly using tableau as a visualizer to see if there’s anything going on in their business. In doing so, they make analysts more productive and streamline their businesses for greater agility and so gives themselves a competitive advantage.

    Investing in training as a means of driving cultural change

    I am fortunate enough to share my analytics experience with different companies with the goal of helping them becoming more data driven.

    The most successful deployments of analytics in organizations often involve buying tools like alteryx and tableau followed up with ongoing investment in  up skilling staff to take advantage of the full capabilities of such tools.

    img_5862

    Training drives cultural change and cross company collaboration

    While tableau and alteryx are easy tools to learn,  investing in staff training often has a multiplier effect on cultural change. In fact it is best if at each training session, the stakeholders come from diverse business units such as sales, marketing, finance. Often in larger organizations analysts are in different pockets and don’t often may not know each other. Other times the various stakeholders know each other well and training becomes an opportunity to further cement those relationships

    Training is more than a recipe, rather a sharing of experiences

    Just as anyone can brew a cup of coffee, learning from a barrister on how to brew the best cup of coffee is something of a different experience. Often learning is not just about technical skills rather it is a knowledge sharing session of what works within their organisation, what pitfalls to avoid and how to actually use that knowledge once people have gotten back to their day jobs.

    I also enjoy learning from new users to alteryx and tableau and are often surprised by the many different ways of thinking that they come up with within the tools.

    Alteryx+TableauPic

    Learn how to drive the Ferrari 

    Alteryx as a tool is incredibly deep as it covers the full spectrum of analytics from ETL to spatial, predictive and prescriptive. Investing in training enables a person to quickly learn how to ‘drive’ it like a Ferrari and not a scooter.

    Similarly tableau is simple to learn however there’s a lot to mastery of visualization technique and finding the best way to present the message to influence business change.
    Start user groups whether formally or informally to keep the momentum

    Often within organisations there will be more advanced users as well as those just starting the journey. Encourage sharing of skills within Alteryx and Tableau with users of all skill levels with each other. The advanced user through teaching reinforces his knowledge while the newer user also gets the benefit of a mentor.

    But it costs money…. Not training is a false economy

    From an executive and management point of view, companies often heavily invest in training at both the senior leader and manager level thinking it will drive improved performance.

    This alone is not sufficient to drive the organization, training needs to filter down to gassroots level and not just be a computer based training routine to save on costs.

    It’s about engagement

    Some companies spend $1k per FTE per year. Assuming a decent analyst costs $100k per annum, in essence this equates to only 1% of the cost structure.  Trust me 1k doesn’t get you much these days. Companies  should be targeting a training pool of $3k-5k per analysts per year. This equates to between 3-5% of cost base. In return the analyst will drive either process improvements or highlight revenue and cost opportunities a magnitude of that!

    More good stuff from Alteryx inspire 2016

     

    The good stuff continues with George Mathew showcasing new features

    1. Data profiling out of the box, improved reporting web scheduling

    2.  Prescriptive analytics – Next release of Alteryx to support optimizationssimulations out of the box with a few clicks. Traditionally you would need a data scientist for this, now all i need is Alteryx. Good news for the layman like me..

    Want to know where is the best time to Surf in San Diego ?

    George Mathew and Cam showcasing the power of using time series and spatial data to see where is the best time to surf in San Diego!

     

     

    Alteryx + Microsoft Azure machine learning = Easy analytics 

    I could not believe how easy it was to use machine learning in Azure! Literally it was 3 clicks to drag and drop sentiment analysis in Alteryx. Unbelieveable..

    Applause from the Alteryx ACEs!

    HiMum

     

    Customer stories – Southwest airlines and Chickfil-A

    Don’t be  chicken! Use Alteryx..

     

    Customer share their tribulations and triumphs..

    Alteryx Developers getting direct feedback from customers

    Back at the solution centre I had the privilege of meeting Linda Thompon who is the development manager for alteryx engine. A lot of the functionality you see in alteryx today is done by her team. It’s great to be able to give direct feedback to her and the team to help improve the product in the future.

    LindaThompon

     

    Introduction to predictive district by Dr Dan

    For those new to predictive, I was lucky enough to attend Dr Dan’s session on predictive district. The key takeaway is that remember to check out the predictive district on public gallery where there’s so many more predictive tools to suit your needs. Please before you start coding any model check this out as it may have already been done for u!

    https://gallery.alteryx.com/#!districts/56327e37aa690a17f0760bdc

     

     

    Advanced Alteryx training for R users by Ramnath , Alteryx’s data scientist

    Great session as Ramnath showed us the tips and tricks of coding in writing portable R code (between R studio and alteryx). Alteryx to R to Alteryx made easier!

    The great thing about being trained in the US is that they focus a lot on the concepts and explaining why you are doing something rather than following a recipe!

     

     

    Next Alteryx inspire is at Las Vegas!! See u all then