Monkey Business: Always be Ready to Demo

The famous Tableau Superstore demo data set
The famous Tableau Superstore demo data set

Usually, I really don’t like looking on the screens of other passengers. On this early morning train from Frankfurt to Cologne, however, the screen of my seatmate caught my attention. Where have I seen the logo on his slide deck before? Two coffee sips later, it came to me: it was the logo of Monkey 47, a very delicious gin, distilled in the heart of the Black Forest. So I asked my neighbor: “Is that the Monkey 47 logo?”

He was happy that I recognized his brand and we had a small chat about gin and the Black Forest. Turns out his name is Thomas, and he is the head of Sales and Marketing for Monkey 47. Thomas mentioned that his team is planning a tour to promote Monkey 47 in a number of cities. That sounds similar to what we are doing with the Tableau Cinema Tour, so I showed him our Cinema Tour landing page and explained briefly who we are and what our mission is.

I asked him how he is organizing his data. Thomas revealed that he lives in Excel hell: “spreadsheets with thousands of rows and way too many columns”. This also sounded familiar. I opened up our Superstore.xlsx in Excel and asked: “Do your Excel sheets look like this?” Thomas replied: “Yes!”

Here we go! I drag’n’dropped the file on my Tableau desktop icon and paced through a 7-minute-demo ending with an interactive dashboard. Thomas was flabbergasted. To polish things off, I showed him the interactive Twitter Sentiment Dashboard embedded in my blog. Thomas grabbed his jacket and gave me a business card, saying: “We need Tableau!”

Monkey 47 business card (back side)
Monkey 47 business card (back side)

This story was originally written for Tableau’s EMEA Sales Newsletter. I think it’s a good read for the holidays, and wish you all Merry Christmas!

TabPy Tutorial: Integrating Python with Tableau for Advanced Analytics

TabPy allows Tableau to execute Python code on the fly
TabPy allows Tableau to execute Python code on the fly

In 2013 Tableau introduced the R Integration, the ability to call R scripts in calculated fields. This opened up possibilities such as K-means clustering, Random Forest models and sentiment analysis. With the release of Tableau 10.2, we can enjoy a new, fancy addition to this feature: the Python Integration through TabPy, the Tableau Python Server.

Python is a widely used general-purpose programming language, popular among academia and industry alike. It provides a wide variety of statistical and machine learning techniques, and is highly extensible. Together, Python and Tableau is the data science dream team to cover any organization’s data analysis needs.

In this tutorial I’m going to walk you through the installation and connecting Tableau with TabPy. I will also give you an example of calling a Python function from Tableau to calculate correlation coefficients for a trellis chart.

1. Install and start Python and TabPy

Start by clicking on the Clone or download button in the upper right corner of the TabPy repository page, downloading the zip file and extracting it.

TabPy download via GitHub web page

Protip: If you are familar with Git, you can download TabPy directly from the repository:

> git clone git://github.com/aloth/TabPy

TabPy download via Git command line interface

Within the TabPy directory, execute setup.sh (or setup.bat if you are on Windows). This script downloads and installs Python, TabPy and all necessary dependencies. After completion, TabPy is starting up and listens on port 9004.

2. Connecting Tableau to TabPy

In Tableau 10.2, a connection to TabPy can be added in Help > Settings and Performance > Manage External Service Connection:

Tableau main menu
Tableau main menu

Set port to 9004:

External Service Connection dialogue
External Service Connection dialogue

3. Adding Python code to a Calculated Field

You can invoke Calculated Field functions called SCRIPT_STR, SCRIPT_REAL, SCRIPT_BOOL, and SCRIPT_INT to embed your Python script in Tableau:

Python script within Tableau
Python script within Tableau

4. Use Calculated Field in Tableau

Now you can use your Python calculation as Calculated Field in your Tableau worksheet:

Tableau workbook featuring a Python calculation
Tableau workbook featuring a Python calculation

Feel free to download the Tableau Packaged Workbook (twbx) here.

[Update 3 Jan 2017]: Translated to Japanese by Tomohiro Iwahashi: Tableau + Python 連携 (Tabpy) を使ってみよう!

[Update 30 Mar 2017]: A German translation of this post is published on the official Tableau blog: TabPy Tutorial: Integration von Python mit Tableau für Advanced Analytics

7 Fragen, die Unternehmen helfen ihr Ergebnis mit Social Media zu steigern

Twitter Sentiment Analysis: klicken, um interaktives Dashboard zu öffnen
Twitter Sentiment Analysis: klicken, um interaktives Dashboard zu öffnen

Ist der Einsatz sozialer Netze in Ihrem Unternehmen auf Marketing beschränkt, und lässt dadurch Chancen ungenutzt?

Noch immer schöpfen viele Unternehmen in Deutschland die Möglichkeiten von Social Media nur unzureichend aus. Die meisten Firmen nutzen Social Media lediglich als Marketinginstrument, senden zum Beispiel in Intervallen die gleichen Inhalte. Wesentlich weniger Unternehmen setzen Social Media dagegen in der externen Kommunikation, in Forschung und Entwicklung, zu Vertriebszwecken, oder im Kundenservice ein.

Nachfolgend betrachten wir die Twitter-Kommunikation von vier Social-Media-affinen Unternehmen etwas näher, und zeigen anhand sieben Fragestellungen was sie anders machen und wo die übrigen Nachholbedarf haben.

1. Wann und wie werden Tweets gesendet?

Ein Blick auf das Histogram lässt auf reichlich Interaktion schließen (Tweets und Replies), während das Weiterverbreiten von Tweets (Retweets) eher sporadisch auftritt:

 

2. Wie umfangreich sind die Tweets?

Wie es scheint, reitzen die meisten Tweets die von Twitter vorgesehenen 140 Zeichen aus – oder sind zumindest nahe dran:

 

3. An welchen Wochentagen wird getweetet?

Am Wochenende lässt die Kommunikation via Twitter nach. Die Verteilung der Emotionen bleibt dabei gleich, unterscheidet sich aber von Unternehmen zu Unternehmen:

 

4. Zu welcher Tageszeit wird getweetet?

Auch nachts werden weniger Tweets verfasst. Bei Lufthansa kommt es dabei recht früh zu einem Anstieg durch Pendler-Tweets, etwas später tritt dieser Effekt bei der Deutschen Bahn ein: 

 

5. Welche Art der Kommunikation herrscht vor?

Der hohe Anteil an Replies bei Telekom, Deutsche Bahn und Lufthansa impliziert, dass diese Unternehmen Twitter stark zum Dialog nutzen. Unter den Tweets der Deutsche Bank ist hingegen der Anteil an Retweets – insbesondere bei jenen mit Hashtag – deutlich höher, was auf einen höheren Nachrichtengehalt schließen lässt:

 

6. Welche User sind besonders aktiv?

Nun betrachten wir die Twitter-User, welche die entsprechend Twitter-Handles der Unternehmen besonders intensiv nutzen:

 

7. Welche Tweets erzeugen Aufmerksamkeit?

Diese Frage lässt sich am besten interaktiv im Dashboard (siehe auch Screenshot oben) untersuchen. Entscheidend ist bei dieser Betrachtung die Ermittlung der Emotion durch eine Sentiment-Analyse.

Je nach Emotion und Kontext ist es vor allem für das adressierte Unternehmen von Interesse rechtzeitig und angemessen zu reagieren. So lässt sich eine negative Stimmung frühzeitig relativieren, und so Schaden an der Marke abwenden. Positive Nachrichten können hingegen durch Weiterreichen als Multiplikator dienen.

How to implement Sentiment Analysis in Tableau using R?

Interactive sentiment analysis with Tableau 9.2
Interactive sentiment analysis with Tableau 9.2

In my previous post I highlighted Tableau’s text mining capabilities, resulting in fancy visuals such as word clouds:

Today I’d like to follow up on this and show how to implement sentiment analysis in Tableau using Tableau’s R integration. Some of the many uses of social media analytics is sentiment analysis where we evaluate whether posts on a specific issue are positive, neutral, or negative (polarity), and which emotion in predominant.

What do customers like or dislike about your products? How do people perceive your brand compared to last year?

In order to answer such questions in Tableau, we need to install an R package that is capable of performing the sentiment analysis. In the following example we use an extended version of the sentiment package, which was initiated by Timothy P. Jurka.

The sentiment package requires the tm and Rstem packages, so make sure that they are installed properly. Execute these commands in your R console to install sentiment from GitHub (see alternative way to install at the end of this blog post):

The sentiment package offers two functions, which can be easily called from calculated fields in Tableau:

Screenshot 2016-01-31 15.25.24 crop

The function get_polarity returns “positive”, “neutral”, or “negative”:

The function get_emotion returns “anger”, “disgust”, “fear”, “joy”, “sadness”, “surprise”, or “NA”:

The sentiment package follows a lexicon based approach and comes with two files emotions_english.csv.gz (source and structure) and subjectivity_english.csv.gz (source and structure). Both files contain word lists in English and are stored in the R package library under /sentiment/data directory.

If text is incorrectly classified, you could easily fix this issue by extending these two files. If your aim is to analyze text other than English, you need to create word lists for the target language. Kindly share them in the comments!

Feel free to download the Packaged Workbook (twbx) here.

[Update 11 Aug 2016]: If you are having trouble with install_github, try to install directly form this website:

How to perform Text Mining at the Speed of Thought directly in Tableau?

Interactive real-time text mining with Tableau 9.2
Interactive real-time text mining with Tableau 9.2

When I was doing text mining, I was often tempted to reach out for a scripting language like R, Python, or Ruby – and then I feed the results into Tableau. Tableau served as a communications tool to represent the insights in a pleasant way.

Wouldn’t it be handy to perform text mining and further analysis at the speed of thought directly in Tableau?

Tableau has some relatively basic text processing functions that can be used for calculated fields. This is, however, not enough to perform text mining such as sentiment analysis, where it is required to split up text in tokens. Also Tableau’s beloved R integration will not help in this case.

As a workaround, I decided to use Postgres’ built-in string functions for such text mining tasks, which perform much faster than most scripting languages. For the following word count example, I applied the function regexp_split_to_table that takes a piece of text (such as a blog post), splits it by a pattern, and returns the tokens as rows:

I joined this code snippet as a Custom SQL Query to my Tableau data source, which is connected to the database that is powering my blog:

Screenshot 2016-01-14 15.34.46

And here we go, an interactive word count visualization:

 

This example could be easily enhanced with data from Google Analytics, or altered to analyse user comments, survey results, or social media feeds. Do you have some more fancy ideas for real-time text mining with Tableau? Leave me a comment!

[Update 19 Jan 2016]: How to identify Twitter hashtags? Do I need another RegEx?

Another regular expression via a Custom SQL Query is not required for identifying words within tweets as hashtags. A simple calculated field in Tableau will do the job:

Looking for an example? Feel free to check out the Tweets featuring #tableau Dashboard on Tableau Public and download the Packaged Workbook (twbx):

Tweets featuring #tableau Dashboard

Any more feedback, ideas, or questions?