5 Takeaways from Tableau’s Hybrid Transactional/Analytical Processing

What makes Hyper so fast?
The Future of Enterprise Analytics: Hyper can handle both OLTP and OLAP simultaneously. In the future it will address NoSQL and graph workloads.

1. What is Hyper’s key benefit?

Hyper is a Hybrid transactional/analytical processing (HTAP) database system and replaces Tableau Data Extracts (TDE). The change will be mostly transparent for end users, other than everything being faster. Hyper significantly improves extract refresh times, query times and overall performance.

2. What is Hybrid transactional/analytical processing?

As defined by Gartner:

Hybrid transaction/analytical processing (HTAP) is an emerging application architecture that “breaks the wall” between transaction processing and analytics. It enables more informed and “in business real time” decision making.

The two areas of online transaction processing (OLTP) and online analytical processing (OLAP) present different challenges for database architectures. Currently, customers with high rates of mission-critical transactions have split their data into two separate systems, one database for OLTP and one so-called data warehouse for OLAP. While allowing for decent transaction rates, this separation has many disadvantages including data freshness issues due to the delay caused by only periodically initiating the Extract Transform Load (ETL) data staging and excessive resource consumption due to maintaining two separate information systems.

3. Does Hyper satisfy the ACID properties?

Hyper, initially developed at the Technical University of Munich and acquired by Tableau in 2016, can handle both OLTP and OLAP simultaneously. Hyper possesses the rare quality of being able to handle data updates and insertions at the same time as queries by using hardware-assisted replication mechanisms to maintain consistent snapshots of the transactional data. Hyper is an in-memory database that guarantees the ACID properties (Atomicity, Consistency, Isolation, Durability) of OLTP transactions and executes OLAP query sessions (multiple queries) on the same, arbitrarily current and consistent snapshot.

4. What makes Hyper so fast?

The utilization of the processor-inherent support for virtual memory management (address translation, caching, copy on update) yields both at the same time: unprecedentedly high transaction rates as high as 100,000 per second and very fast OLAP query response times on a single system executing both workloads in parallel. This would support real-time streaming of data in future releases of Tableau. These performance increases come from the nature of the Hyper data structures, but also from smart use of contemporary hardware technology, and particularly nvRam memory. Additional cores provide a linear increment in performance.

5. What does this mean for Tableau?

With Hyper now powering the Tableau platform, your organization will see faster extract creation and better query performance for large data sets. Since Hyper is designed to handle exceptionally large data sets, you can choose to extract your data based on what you need, not data volume limitations. Hyper improves performance for common computationally-intensive queries, like count distinct, calculated fields, and text field manipulations. This performance boost will improve your entire Enterprise Analytics workflow.

Join our “The Future of Enterprise Analytics” events and get a sneak peak at upcoming features and the Tableau Roadmap: 14th of November in Düsseldorf and 6th of December in Frankfurt.

Data Science Toolbox: How to use Julia with Tableau

R allows Tableau to execute Julia code on the fly
R allows Tableau to execute Julia code on the fly

Michael, a data scientist, who is working for a German railway and logistics company, recently told me during an FATUG Meetup that he loves Tableau’s R and Pyhton integration. As he continued, he raised the raised the question for using functions they have written in Julia. Julia, a high-level dynamic programming language for high-performance numerical analysis, is an integral part of newly developed data strategy in the Michael’s organization.

Tableau, however, does not come with native support for Julia. I didn’t want to keep Michael’s team down and was looking for an alternative way to integrate Julia with Tableau.

This solution is working flawless in a production environment since several months. In this tutorial I’m going to walk you through the installation and connecting Tableau with R and Julia. I will also give you an example of calling a Julia statement from Tableau to calculate the sphere volume.

1. Install Julia and add PATH variable

You can download Julia from julialang.org. Add Julia’s installation path to the PATH environment variable.

2. Install R, XRJulia and RServe

You can download base R from r-project.org. Next, invoke R from the terminal to install the XRJulia and the RServe packages:

> install.packages("XRJulia")
> install.packages("Rserve")

XRJulia provides an interface from R to Julia. RServe is a TCP/IP server which allows Tableau to use facilities of R.

3. Load libraries and start RServe

After packages are successfully installed, we load them and run RServe:

> library(XRJulia)
> library(Rserve)
> Rserve()

Make sure to repeat this step everytime you restart your R session.

4. Connecting Tableau to RServe

Now let’s open the Help menu in Tableau Desktop and choose Settings and Performance >Manage External Service connection to open the External Service Connection dialog box:

TC17 External Service Connection

Enter a server name using a domain or an IP address and specify a port. Port 6311 is the default port used by Rserve. Take a look on my R tutorial to learn more about Tableau’s R integration.

5. Adding Julia code to a Calculated Field

You can invoke Calculated Field functions called SCRIPT_STR, SCRIPT_REAL, SCRIPT_BOOL, and SCRIPT_INT to embed your Julia code in Tableau, such as this simple snippet that calculates sphere volume:

6. Use Calculated Field in Tableau

You can now use your Julia calculation as an alternate Calculated Field in your Tableau worksheet:

Using Julia within calculations in Tableau (click to enlarge)
Using Julia calculations within Tableau (click to enlarge)

Feel free to download the Tableau Packaged Workbook (twbx) here.

Further reading: Mastering Julia

Social Media and the Customer-centric Data Strategy #data17 #resources

Social media marketing mix
Do you analyze your social media marketing mix? | Photo Credit: via Richard Goodwin

With over 3 billion active social media users, establishing an active presence on social media networks is becoming increasingly essential in getting your business front of your ideal audience. These days, more and more consumers are looking to engage, connect and communicate with their favorite brands on social media.

Adding social media to your customer-centric data strategy will help boost brand awareness, increase followership, drive traffic to your website and generate leads for your sales funnel. In 2017, no organization should be without a plan that actively places their brand on social media, and analyzes their social media data.

Once you’ve started diving into social media analytics, how do you bring it to the next level? This session covers a customer-centric data strategy for scaling a social media data program.

Here are the links (i.e. additional resources) featured during the session to help you formulate your social media data program in order to build a stronger presence and retrieve powerful insights:

The Data Opportunity

TC17 Social Media Slides: The Data Opportunity

Focus on relevant metrics for your strategy

TC17 Social Media Slides: Sentiment Analysis

How to get Social Media in Tableau?

TC17 Social Media Slides: 3rd Party Platform Talkwalker

Tips to Level Up

TC17 Social Media Slides: Unshorten URLs in Tableau with R

Tutorials and Slide Set

The slides and tutorials presented at Tableau Conference on Tour in Berlin are also available on SlideShare, and on YouTube in English and German.

English Tutorials

German Tutorials

Slide Set

Tableau Conference TC17 Sneak Peek: Integrating Julia for Advanced Analytics

Using Julia within calculations in Tableau (click to enlarge)
Using Julia calculations within Tableau (click to enlarge)

We have already seen some love from Tableau for R and Python, boosting Tableau’s Advanced Analytics capabilities.

So what is the next big thing for our Data Science Rockstars? Julia!

Who is Julia?

JuliaJulia logo is a high-level dynamic programming language introduced in 2012. Designed to address the needs of high-performance numerical analysis its syntax is very similar to MATLAB. If you are used to MATLAB, you will be very quick to get on track with Julia.

Compared to R and Python, Julia is significantly faster (close to C and FORTRAN, see benchmark). Based on Tableau’s R integration, Julia is a fantastic addition to Tableau’s Advanced Analytics stack and to your data science toolbox.

Where can I learn more?

Do you want to learn more about Advanced Analytics and how to leverage Tableau with R, Python and Julia? Meet me at the 2017 Tableau Conferences in London, Berlin or Las Vegas and join my Advanced Analytics sessions:

Will there be an online tutorial?

Yes, of course! I published tutorials for R and Python on this blog. And I will also publish a Julia tutorial soon. Feel free to follow me on Twitter @xlth, and leave me your feedback/suggestions in the comment section below.

Further reading: Mastering Julia

A German translation of this post is published on the official Tableau blog: Tableau Conference On Tour Sneak Peek: Julia-Integration für Advanced Analytics

[Update 11 Oct 2017]: The Julia+Tableau tutorial blog post is now published.

Take the Survey: The Stage of Digital Transformation

This survey is part of my thesis that examines the topology, structure, and evolution of the digital transformation within organizations. Your contribution to this survey helps to better understand the perception regarding the current stage of digital transformation.

Follow this link to take the survey:

Spread the word and share this article! Thanks!