News

Data Science Tool Adds Apache Spark Support

• By David Ramel
• October 16, 2015

The latest update of the data science tool from startup Dataiku includes support for Apache Spark, the open source data processing engine rapidly becoming one of the most popular technologies in use for Big Data analytics.

Version 2.1 of the company's Data Science Studio (DSS) is integrated with Spark and also provides better graphs and charts, notebooks with iPython, SparkR, R, Hive and Impala code samples, and certified plugins that let developers connect to varied data sources, Dataiku said.

Spark is ascending rapidly in the data analytics ecosystem, commonly described as the most active and popular big Data-related open source project and one of the most popular open source products of any kind.

"Pairing the capabilities of Spark with the advanced analytics features of DSS creates significant opportunities for those looking to leverage very large Hadoop data sets, often ranging into the terabytes, and it also allows users to process that information much more quickly," the company said in a statement.

The core components of DSS, called Visual Recipes, can now be executed on the Spark framework, letting developers take advantage of the SparkSQL programming language.

"Apache Spark integration also gives DSS the ability to work with Spark R, SparkSQL and PySpark, which brings R, SQL, and Python-based programing to the Spark environment," Dataiku said. "Much like the other components of Spark, PySpark and Spark R eases and speeds the native capabilities found in DSS and makes Spark a viable alternative to the traditional Hadoop/Hive stack, while also allowing analysts to share data engineering recipes and limit the need to recode or redevelop algorithms."

Having this year raised $3.7 million, the Paris-based startup now claims more than 50 customers of its " all-in-one predictive analytics development platform."