Released over 2014, it was a major release as it adds on a major new component SPARK SQL for loading and working over structured data in SPARK. It was a major release as Spark was updated with Scala 2.10. Spark streaming’s were improved Streaming listeners were introduced. we can use the Graph library to build graphs from RDD and then we can transform graphs and extract subgraphs.
#Latest version of spark for mac windows#
Windows operators were speeding up to 50%. Spark Conf class was now a preferred way to configure advanced settings on our spark context. It includes the first version of GraphX, a powerful tool for graph processing. It updates spark over scala2.10 with various libraries’ addition and improvements. It is a major release over the starting of 2014. The Deployment of applications was easy with extending EC2 capabilities. Support for YARN was introduced over this version. Machine Learning Library was introduced and now we can run our spark jobs over the YARN cluster. it contains all the information about running, completed, failed jobs. Monitoring UI and dashboards were introduced by a default port 4040. This version of spark was released in Sept 2013. Shuffle operations were introduced and performance improvements were introduced. EC2 was introduced which reads s3 credential from AWS_ACCESS_KEY and AWS_SECRET_KEY that made it easy to access s3. Some native libraries were introduced NumPy, SciPy.
It was a major release as python API was introduced known as Pyspark that makes it possible for the spark to use with python.
Version 0.7 was introduced over the starting of 2013. It was deployed over Maven central and now can be used over maven projects. Persist() method was used over RDD and more join operators were introduced. Standalone deploy mode was introduced that made it easy to launch the cluster without installing an external cluster manager. Just after the release of 0.5 after few months October 2012 spark made a new release of a version that brought several new features and architectural changes and performance enhancements.
0 kommentar(er)
