Analytics Blog

Notes on Upgrading to Spark 1.5.1

The speed at which Spark is evolving is impressive, and it has become the standard in optimizing performance for iterative models on Hadoop. However, the burden it presses on companies that rely on it is its need to be updated constantly. While Spark does a good job with maintaining API compatibility, there are always some interesting hang ups when migrating from one version to the next.Alpine is a power user of Spark as it allows our customers to achieve optimum performance and scalability within the product. Our Director of Engineering, Chester Chen, has written up a nice summary of notes from our own Spark upgrade to make your transition more seamless.

Download the PDF here