编程知识 cdmana.com

Apache spark 3.0 embraces big data processing and analysis engine

Catalog

Resource Overview

Speed

Easy to use

An introduction to

Everywhere


  Resource Overview

github The code base :https://github.com/apache/spark

Python,Scala,Java Take a look at the example :http://spark.apache.org/examples.html

Spark Java Version example :https://github.com/apache/spark/tree/master/examples/src/main/java/org/apache/spark/examples

As Java Programmers , The official website has made it as convenient as possible , All you want is here :

  It is said that , It is better to know a son than a father ,Apache spark Features and version guides can be found exactly :http://spark.apache.org/documentation.html

Spark Can do , Please keep in mind the following characteristics :

Speed

The speed of running the workload has increased 100 times .

Apache Spark Use the latest DAG The scheduler , Query optimizer and physical execution engine , High performance for batch and streaming data .

Hadoop and Spark Logical regression in

Easy to use

Use Java,Scala,Python,R and SQL Write applications quickly .

Spark Provides 80 Multiple senior operators , It's easy to build parallel applications . You can   from Scala,Python,R and SQL Shell  Interaction Use it .

df = spark . read .json “ logs.json”  df. among “ Age > 21”    . choice “ name.first” . Show
Spark Of Python DataFrame API
Infer read through automatic mode JSON file

An introduction to

Use a combination of SQL, Flow and complex analysis .

Spark power pool , Including a stack of  SQL and DataFrames,MLlib machine learning , GraphX and Spark flow . You can seamlessly combine these libraries in the same application .

Everywhere

Spark Can be found in Hadoop,Apache Mesos,Kubernetes, Stand alone or in the cloud . It has access to a variety of data sources .

You can go to EC2,Hadoop YARN,Mesos or Kubernetes Use it on Independent cluster mode function Spark . visit HDFSAlluxioApache CassandraApache HBaseApache Hive And hundreds of other data sources .

Bear in mind : The official website has always been the best teacher , Hearsay is not to be trusted !

版权声明
本文为[boonya]所创,转载请带上原文链接,感谢

Scroll to Top