build the union of a list of rdds

Solutions on MaxInterview for build the union of a list of rdds by the best coders in the world

Search
showing results for - "build the union of a list of rdds"
Riccardo
07 Nov 2018
1# Build the union of a list of RDDs
2
3path = os.path.join(tempdir, "union-text.txt")
4with open(path, "w") as testFile:
5  _ = testFile.write("Hello")
6textFile = sc.textFile(path)
7textFile.collect()
8# ['Hello']
9parallelized = sc.parallelize(["World!"])
10sorted(sc.union([textFile, parallelized]).collect())
11# ['Hello', 'World!']
similar questions
return an rdd created by coalescing all elements within each partition into a list return a new rdd containing the distinct elements in this rdd applies a function to all elements of this rdd return an rdd of grouped items return the intersection of this rdd and another onereturn a new rdd by applying a function to each element of this rdd
queries leading to this page
union of set in pythonbuild the union of a list of rddsbuild the union of a list of rdds