connect spark to postgres 3b connect spark to database

Solutions on MaxInterview for connect spark to postgres 3b connect spark to database by the best coders in the world

home

app

help

profile

we are a community of more than 2 million smartest coders

registration for

employee referral programs

are now open

get referred to google, amazon, flipkart and more

showing results for - "connect spark to postgres 3b connect spark to database"

Silvia

17 Jan 2018

1spark = SparkSession.builder.config(conf=spark_conf).getOrCreate()
2
3#make sure the necessary jar files are loaded
4#@ opt/spark/jars folder
5
6df = spark.read \
7    .format("jdbc") \
8    .option("url", "jdbc:postgresql://<host>:<port>/<db_name>") \
9    .option("dbtable", "<schema>.<tablename>") \
10    .option("user", "<username>") \
11    .option("password", "<password>") \
12    .option("driver", "org.postgresql.Driver") \
13    .load()