Creating a connection
Note:
- config values passed from SparkSubmitOperator takes precedence over the connection parameters.
- Using the spark-home in extra properties will work for all versions in case of multiple spark installations
- default deployment mode is client. Adding 'deploy-mode':'cluster' in extra properties will change it to cluster mode
No comments:
Post a Comment