Job Title:- Lead GCP Data Engineer
Location:- Hybrid ( 4 days a month to one of the Client offices in Irving, TX/ Tampa, FL)
Duration:- 8+ months
Description:
Data engineer who can work on migrating the Spark (Pyspark / Scala Spark) jobs to Cloud-based environments, Data pipeline building using Spark, and Bigquery, Deploying the jobs to multiple environments, and monitoring the Production jobs. Coordinate with offshore & onsite teams, for any report generation. Migrating the scripts from Oozie to GCP Composer-based scripts
Tech Stack :
Location:- Hybrid ( 4 days a month to one of the Client offices in Irving, TX/ Tampa, FL)
Duration:- 8+ months
Description:
Data engineer who can work on migrating the Spark (Pyspark / Scala Spark) jobs to Cloud-based environments, Data pipeline building using Spark, and Bigquery, Deploying the jobs to multiple environments, and monitoring the Production jobs. Coordinate with offshore & onsite teams, for any report generation. Migrating the scripts from Oozie to GCP Composer-based scripts
Tech Stack :
- Spark
- GCP Composer (Dag's creation)
- GCP Dataproc
- GCP BigQuery, GCS
- Oozie
- HiveQL
- Teradata SQL