Experience: 5+ years

Location: Pune, India (Work from Office)

Must Have:

• Spark development with batch and streaming experience
• Experience with Pyspark and Data Engineering
• ETL implementation and migration to spark
• Programming knowledge on scala, java & python
• Experience with Kafka and Spark streaming (Dstream and Structured Streaming)
• Experience with using Jupyter notebooks or any other developer tool
• Experience with Airflow or other workflow engines

Good to Have Skills:

• Flink and Kudu streaming
• Automation of workflows CI/CD
• Nifi streaming and transformation
• Informatica workflow migration

Apply for this position

The post Senior Big Data Engineer appeared first on Emergys.