Bigdata Engineer

• Experience: Min 5 Years
• Must have good technical experience and should be able to understand and develop the modules and bring the task to closure on time.
• At least 4+ years of development work experience in Hadoop programming (HDFS) using PySpark/Scala/Java with Hive-based Data warehouse projects. And, with good Shell Scripting experience.
• Able to Understand the requirements and ingest the data into the Bigdata platform using Spark with HIVE.
• Should be able to write the SQL scripts and write few complex SQL Scripts to process the data with SCD handling exposure.
• Responsible for meeting the quality, and scope as per the project plans must be able to meet the stringent timelines.
• Flexible to work as per the regional timing.
• Excellent communication and documentation skills
• Very good at team playing and flexibility to work with the Offshore team in different time zones based on project needs.
• Willingness to do Certification in Databricks Certified Developer: Apache Spark 3. X

Nice to Have

• Autosys, Bit bucket, Jenkin, etc.
• Proactive, and have good communication skills to articulate technical issues.
• Exposure to is Confluence/JIRA
• Should be able to work independently Prior Experience to any Databases like Oracle/SQL Server/ETL will be added advantage.
• At least 1+ years of Module Lead Experience
• Hands-on experience in data analysis and debugging the issues in SQL.
• Performance tuning the Hive QLs and the Cluster nodes Functional Testing Functional Test plan and execution experience.

Originally posted on Himalayas