Job Information
The Prudential Insurance Company of America Lead Data Engineer in Newark, New Jersey
Lead Data Engineer (The Prudential Insurance Company of America, Newark, NJ): Design and develop highly scalable and efficient data architectures that align with business requirements and support analytical use-cases. Provide technical expertise to the data engineering team, offering guidance on best practices, coding standards, and architectural designs. Design and implement robust ETL/ELT pipelines, leveraging AWS Cloud, Big Data technologies (Spark & Python), and Snowflake for seamless data ingestion, transformation, and loading from diverse sources. Demonstrate expertise in AWS data services, including S3, Glue, Redshift, EMR, Kinesis, Lambda, and showcase strong knowledge in Snowflake services such as external stage, security, assisted engineering, and the Snowflake marketplace. Create automations and CI/CD pipelines using tools like BitBucket, Jenkins, and Cloud Formation to streamline development and deployment processes. Optimize data storage, query performance, and costs on AWS and Snowflake resources to enhance operational efficiency. Ensure the implementation of data security best practices, encryption measures, and access controls to meet industry standards and regulations. Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and deliver actionable insights. Maintain comprehensive documentation of data engineering processes, data models, and infrastructure configurations to ensure transparency and ease of knowledge transfer.Telecommuting permitted up to 3 day(s) per week. Full time employment, Monday - Friday, 40 hours per week.MINIMUM REQUIREMENTS:Must have a Bachelor’s degree or foreign equivalent in Computer Science, Information Technology, or a related field, and 5 years of progressive post-baccalaureate related work experience. Alternatively, employer will accept a Master’s degree or foreign equivalent in Computer Science, Information Technology, or a related field, and 3 years of related work experience.Must have 3 years of experience in each of the following: •Extract-Transform-Load Experience; •Python; •Spark, Hadoop, Hive, and Pyspark; •Database, including SQLServer, Snowflake and Redshift; •Job Orchestrations, including Step Function and Airflow; •Version Control Tools, such as BitBucket, Github or Gitlab; •CI/CD Pipelines, specifically Jenkins or flyway pipelines; and, •Infrastructure as Code including CloudFormation, CDK or Terraform. Telecommuting permitted up to 3 day(s) per week.To Apply: Apply online at "https://bit.ly/3DFgESE". Should you have any difficulty in applying for this position through our website, please contact ramona.wallace@prudential.com for assistance in the application process.