AMVETS Jobs

We are seeking a Data Engineer to design, develop, and optimize scalable data pipelines supporting advanced analytics and machine learning solutions in a cloud-based environment . The ideal candidate has hands-on experience with Azure Data Services and Databricks , a strong background in data pipeline orchestration , proven expertise in data quality management and process automation , and experience in Procurement or Supply Chain.

Key Responsibilities:

1. Data Pipeline Architecture & Development:

• Design, develop, and maintain robust ETL/ELT pipelines to handle large-scale data ingestion, transformation, and integration.

• Build and optimize data workflows using Azure Data Factory, Databricks (PySpark, Spark SQL), and Azure Synapse Analytics.

• Ensure pipeline scalability, fault tolerance, and efficiency across diverse data sources, primarily structured (tabular) datasets.

• Implement incremental loads, change data capture (CDC), and other advanced data ingestion strategies.

2. Automation & Process Optimization:

• Develop and maintain automated data pipelines with a focus on performance optimization and cost-efficiency in the Azure environment.

• Implement CI/CD pipelines for seamless deployment of data solutions, leveraging DevOps tools and Databricks Workflows.

• Collaborate with cloud architects to optimize resource usage and adhere to cloud governance best practices.

3. Data Management & Quality Assurance:

• Lead the design and implementation of data quality frameworks to ensure data integrity, consistency, and compliance across systems.

• Develop monitoring solutions for pipeline health, data freshness, and anomaly detection.

• Maintain comprehensive documentation covering data models, transformation logic, and operational procedures.

4. Cross-functional Collaboration & Stakeholder Engagement:

• Partner with Data Scientists, Analysts, and Business Stakeholders to understand data needs and translate them into effective solutions.

• Facilitate integration of machine learning models into production data pipelines.

• Provide technical mentorship to junior data engineers and contribute to team knowledge-sharing initiatives.

Required Skills & Qualifications:

Education: Bachelor’s degree in Computer Science, Data Engineering, Analytics, Statistics, Mathematics, or a related field. (Master’s degree is a plus.)

Experience:

• 3+ years of hands-on experience in data engineering or a related discipline.

• Proven experience designing and deploying end-to-end data pipelines in Azure and Databricks environments.

Language: Proficiency in English (written and spoken) is required, with strong English skills being prioritized.

Technical Skills:

Programming & Data Processing:

• Advanced proficiency in SQL and Python for data manipulation, transformation, and analysis.

• Extensive experience with PySpark and Spark SQL for big data processing in Databricks.

Cloud & Data Services (Azure):

• In-depth knowledge of Azure services, including:

• Azure Data Factory (ADF) for pipeline orchestration

• Azure Data Lake Storage (ADLS) for data storage and management

• Azure SQL Database for relational data management

• Experience with Azure Functions and event-driven architectures is a plus

Automation & DevOps:

• Hands-on experience implementing CI/CD pipelines using tools like Azure DevOps, GitHub Actions, or similar.

• Familiarity with infrastructure-as-code (IaC) tools such as Terraform or ARM templates.

• Experience with Databricks Workflows and job orchestration tools.

Data Management & Warehousing:

• Strong understanding of data lakehouse architectures and data warehousing solutions (e.g., SQL Server, Redshift, BigQuery).

• Experience designing and maintaining data models and schema designs for analytical use cases.

• Familiarity with data governance, security best practices, and compliance standards.

Machine Learning Integration (Preferred):

• Experience supporting machine learning workflows and integrating models into production pipelines.

• Understanding ofMLOps practices is a plus.

Preferred Qualifications:

• Experience with real-time data processing (e.g., Apache Kafka, Azure Stream Analytics).

• Familiarity with Power BI data connections and reporting structures.

• Hands-on experience with Databricks Workflows for complex pipeline orchestration.

Corteva Agriscience™ is an equal opportunity employer. We are committed to boldly embracing the power of inclusion, diversity, and equity to enrich the lives of our employees and strengthen the performance of our company, while advancing equity in agriculture. Qualified applicants will be considered without regard to race, color, religion, creed, sex, sexual orientation, gender identity, marital status, national origin, age, veteran status, disability or any other protected class. Discrimination, harassment and retaliation are inconsistent with our values and will not be tolerated. If you require a reasonable accommodation to search or apply for a position, please visit:Accessibility Page for Contact Information

For US Applicants: See the ‘Equal Employment Opportunity is the Law’ poster. To all recruitment agencies: Corteva does not accept unsolicited third party resumes and is not responsible for any fees related to unsolicited resumes.