Data Engineer

Job Summary (List Format) Data Engineer

• Design, build, and maintain scalable data pipelines for batch and near-real-time processing.

• Develop robust data integration solutions connecting databases, APIs, cloud platforms, and external data sources.

• Write production-quality Python code for data ingestion, transformation, automation, and pipeline development.

• Build and maintain data transformation workflows using dbt.

• Develop and optimize complex SQL queries, stored procedures, and data transformations.

• Manage automated file transfers with external partners/vendors via SFTP, including scripting, validation, and troubleshooting.

• Build, schedule, monitor, and troubleshoot workflows using Apache Airflow.

• Design data models and structures to support analytics, reporting, and downstream applications.

• Implement data quality, validation, error handling, logging, and monitoring across data pipelines.

• Collaborate with stakeholders and technical teams to gather data requirements and deliver scalable solutions.

• Troubleshoot pipeline failures, performance issues, and data discrepancies.

• Develop reusable frameworks/components for data ingestion and integration.

• Ensure data pipelines are secure, reliable, scalable, and maintainable.

• Participate in code reviews, testing, documentation, and deployment processes.