DetailPage logo

Data Engineer II

DetailPage

On-siteseniorPosted 5h ago

Job description

Overview

DetailPage.com helps Amazon brands grow organic traffic through content optimization and market segment share insights, using a patented (US Patent No. 12,093,340), AI-assisted process to identify high-traffic keywords and rewrite product content for sustainable organic growth. We're looking for a Data Engineer to build and optimize scalable data pipelines, support our ETL/ELT and API processes, and keep data flowing seamlessly across our organization and to our end users โ€” working closely with engineering, analytics, and product.

What you'll do

  • Design, develop, and maintain scalable data pipelines using a variety of AWS services

  • Work with relational databases such as Aurora (PostgreSQL), Redshift, and AWS Glue for large-scale data management

  • Develop and optimize ETL/ELT processes for smooth data integration from multiple sources

  • Collaborate with cross-functional teams to ensure data accessibility and integrity across platforms

  • Use Python, SQL, and Pandas for data manipulation, analysis, and workflow automation

  • Implement unit testing (PyTest) to ensure code reliability

  • Manage API integrations and development using FastAPI

  • Assist with infrastructure as code (IaC) using CDK and Terraform

  • Build CI/CD pipelines in Jenkins

What we're looking for

  • SQL mastery โ€“ expertise in complex SQL queries and database optimization

  • 2-3 years of experience in Data Engineering with AWS and Python

  • Linux proficiency โ€“ comfortable with shell scripting, common CLI tools, and building/testing Linux-based Docker images

  • Advanced Python โ€“ strong hands-on experience, ideally with Data Engineering tools like PySpark, Pandas, and SQLAlchemy

  • Git proficiency โ€“ comfortable with version control, branching, and collaboration

  • ETL/ELT experience โ€“ proven ability to build and optimize extract, transform, and load processes

  • Unit testing โ€“ familiarity with PyTest or similar tools

Nice to haves

  • AWS experience with Aurora (PostgreSQL or MySQL), Redshift, Fargate, Lambda, SQS, and IAM, plus tools like Boto3 and the AWS CLI

  • Airflow expertise, particularly AWS Managed Airflow, for scheduling and orchestrating data workflows

  • API development experience with API Gateway and FastAPI

  • Infrastructure as Code experience with CDK or Terraform

  • Familiarity with caching tools like Redis

  • Basic DevOps knowledge for CI/CD pipelines and dev workflow optimization

  • Familiarity with DBT

Compensation

Compensation and equity details to be finalized by hiring team.