Data Engineer (Databricks)

Public sector agencies, utilities and large health systems licence our asset management platform under multi-year contracts. Annual releases gave way to two-week sprints years ago, and the platform now handles over 200 million API calls a day. Founded in 1973, we now have 2,900 employees.

The role

Torres Apps needs someone who writes solid SQL and Python and treats pipelines like production software. This hire will help replace a set of fragile nightly scripts with tested, monitored pipelines. The company is based in San Francisco, the US. The team is spread across multiple countries. We engage team members in Bangladesh as full-time contractors. Most of our contractors have been with us for more than two years. Our standups are at 9:00 AM Pacific. We use Slack, Notion and Google Meet. Cameras on for team calls.

Duties

• Build batch and streaming pipelines with Airflow, Dagster or Prefect
• Design table layouts and partitioning for Parquet, Delta Lake or Iceberg
• Model data in dbt using staging, intermediate and mart layers
• Ingest data from APIs, SaaS tools and databases with Fivetran, Airbyte or custom Python
• Maintain CI for data code and review pull requests
• Tune slow queries and expensive jobs
• Respond to pipeline failures and backfill data when needed

Required qualifications

• 3-5 years of data engineering, or backend work with data at volume
• Understanding of dimensional modelling and slowly changing dimensions
• Hands-on with an orchestration tool such as Airflow
• Working knowledge of AWS, GCP or Azure data services

Preferred qualifications

• Reverse ETL tools like Hightouch or Census
• Dbt Certified Developer or Databricks certification

Salary range: $1,300-$1,650 monthly

Pay and benefits

• Paid in USD
• Monthly payments via wire transfer or Wise
• Company laptop
• Annual rate review

Please submit your CV in English.

Include your expected salary and earliest start date with your application.

Duration: 6 months. Start: Within 30 days.

Demo job created on 11jobs for testing and demonstrations. It is not an actual opening at Torres Apps, LLC.