Senior Data Engineer (Spark)

Our engineering, trust and safety, and customer support teams work across offices in Houston and several other countries. More than 6,700 employees work on the product, from search ranking to fraud detection.

Role overview:

Velvetridge needs someone who writes solid SQL and Python and treats pipelines like production software. We hire in India through an employer of record, so this is a regular full-time job with local payroll and statutory benefits. Core hours overlap with US Central time, 9:00 AM to 1:00 PM CT. Reliable internet and a power backup are a must.

Job duties:

Write data quality tests with dbt or Great Expectations and alert on failures. Design table layouts and partitioning for Parquet, Delta Lake or Iceberg. Work with analysts to define metrics and fix discrepancies between reports. Process large datasets with Spark on Databricks or EMR. Maintain CI for data code and review pull requests. Tune slow queries and expensive jobs. Document datasets and lineage in a catalogue such as DataHub or Atlan. Build batch and streaming pipelines with Airflow, Dagster or Prefect.

Required qualifications:

Expert SQL, including window functions and query tuning. Understanding of dimensional modelling and slowly changing dimensions. Patience with messy source data and the habit of documenting assumptions. Strong Python for data processing. Familiarity with Git, CI/CD and testing practices for data code.

Nice to have:

Dbt Certified Developer or Databricks certification. Terraform for data infrastructure. Background in finance or marketing data.

Salary: ₹36,00,000 to ₹49,00,000 pa

What you get:

Health insurance through our employer-of-record partner. Annual team retreat. Paid sick days. Co-working space stipend. Home office setup budget. Paid parental leave.

Start: Immediately.

Demo job created on 11jobs for testing and demonstrations. It is not an actual opening at Velvetridge.