Data Engineer (Databricks)

Overview:

We're looking for a Data Engineer (Databricks) to join our team in Porto. Data engineering role supporting analysts and data scientists. Most of the work is in dbt, Airflow and the cloud warehouse. The Finance Data team moves data from about 30 source systems into Snowflake. You'll own ingestion, modelling and data quality checks. This position is on site in Porto.

What you'll be doing:

• Write data quality tests with dbt or Great Expectations and alert on failures.
• Process large datasets with Spark on Databricks or EMR.
• Respond to pipeline failures and backfill data when needed.
• Maintain CI for data code and review pull requests.
• Manage warehouse objects, roles and spend in Snowflake, BigQuery or Redshift.
• Design table layouts and partitioning for Parquet, Delta Lake or Iceberg.
• Ingest data from APIs, SaaS tools and databases with Fivetran, Airbyte or custom Python.
• Document datasets and lineage in a catalogue such as DataHub or Atlan.

Qualifications:

• 3+ years of data engineering, or backend work with data at volume.
• Experience with a cloud data warehouse (Snowflake, BigQuery, Redshift or Synapse)
• Strong Python for data processing.
• Understanding of dimensional modelling and slowly changing dimensions.

Preferred qualifications:

• Kafka, Flink or Spark Structured Streaming.
• Background in finance or marketing data.

Compensation and benefits:

€2,250 to €2,700 monthly

• Private health insurance.
• Flexible start and finish times.
• Meal vouchers.
• Training budget.

About TF Labs:

TF Labs is a 19-person software shop in Porto. The founders still answer support tickets. Our product started as a side project in 1993 and now has a few hundred paying customers, mostly in Porto.

Only shortlisted candidates will be contacted.

Please submit your CV in English.

Start: Immediately.

Demo job created on 11jobs for testing and demonstrations. It is not an actual opening at TF Labs Lda..