Senior Data Engineer (Streaming)

We run one of the larger SaaS platforms for commercial insurance carriers, processing millions of policy transactions a year. Code review is required on every pull request, and nobody merges their own changes.

Position summary:

We're looking for another Senior Data Engineer (Streaming) to build and maintain the pipelines that feed our warehouse, dashboards and models. This is an in-person role at our Calgary location.

What you'll be doing:

• Manage warehouse objects, roles and spend in Snowflake, BigQuery or Redshift.
• Process large datasets with Spark on Databricks or EMR.
• Tune slow queries and expensive jobs.
• Stream events through Kafka or Kinesis into the lake and warehouse.
• Respond to pipeline failures and backfill data when needed.
• Write data quality tests with dbt or Great Expectations and alert on failures.
• Work with analysts to define metrics and fix discrepancies between reports.

What you need:

• Expert SQL, including window functions and query tuning.
• Understanding of dimensional modeling and slowly changing dimensions.
• Familiarity with Git, CI/CD and testing practices for data code.
• Experience with a cloud data warehouse (Snowflake, BigQuery, Redshift or Synapse)

Preferred qualifications:

• Terraform for data infrastructure.
• Kafka, Flink or Spark Structured Streaming.

Perks and benefits:

• Health spending account.
• RRSP matching.
• Paid vacation.
• Tuition reimbursement.
• Wellness spending account.
• Annual bonus.

Start: Immediately.

Demo job created on 11jobs for testing and demonstrations. It is not an actual opening at SLJ Software Inc..