Data Engineer (ID#5281)
Vancouver - Hybrid
TimeNew Value Solutions, a national IT consulting company, is seeking a Data Engineer to support the rebuild of reporting assets, including scorecards, dashboards, and regular operational reports, as well as reporting submissions to a government Ministry, which require record-level data from multiple community systems. A key objective is to increase the speed of rebuilding data assets and reconnecting reports following system go-lives, while reducing periods of non-compliance.
Working within Microsoft Azure with a strong focus on Azure Databricks and Delta Lake architecture, the engagement will provide hands-on data engineering and data solutions development support. In addition to rebuilding pipelines and analytic objects, the role is expected to contribute to optimizing data flows, strengthening validation processes, and implementing scalable solutions that enhance resilience and reduce delays during ongoing clinical system transformations.
The Data Engineer will possess expertise and recent experience using Databricks for data transformations, DLTs, and optimization. You should have experience working within the Azure platform using data services such as Azure Databricks, Azure Data Factory, and Azure DevOps. Additionally, you must have a strong understanding of healthcare systems and prior experience developing data solutions with data from systems such as Oracle Cerner, PARIS, Alayacare, and Pixalere.
Responsibilities:
Ingestion and Integration of Data from new Systems
- Collaborate with the project team to offer leadership and advice on best practices in developing robust data pipelines.
- Work with analytics partners to understand workflows, practices, and analytical requirements to provide solutions and architecture optimized to business needs.
- Support the design and implementation of master data products.
- Assist with project activities including defining technical scope, analyzing tasks, eliciting requirements, drafting specifications, and planning technical tasks.
- Serve as the primary resource for addressing implementation issues and activities escalated by the organization team.
- Actively participate in the implementation of data pipelines and products – the Data Engineer will be fully integrated into the project, designing and developing usable products based on business specifications.
Optimization of Technical Areas
- Collaborate with the core Data Engineering platform team to advise on best practices for data ingestion via Databricks, including DLTs, code optimization, cluster configuration, z-ordering, partitioning, and data consumption strategies such as interactive clusters vs SQL warehouses and scaling.
- Offer continued guidance and support to the technical team in optimizing and refining the processes of data ingestion, transformation, and consumption within the new platform.
- Communicate regularly and openly with project team and departmental leadership to foster team collaboration, provide project updates, and highlight any risks to meeting project deadlines and deliverables.
Requirements:
- Proven experience as a data engineer using Microsoft technologies, such as Azure Data Factory, Azure DevOps.
- Expert experience with Azure Databricks .
- Strong proficiency using Python, Spark, and other big data technologies.
- Strong understanding of healthcare systems and their data models – Cerner, PARIS, Pixalere, Alayacare is preferred.
- Familiarity with SQL queries, SQL Server, and relational databases.
- Knowledge of data warehousing designs, data modeling.
- Strong analytical, problem-solving, and organizational skills.
- Excellent verbal and written communication abilities.
If you have this expertise, and are able to work in Canada, please submit your resume. While we thank all candidates in advance for their application, only those candidates who are shortlisted will be contacted.
[applyform]