Data Engineer » Pointerz

Data Engineer (MSBI and SQL Server Expert)

Reporting to the Manager of Data Engineering, the Data Engineer is someone who enjoys challenging work in a fast-paced environment and solving complex business problems with data and analytics, enabling our customers in their pursuit of better outcomes. You will experience the unique opportunity to migrate from traditional onprem data and technologies to cloud-based big data technologies in a multi-cloud data platform.

Responsibilities

Understanding and experienced in working on MSBI stack.
Expert in SQL Server.
Expert in creating and managing data pipelines end-to-end.
Expert in writing, analyzing and debugging T-SQL.
Proficient in SSIS and SSRS.
Experience in using Visual Studio and Git.
Build, automate and enhance data pipelines using batch processing and real-time streaming technologies and patterns.
Ensures data integrity throughout the ETL process and appropriately handle errors.
Design and implement onprem ETL solutions to Ingest, validate, transform, and publish both real-time stream data and batch data.
Create, modify, and optimize SSIS packages and SQL execution to improve performance.
Define and implement data models with SQL Server database technologies.
Build, maintain and optimize data warehouse tables and databases.
Ensures data integrity throughout the ETL process and appropriately handle errors.

Additional Responsibilities

Orchestrate data from multiple sources and sinks using Airflow and Azure Data Factory.
Design, develop and implement ETL solutions using cloud and non-cloud-based technologies for parallel processing using Spark and Databricks.
Define and implement data models with various database technologies like MongoDB, CosmosDB, Neo4j and MariaDB.
Migration of data and functionality from the onprem data platform and technologies to the cloud data platform and technologies.
Ingest and publish data from sources and to destinations via an API.

Skills

Bachelor’s degree in Computer Science, Management Information Systems or related discipline.
5+ years of experience designing, developing, validating, and automating ETL processes.
3+ years of experience traditional ETL tools such as Visual Studio, SQL Server Management
Studio, SSIS, SSAS and SSRS.
2+ years of experience with cloud technologies and platforms, such as: Kubernetes, Spark, Kafka,
Azure Data Factory, Snowflake, ML Flow, Databricks, Airflow or similar.
Must have experience with designing and implementing data access layers.
Must be an expert with SQL/T-SQL and Python.
Exposure to ETL/ELT with using Kafka or Azure Event Hubs with Spark or Databricks is a plus.Exposure to healthcare technologies and integrations for FHIR API, HL7 or other HIE protocols is a plus.