Data Engineer

The Lead Data Engineer is responsible for designing and implementing modern data platforms that meet business objectives. This involves identifying data opportunities, developing end-to-end data solutions, and working with internal and external teams to ensure successful implementation.

Responsibilities:

• Lead and mentor a team of data engineers to develop cloud data solutions using a combination of technologies.
• Take technical ownership of all solutions designed and developed as part of end-to-end data solutions.
• Lead technical work streams in a Scrum/Agile software engineering practice focused on delivery.
• Engage with analysts, analytics, and project teams early on to frame use cases and identify data challenges and opportunities.
• Develop and establish best practice design patterns and development processes.
• Develop and establish DevOps best practice and work closely with hosting and infrastructure teams to implement best practices.
• Engage in hands-on development of data models, ingestion pipelines, and analytics datasets for batch and streaming data.
• Document technical implementation details and artifacts.
• Serve as a subject matter expert for the technology stack and demonstrate initiative and ownership of deliverables during development, testing, and post-production phases.

Qualifications:

• Bachelor’s Degree in Computer Science or Software Engineering or equivalent experience.
• 7+ years of experience in Data Engineering and 10+ years in Information Technology.
• Experience building batch and stream pipelines on Azure or other cloud platforms.
• Experience with diverse ways of storing and processing data (file vs SQL vs NO-SQL, batch vs streaming, stateful vs stateless).
• Experience with relevant technology stacks such as Azure, Apache Airflow, Blob, Azure Data Lake, Talend, Azure Streaming, Spark, Azure Data Factory, SQL Server, Power BI, Domo, or similar analytic tools.
• Experience modeling and building data warehouses for analytics consumption.
• Strong interpersonal skills, customer-centric attitude, proven team player, and team builder.