Overview
We are seeking a highly skilled and motivated individual to join our organization as the Manager of Data Engineering. As the Manager of Data Engineering within our Enterprise Data Platform (EDP) Data Operations Team, you will be responsible for overseeing and leading a team of data engineers to develop and maintain our data pipelines, data infrastructure and systems. You will collaborate closely with cross-functional teams, including data scientists, analysts, and software engineers, to ensure efficient and reliable data integration, storage, and processing. This is a crucial role that requires technical expertise, leadership abilities, and a strategic mindset.
Responsibilities
- Team Leadership: Lead and manage a team of data engineers, providing guidance, mentorship, and technical expertise. Set clear goals, establish priorities, and ensure successful project execution. Foster a collaborative and innovative work environment.
- Data Infrastructure Strategy: Develop and execute the data engineering strategy to support the organization's data needs, ensuring scalability, reliability, and performance. Continuously evaluate and recommend new technologies, tools, and methodologies to optimize data engineering processes.
- Data Pipeline Development: Oversee the design, development, and maintenance of robust and scalable data pipelines for efficient data ingestion, processing, transformation, and storage. Ensure data quality and integrity throughout the pipeline.
- Data Modeling and Integration: Collaborate with data scientists and analysts to design and implement data models that support business intelligence, analytics, and reporting needs. Integrate data from various sources, both internal and external, ensuring compatibility and seamless data flow.
- Performance Optimization: Identify and address bottlenecks, inefficiencies, and data processing issues to optimize the performance and reliability of data systems. Monitor system performance, conduct regular capacity planning, and implement necessary optimizations.
- Data Security and Governance: Establish and enforce data security and governance policies and best practices to ensure data privacy, confidentiality, and compliance with regulatory requirements. Implement and maintain appropriate access controls and data protection measures.
- Collaboration and Communication: Foster effective collaboration and communication with stakeholders across the organization, including data science teams, business units, and executive leadership. Clearly communicate technical concepts to non-technical stakeholders.
- Continuous Learning and Development: Stay updated with the latest trends, techniques, and advancements in data engineering, and facilitate knowledge sharing within the team. Encourage a culture of continuous learning and professional growth.
Qualifications
- Bachelor's or Master's degree in computer science, data science, or a related field. Advanced certifications in data engineering or relevant domains are a plus.
- Proven experience (X+ years) in data engineering, including hands-on experience with data integration, data modeling, and ETL processes.
- Strong leadership skills with experience in managing and leading data engineering teams.
- Proficiency in programming languages like Python, SQL, and Java/C#. Experience with big data technologies such as Hadoop, Spark, and NoSQL databases.
- In-depth understanding of data warehousing concepts, data architecture, and cloud-based data platforms (e.g., AWS, Azure, GCP).
- Knowledge of data governance, security, and compliance practices.
- Excellent problem-solving, analytical, and troubleshooting skills.
- Strong communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams.
- Demonstrated ability to manage multiple projects and priorities in a fast-paced environment.
