parttime Data-, Telecomspecialist vacatures

Lead Engineer - GCP_ Snowflake

About the job

Logistics at full potential.

At GXO, we're constantly looking for talented individuals at all levels who can deliver the caliber of service our company requires. You know that a positive work environment creates happy employees, which boosts productivity and dedication. On our team, you'll have the support to excel at work and the resources to build a career you can be proud of.

As the Lead Data Engineer, you will lead the design and development of robust and scalable data pipelines on a modern data platform. You will work closely with cross-functional teams to understand data requirements, implement efficient solutions, and ensure the reliability and performance of our data infrastructure. The ideal candidate will have a strong background in data engineering, expertise in working with cloud platforms such as Google Cloud Platform (GCP), and experience with modern data warehousing solutions like Snowflake. The Lead Data Engineer will mentor less experienced developers and collaborate with cross-functional teams, stakeholders, and vendors to achieve project milestones and deliver a robust and scalable data lake solution.

What you'll do on a typical day:

• Data Pipeline Design and Development: Lead the design, implementation, and maintenance of data pipelines to support data ingestion, transformation, and storage on GCP and Snowflake.
• Collaboration: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver data solutions that meet business objectives.
• Platform Optimization: Optimize and enhance the performance, scalability, and reliability of existing data pipelines, ensuring efficient data processing and storage.
• Technology Stack: Stay abreast of industry trends and emerging technologies in data engineering and incorporate them into the data architecture where applicable. Experience with the following modern data stack highly preferred: GCP, Snowflake, Fivetran, and dbt.
• Quality Assurance: Implement best practices for data quality, validation, and testing to ensure the accuracy and integrity of data throughout the pipeline.
• Documentation: Create and maintain comprehensive documentation for data pipelines, ensuring knowledge transfer and supportability.

What you need to succeed at GXO:

At a minimum, you'll need:

• Bachelor's degree in Computer Science, Data Science,or equivalent related work
• 5+ in designing and building data pipelines on cloud platforms, preferably GCP and Snowflake.
• Expertise in working with modern data warehousing solutions such as Snowflake, ie, Snowpipe Streaming, Warehouse Optimization, and clustering. Highly preferred, real-time data processing experience using Kafka.
• Python and Advanced SQL: Must be adept at scripting in Python, particularly for data manipulation and integration tasks, and have a solid grasp of advanced SQL techniques for querying, transformation, and performance optimization.
• Data Modeling: Understanding of best practices for data modeling, including star schemas, snowflake schemas, and data normalization techniques. A good understanding of the data structures that support email targeting.
• ETL/ELT Processes: Experience in designing, building, and optimizing ETL/ELT pipelines to process large datasets using dbt.
• Apache Airflow: Experience in building, deploying, and optimizing DAGs in Airflow or a similar tool.
• GitHub: Experience with version control, branching, and collaboration on GitHub.
• Data Visualization: Knowledge of tools like Superset, Looker or Python visualization libraries (Matplotlib, Seaborn, Plotly…etc)
• Collaboration and Communication: Ability to work closely with data scientists, analysts, and other stakeholders to translate business requirements into technical solutions. Strong documentation skills for pipeline design and data flow diagrams.
• Data Privacy: Knowledge of data privacy laws including an understanding of GDPR, CCPA, and other regulations to ensure compliance in data privacy practices.