data engineer.

Position Objective

As a Data Engineer, you will be responsible for designing, building, and optimizing complex data pipelines that support our data-driven initiatives. You will work closely with cross-functional teams to deliver high-quality, reliable data solutions in a cloud environment. You will work on collecting, storing, processing, and analyzing huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company. Because we work on the cutting edge of a lot of technologies, this job requires a creative problem solver, resourced in getting things done, and productive working independently or collaboratively.

Position Summary



Programming & Code Writing

Design and implementation of scalable data pipelines using GCP services.
Write clean, efficient, modular and well-documented code using python, SQL and other programming languages
Perform performance tuning and optimisation of data pipelines
Follow industry best practices for building data pipelines
Completes work with some independence and some guidance from senior members of the team. Build and implement ETL processes. Modify ETL jobs to improve performance and efficiency
Utilize agile and DevOps methodologies


Consulting & Partnership

Partner with business analysts and other business stakeholders to plan project execution, including appropriate product and technical specifications, direction, and resources, and establishing realistic completion times.
Understand data technology trends and identify opportunities to implement new technologies and provide forward-thinking recommendations
Proactively partner with internal stakeholders to bridge gaps, provide historical references, and design the appropriate processes.
Mentor junior data engineers and provide technical guidance


Troubleshooting & Continuous Improvement

Resolve escalated reporting requests and communicate proactively and timely
Maintain new release, migration, and sprint schedules for software upgrades, enhancements, and fixes to aid with product evolution.
Write QA/QC Scripts to conduct the first round of testing and partner with the BA team for test validation for new developments before moving to production.
Use industry knowledge & feedback to aid in developing a technology roadmap and future product(s) vision.
Document standard ways of working via QRGs, intranet pages, and video series

Experience 2+ yrs of experience on Google cloud platform, Google BigQuery, Google cloud composer (A

Knowledge, Skills and Abilities

Strong data engineering (ETL) experience in any cloud platform (AWS/GCP/Azure). Google cloud platform experience is preferred
Experience with integration of data from multiple data sources utilizing Apache Airflow, Google CloudRun, Cloud dataform, etc.
Strong experience and understanding of databases, data warehouses, data lakes, etc.
Strong knowledge of and experience with database systems (Oracle, PSQL. MySQL, Google BigQuery, Elastic Search etc.)
Advanced knowledge of Python, Java, SQL, and software engineering best practices.
Experience in CICD frameworks
Good understanding and experience with Git
Excellent interpersonal communication skills and intrinsic motivation
Proactive leadership, project management, time management, and problem solving skills
Extensive experience with data analysis , modeling and data pipelining including : data cleaning, standardizing, scaling, tuning , scheduling and deployment
Ability to prioritize and manage multiple projects, tasks, and meeting deadlines while maintaining quality
Strong drive and commitment for delivering outstanding results
Experience with TDD
Effective verbal and written communication skills
Curious and willing to learn.

Education

Bachelor’s degree in Computer Science or related field
Master’s degree a plus

Universidad