Randstad, Yacht and Tempo-Team possess an enormous quantity of data: the characteristics of millions of candidates and business data on clients, potentials and our own organisation. Your team, DataHub, is responsible for providing the basis for our data infrastructure. Our data engineers work on a self-service data platform, helping our data make its way from a wide variety of sources to the right place. This in turn enables our Data Scientists and Operational Insights teams to build things like AI matching solutions.
As Data Engineering Tech Lead you take on a lead role within your scrum team. You help ensure the quality of the work done by the team on a technical level and the quality of our data platform, as well as support other data engineers on the team.
You will also be a vital part of designing and developing our cloud-based data platform. Together with the rest of the team you will be involved in the full product development process, from design and implementation, to testing, documentation and automated deployment.
This includes:Building and managing the DataHub frontend, including a data catalog, DataHub users and project management, data subscriptions and moreDeveloping ways to improve self-service data consumption and data publishing including building and managing ETL pipelines in Airflow (responsible for ingesting the data and making it available to users), developing standard ways to deliver data in the DataHub, developing CI/CD pipelines and moreProducing high quality code and reusable components, using containerization, CI/CD and other automation technologiesYou will be responsible for creating a backend for high availability and scalability, while at the same time being easily deployable, manageable and secureHelp define quality standards for the team and share your knowledge
What is our Stack?PythonLinuxECS, DockerCloudFormation, JenkinsAirflowAWS (S3, EC2, ECS, lambda, RDS, Redshift, EMR Spark, Athena, Glue)