Data Engineer
We are on the lookout for a skilled Data Engineer to become an integral part of our Data Engineering Team. You will not only maintain and optimize our data infrastructure but also spearhead its evolution. Built predominantly on AWS, and utilizing technologies like Pyspark and Iceberg, our infrastructure is designed for scalability, robustness, and efficiency. You will be part of developing sophisticated data integrations with various platforms, developing real-time data solutions, improving automation, and enabling crucial business intelligence.
- Design and Build Data Pipelines: Take ownership of designing, developing, and maintaining robust data pipelines and ETL processes using AWS services such as AWS Glue, AWS Lambda, and AWS S3.
- Contribute to Cloud Migration: Actively participate in and support the migration of our existing Data Warehouse from SQL Server to AWS, with a focus on implementing best practices using S3 and Redshift.
- Ensure Data Quality and Integrity: Develop, implement, and maintain data quality checks and validation procedures to ensure the accuracy and reliability of our data.
- Implement Architectural Solutions: Contribute to the design and implementation of data lakehouse architectures and data warehousing solutions.
- Collaborate and Deliver: Work closely with data scientists and analysts to support the deployment and operationalization of machine learning and advanced analytical solutions.
- Maintain and Optimize: Develop and maintain data documentation and operational procedures, and proactively investigate and resolve data quality issues and performance bottlenecks.
- Stay Current and Innovate: Stay up-to-date with the latest data technologies and industry best practices and apply this knowledge to improve our platform.
- Provide Technical Support: Offer technical guidance and support to junior data engineers and other team members when needed.
- Strong AWS Proficiency: Solid understanding and practical experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. (required)
- Core Technical Skills: Proven proficiency in Python, SQL, Pipeline Orchestration, PySpark and a deep understanding of data warehousing concepts. (required)
- Problem-Solving: Demonstrated ability to independently diagnose and resolve complex data issues and performance bottlenecks. (required)
- Communication: Ability to effectively communicate technical concepts and project updates to both technical and non-technical stakeholders. (required)
- Experience with Data Architectures: Hands-on experience with data lakehouse architectures and data warehousing solutions. (required)
- Agile Experience: Experience with Agile development methodologies. (required)
- Security and Best Practices: A solid understanding of data security and privacy best practices. (required)
- A competitive rate of pay and pension contribution ( £45,000 to £60,000) (required)
- Generous discretionary bonus schemes, incentives and competitions (required)
- An annual leave entitlement that increases with length of service (required)
- Access to an online GP 24/7, 365 days a year for you and your immediate family. (required)
- Employee wellbeing support through our Employee Assistance Programme (required)
- Enhanced Maternity & Paternity Pay (required)
- Long Service Recognition (required)
- Access to a pay day savings scheme, financial coach and up to 40% of your earned wage ahead of payday, through Wagestream. (required)
Betfred is a betting and gaming company that has operated for over 50 years, founded by Fred and Peter Done. It runs more than 1,300 betting shops and employs around 9,500 people, combining a large retail estate with online sports betting and gaming. In 2019 it entered the US sports betting market as Betfred Sportsbook, now operating in several states, and has established a presence in South Africa, where it holds a majority stake in the online betting company LottoStar. Headquartered in Warrington, England, Betfred emphasises customer service and responsible gambling across its operations.
