Do you thrive on simplifying complicated systems and creating structure? We're looking for a data engineer with a passion for designing data platforms and solutions for businesses to access the information they need easily.
You'll have a proven track record in the architecture and optimisation of data systems and building the data pipeline from the bottom up. You must be able to select appropriate technology to match the needs of the business, while taking into account their current applications, standards and assets.
As a senior programmer, you'll have a methodical approach to streamlining data sources and be able to communicate effectively across teams to solve their challenges. In return, we'll offer a competitive rate and some amazing perks, plus we're a fun team to work with, we promise.
- Plan and implement an optimal data pipeline architecture
- Make sense of complex data sets and models to fulfil business requirements
- Identify and implement process improvements, automate manual processes, optimise data delivery, and design the infrastructure for scalability
- Enable optimal extraction, transformation, and loading of data from a range of data sources using SQL and big data technologies
- Build analytics tools from the data pipeline to provide actionable business performance metrics such as for customer acquisition and operational efficiency
- Keep data secure
- Strive for innovative functionality of data systems by working with data and analytics experts
Skills & Competencies:
The successful candidate will have:
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL)
- Programming languages including Python, possibly Node
- Working knowledge of a variety of databases (desirable)
- Experience designing, building and optimizing 'big data' data pipelines, architectures and data sets
- Strong analytic skills related to working with unstructured datasets
- Experience building processes supporting data transformation, data structures, metadata, dependency and workload management
- A successful history of manipulating, processing and extracting value from large disconnected datasets
- Strong project management and organisational skills
- 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another similar field.
- Google Cloud Platform (GCP), AWS, Apache Spark, Hadoop
- Devops, CI/CD and infrastructure as code
- Open source data visualisation platforms such as Plotly, Tableau, Qlikview, Fusioncharts, Sisense, Chartio or similar