Role Description: Senior Data Engineer
W1 London - United Kingdom
Agilisys Data & Decisions
The way we work reflects the values at the heart of our business:
Partnership, we become one team and family with organisations, helping them to navigate change and stay agile.
Integrity, our people really care, going beyond the brief to make change happen for organisations and citizens.
Innovation, we bring together the right technologies and services to design solutions that work.
Passion, we are passionate about — and dedicated to — public services and improving people’s lives.
In your role as a Senior Data Engineer, you will be at the leading edge of data and insight transformation in local government and the wider public sector.
Agilisys has set a clear ambition to become the number one provider of data and insight services to the UK public sector. Our rapidly expanding team blends huge knowledge of data and insight with a deep understanding of public sector clients’ needs and values. Collectively, our team has many decades of experience working in and with the sector, and we have united that knowledge and affinity with best-in-class data architecture, platform engineering and business intelligence. We lead with people and culture, because this is what leads to lasting change – not least because only an organisation that can use data effectively is able to unlock the promise of technology.
You will be responsible for building, delivering, expanding and optimising our suite of data platform and engineering services. You will lead on data development and management, and will support our data architects to design, build, optimise and manage data platforms, and to set up and configure components within client and internal architectures.
This role will involve working with data architects, developers, and data analysts on data initiatives, and will be central to the design and deployment of data services to our clients.
Whatever your core skills, we will develop your capabilities to allow you to provide deep support to clients in these areas to enable them to meet the future expectations of their customer/citizens particularly in a more digital world.You will be self-directed and comfortable supporting the needs of multiple teams, systems and products.
This is a full-time role based in our London Scale Space office.
The key responsibilities are:
- Assemble large, complex data sets that meet functional/non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, where necessary supporting the redesign of infrastructure for greater scalability, etc.
- Help to build infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure 'big data'
- Build analytics tools that utilise the data pipeline to provide actionable insights into financial management, operational efficiency, and other key business performance metrics.
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data platform infrastructure needs.
- Keep our data separated and secure across national boundaries through multiple cloud regions, while safeguarding our clients' data at rest and in transit.
- Develop deep expertise in typical client data-related activities such as statutory returns, subject access requests and the use of standard reporting suites, using your knowledge to structure plans and solutions for our clients.
- Analyse the quality of data, establishing rules to determine and improve it, and planning, delivering and overseeing cleansing activities as required.
- Plan and undertake quantitative research to baseline, track and evaluate services and the impact of proposed and actual change.
- Produce briefing notes, reports and other written outputs relating to the content, structure and quality of data.
- Keep our data separated and secure across national boundaries through multiple cloud regions.
- Create data tools for analytics and data scientist team members that assist them in building and optimising our full suite of data and insight products and services.
- Work with data and analytics experts to strive for greater functionality in our data platform.
The role holder will have ongoing, general objectives, alongside other specific objectives that will be set from time to time.
- We are seeking to strengthen our team with people who are passionate about data, and who want to help use data to improve public services. Your overall objective will be to help us build and strengthen a practice whose guiding mission is to help public sector organisations to find, make sense of, and use their data to better serve citizens and the places where they live.
- Colleagues at Senior Data Engineer level will be able to undertake day-to-day management of technical delivery for small and medium-sized projects, and to directly manage those projects on occasion.
- Our Senior Data Engineers help us to develop relationships with clients, partners, staff and associates, and focus on growing their abilities in translating clients' business requirements into technical work programmes.
- Our Senior Data Engineers also identify new opportunities to solve client's future challenges and help to contribute to profitability and strong governance on our projects.
- You will, like all of our staff, help to support the Agilisys Data & Insight practice, helping to develop our sales, delivery, and knowledge processes, writing and presenting content to help raise our profile, and modelling best practice and the company's values.
- As a Senior Data Engineer, there is also the opportunity for further progression to gain further expertise and/or more complex technical project management experience.
What we are looking for
- Desire to learn and always be better than yesterday.
- Objective and effective listening.
- Creative results-oriented problem solving.
- Articulate and clear in communications to mixed audiences- in writing, through presentations and one to one.
- Significant experience working with, or within, the public sector.
- Expertise in one or more areas of data and insight, such as analytics, data strategy, data science, data governance and management, research and evaluation, or information governance.
Very good looks like (Data Engineer)
· Examples of working in a team, or specific workstreams within wider projects or programmes
· Mentoring of more junior staff
· Experience of delivering projects or services with a significant data engineering element, including data management, transformation, cloud services, relational databases, and/or scripting languages
· Experience performing root cause analysis on internal and external data quality issues and processes, to answer specific business questions and identify opportunities for improvement
· Experience of providing support to data transformation, data structures, metadata and/or other data services and engineering areas
· Preferably, experience of supporting big data or data science projects or capabilities
· Preferably, experience of working in the public sector, or demonstrable interest in doing so
Skills, qualifications, certificates
· Proven analytical skills in working with structured and unstructured datasets, manipulating data to extract and visualise insight, using primarily Excel and Power BI
· Experience working with cloud data services, primarily Azure
· Working knowledge of SQL and experience working with relational databases and query authoring (such as T-SQL)
· Experience with Azure PaaS, Azure Synapse, Power BI, DataBricks, data lakes, Azure Data Factory and/or Polybase
· Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
· Experience with AWS cloud services: EC2, EMR, RDS, Redshift.
· Role model for company values
· Good understanding of, and engagement with, others – ideally including both
close teammates and more distant colleagues, clients and/or external stakeholders