Data engineer
Find out what a data engineer in government does and the skills you need to do the job at each level.
Last updated 31 March 2023 — See all updates
Contents
What a data engineer does
A data engineer develops and constructs data products and services, and integrates them into systems and business processes.
Data engineer role levels
There are 4 data engineer role levels, from data engineer to head of data engineering.
The typical responsibilities and skills for each role level are described in the sections below. You can use this to identify the skills you need to progress in your career, or simply to learn more about each role in the Government Digital and Data profession.
1. Data engineer
A data engineer delivers the designs set by more senior members of the data engineering community.
At this role level, you will:
- implement data flows to connect operational systems, data for analytics and business intelligence (BI) systems
- document source-to-target mappings
- re-engineer manual data flows to enable scaling and repeatable use
- support the build of data streaming systems
- write ETL (extract, transform, load) scripts and code to ensure the ETL process performs optimally
- develop business intelligence reports that can be reused
- build accessible data for analysis
This role level is often performed at the Civil Service job grade of:
- EO (Executive Officer)
- HEO (Higher Executive Officer)
Skill | Description |
---|---|
Communicating between the technical and non-technical Level: awareness Awareness is the first of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: awareness Awareness is the first of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: awareness Awareness is the first of 4 ascending skill levels |
You can:
|
Programming and build (data engineering) Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: awareness Awareness is the first of 4 ascending skill levels |
You can:
|
2. Senior data engineer
A senior data engineer designs and leads the implementation of data flows to connect operational systems, data for analytics and business intelligence (BI) systems.
At this role level, you will:
- recognise opportunities to reuse existing data flows
- lead the build of data streaming systems
- optimise the code to ensure processes perform optimally
- lead work on database management
This role level is often performed at the Civil Service job grade of:
- HEO (Higher Executive Officer)
- SEO (Senior Executive Officer)
Skill | Description |
---|---|
Communicating between the technical and non-technical Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Programming and build (data engineering) Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
3. Lead data engineer
A lead data engineer is responsible for the design and implementation of numerous complex data flows to connect operational systems, data for analytics and business intelligence (BI) systems.
At this role level, you will:
- recognise and share opportunities to reuse existing data flows between teams
- be responsible for the build of data-streaming systems
- co-ordinate teams and set best practice and standards
- apply knowledge of systems integration to your work
- champion data engineering across government
This role level is often performed at the Civil Service job grade of:
- SEO (Senior Executive Officer)
- G7 (Grade 7)
Skill | Description |
---|---|
Communicating between the technical and non-technical Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Programming and build (data engineering) Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
4. Head of data engineering
A head of data engineering leads multi-functional delivery teams to deliver robust data services for their department, other government departments and private sector partners.
At this role level, you will:
- inspire best practice for data products and services within your teams
- build data engineering capability by providing technical leadership and career development for the community
- work with other senior team members to identify, plan, develop and deliver data services
This role level is often performed at the Civil Service job grade of:
- G7 (Grade 7)
- G6 (Grade 6)
Skill | Description |
---|---|
Communicating between the technical and non-technical Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: practitioner Practitioner is the third of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Programming and build (data engineering) Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: expert Expert is the fourth of 4 ascending skill levels |
You can:
|
Level: working Working is the second of 4 ascending skill levels |
You can:
|
Roles that share data engineer skills
Role | Shared skills |
---|---|
Data architect | |
Application operations engineer | |
Data governance manager | |
End user computing engineer | |
Infrastructure engineer |
Updates
Published 7 January 2020
Last updated 31 March 2023
31 March 2023
- The ‘data modelling’ skill description has been updated at practitioner level.
30 August 2022
- The ‘communication skills (data)’ skill has been renamed ‘communicating between the technical and non-technical’ to ensure consistency across the DDaT Profession Capability Framework.
7 January 2020
- First published.