Principal AI Engineer

As we rapidly advance our AI capabilities, we need talented AI engineers to build the underlying infrastructure that powers our future large-scale systems. This is an opportunity to architect and implement innovative solutions that will enable our AI models to scale and integrate easily within complex scenarios.

As an AI Engineer, you will develop the core infrastructure that underpins our future AI stack. You will design systems focused on performance, modularity, reliability and cost efficiency. You will collaborate with applied AI engineers, researchers and other technical roles to create optimised, high performing environments for AI workloads.

About the team 

The Incubator for Artificial Intelligence (i.AI) - is an AI for public good programme to harness the opportunities presented by AI to improve the lives of citizens. It will focus on a number of priority projects in home affairs, health, education and government efficiency projects. The i.AI team will use ethical and secure methods at all times to help deliver better public services. 

Building on the work of No10 Data Science Team (10DS) and other leading technology teams across the civil service i.AI will experiment and prove what is possible in improving the use of AI across the government.

You can see more about our work on ai.gov.uk

Job description

Role Responsibilities

• Design and implement a new AI platform architecture from the ground up focused on scalability, modularity and performance.
• Develop infrastructure to support running large scale AI training and inference jobs and distributed model training.
• Create environments optimised for GPU/TPU workloads and AI-specific hardware.
• Work closely with engineers and researchers to improve infrastructure for scalable AI applications.
• Evaluate and experiment with new tools and technologies to improve speed and aid efficiency of the AI stack.

Further information about the role

Do you have a question about the roles or you want to learn more about the team, the team will be holding a meet the team event on Tuesday 28th November 13.00-14.00. You can join on http://meet.google.com/zda-mvke-mvh

Person specification

• Experience in creating and managing HPC clusters across GPU/TPU and with job schedulers like Slurm and Fsx volumes.
• Knowledge of cloud computing platforms, specifically AWS and experience with infrastructure as code tools like Terraform.
• Experience in machine learning for production using Python and related tools (PyTorch, webdataset, torchdata),
• Understanding of large scale model inference, quantization and distributed computing.
• Mindset to build reusable, modular infrastructure and systems focused on scaling.

We welcome a broad range of applicants and encourage you to apply. Strong candidates come from many different backgrounds. Studies show that talented people, especially those from groups underrepresented in their field, are more likely to doubt themselves and feel like an "imposter". Unique perspectives enrich teams, so we urge you to have confidence in your potential contributions.  If aspects of this role resonate with you, please apply. We look forward to your application highlighting your skills and experiences

Behaviours

We'll assess you against these behaviours during the selection process:

• Working Together
• Seeing the Big Picture
• Delivering at Pace
• Making Effective Decisions
• Communicating and Influencing

We only ask for evidence of these behaviours on your application form:

• Working Together

Technical skills

We'll assess you against these technical skills during the selection process:

• Technical assessment at interview will be based on the listed essential criteria and will be confirmed to candidates at shortlisting stage

Benefits

Alongside your salary of £90,000, Cabinet Office contributes £24,300 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

• Learning and development tailored to your role.
• An environment with flexible working options.
• A culture encouraging inclusion and diversity.
• A Civil Service Pension which provides an attractive pension, benefits for dependants and average employer contributions of 27%.
• A minimum of 25 days of paid annual leave, increasing by one day per year up to a maximum of 30.

Things you need to know

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

To submit an application, you will need to provide an up to date CV. We would also like you to submit a 500 word personal statement. Please ensure your CV and personal statement demonstrates how you meet the following criteria:

• Experience in creating and managing HPC clusters across GPU/TPU and with job schedulers like Slurm and Fsx volumes.
• Knowledge of cloud computing platforms, specifically AWS and experience with infrastructure as code tools like Terraform.
• Experience in machine learning for production using Python and related tools (PyTorch, webdataset, torchdata),
• Understanding of large scale model inference, quantization and distributed computing.
• Mindset to build reusable, modular infrastructure and systems focused on scaling.

Should a large number of applications be received, an initial sift may be undertaken using the behaviour, Working Together.

Assessment and interview process

Your application will be reviewed by a panel. If you have been shortlisted, you will be invited to complete a technical assessment.

Technical assessment

Candidates will be sent through a problem and asked to solve it. You will present your solution technical panel for 30 minutes. We are looking to understand your thought process and hands-on experience.

If you have passed the technical assessment, the final stage will be an interview where you will be asked questions on behaviours:

• Working together

• Seeing the Big Picture, 

• Delivering at Pace 

• Making Effective Decisions and 

• Communicating & Influencing

Candidates that do not pass the interview but have demonstrated an acceptable standard may be considered for similar roles at a lower grade.

Depending on the amount of application we get there might be an additional technical assessment.

Expected timeline (subject to change)

Expected sift date – by 18th December 2023

Technical assessment - 1st-8th January 2024

Expected interview date/s – 15th January 2024

Reasonable Adjustment

If a person with disabilities is put at a substantial disadvantage compared to a non-disabled person, we have a duty to make reasonable changes to our processes. 

If you need a change to be made so that you can make your application, you should: 

• Contact Government Recruitment Service via cabinetofficerecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs. 
• Complete the ‘Assistance required’ section in the ‘Additional requirements’ page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you’re deaf, a Language Service Professional.

Further information

If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'contact point for applicants' section.

Please note that this role requires SC clearance, which would normally need 5 years’ UK residency in the past 5 years. This is not an absolute requirement, but supplementary checks may be needed where individuals have not lived in the UK for that period. This may mean your security clearance (and therefore your appointment) will take longer or, in some cases, not be possible.

Please note terms and conditions are attached. Please take time to read the document to determine how these may affect you.

Any move to Cabinet Office from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility at: http://www.childcarechoices.gov.uk.

A reserve list will be held for a period of 12 months, from which further appointments can be made.

If successful and transferring from another Government Department a criminal record check may be carried out.

In order to process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service/Disclosure Scotland on your behalf.

However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading. 

For further information on the Disclosure Scotland confidential checking service telephone: the Disclosure Scotland Helpline on 0870 609 6006 and ask to speak to the operations manager in confidence, or email Info@disclosurescotland.co.uk

New entrants are expected to join on the minimum of the pay band.

Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment. 

A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government.

Existing Civil Servants and applicants from accredited NDPBs are eligible to apply, but will only be considered on loan basis (Civil Servants) or secondment (accredited NDPBs). Prior agreement to be released on a loan basis must be obtained before commencing the application process. In the case of Civil Servants, the terms of the loan will be agreed between the home and host department and the Civil Servant. This includes grade on return.

Please note, these roles are for Cabinet Office. The GDS Recruitment team is supporting with the recruitment process.

Feedback will only be provided if you attend an interview or assessment.