Skip to main content

Analytical Platform ingestion

The AP offers a number of ways for users and data engineers to move data onto the platform. We collectively refer to these services as “ingestion”.

In general, ingested data will intially be transferred to “landing” S3 buckets, before being moved to “production” buckets from which users can access the data.

  • Data uploader. This is a web application which allows users to upload their own data to S3 without requiring the intervention of date engineers or AP support. Data is registered in Glue and is avaiable to query in Athena or other tooling via access to S3 buckets.
  • Managed pipelines. The are managed by data engineers and transfer data from production operational systems such as NOMIS.
  • Register my data. This is service configured by YAML files in a GitHub repository. It allows users to configure an S3 location on the AP with write permissions, providing them with a way to automate data transfers to the AP.
  • Secure FTP (SFTP). A service managed by the AP team which allows external teams (suppliers) to transfer data from their services to the AP

Ingestion diagram

Ingestion diagram

This page was last reviewed on 17 October 2025. It needs to be reviewed again on 17 April 2026 by the page owner #analytical-platform-notifications .