Skip to main content

ADR-006 AWS Bedrock

Status

✅ Accepted

Context

Our users want to explore and leverage large language models (LLM) to solve business problems.

Our platform lacks the resources required to run these models.

Decision

  • We will offer Amazon Bedrock to our users. Amazon Bedrock is fully managed large language model platform, which offers many foundation models which be customised privately using techniques such as fine tuning and retrieval-augmented generation (RAG)._

Consequences

General consequences

  • Bedrock provides pre-trained models for generations and embeddings
  • Bedrock pricing is based on usage and can vary significantly month-to-month depending on your application’s traffic and costs could spike unexpectedly. Usage is metered and billed per inference request and based on factors like model used, input length, and response length
  • Bedrock models are accessed via an API using AWS permissions

Advantages

  • Serverless access to large language models meaning that our platform and users don’t need to manage and maintain infrastructure
  • Because this is a fully managed service, the compute is managed by AWS and overcomes resourcing limits that currently constrain the platform

Disadvantages

  • Limited model selection, Bedrock offers a few pre-trained models and new models can take time to reach all AWS regions
  • Service is still in process of rolling out to more regions which means we need to be cognizant of data sovereignty issues
This page was last reviewed on 19 December 2024. It needs to be reviewed again on 19 June 2025 by the page owner #analytical-platform-notifications .