vitalibo / cetus

Cost-effective REST API for dense and slowly changing data

Cetus

Cetus is a high-performance, low-latency, global scale and cost-effective REST API for dense and slowly changing data. The core idea revolves around precomputing responses and leveraging cloud storage (S3) and CDN (CloudFront) to serve these responses, thus avoiding the need for always-on web servers, databases, and caches.

Problem Statement

Let's assume we have a dataset that updates once a day, and we want to expose this data via a REST API that can handle high traffic globally. The traditional approach would be to have a web server that queries the database and caches the response. This approach has one major drawback: the web server must always be on, even if the data is updated once a day. This leads to high costs and operational complexity.

Initial Idea

1. Once a day, we precompute using Apache Spark batch processing the responses and store them in Amazon S3. File path in S3 is determined by the request URL, and the file content is the response body.
2. We use Amazon CloudFront to serve these responses globally with low latency and high availability. Amazon CloudFront caches the responses based on the request URL and the response body.

Challenges

Challenge 1: a large number of files in S3

Initially, each possible response was stored in an individual file. This led to a large number of small files, resulting in longer write times. To address this, we grouped responses into larger files. The content within each file is organized in such a way that allows for efficient indexing. Each response is stored with a fixed length, enabling easy calculation of the offset for any given dimensions.

Challenge 2: efficient response lookup

To efficiently look up responses, we need to determine the file and offset for a given request URL. We use a Lambda@Edge function to parse the request URL and determine the file and offset, then use S3 byte-range requests to retrieve the response body. The Lambda@Edge function is deployed globally, ensuring low latency for all requests.

Final Implementation

1. Once a day, Apache Spark batch processing precomputes responses and stores them in S3. Responses are grouped into larger files to reduce write times. Each response is stored with a fixed length to enable efficient indexing. The file path in S3 partially reflects the request URL, other parts are stored in the file content.
2. Amazon CloudFront serves responses globally with low latency and high availability. Before retrieving a response, CloudFront triggers a Lambda@Edge function to determine the file and offset. The Lambda@Edge function parses the request URL and calculates the file and offset, then uses S3 byte-range requests to retrieve the response body.

SVG code

@diagram.svg
<?xml version="1.0" encoding="UTF-8"?>
<!-- Do not edit this file with editors other than draw.io -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1009px" height="400px" viewBox="-0.5 -0.5 1009 400" content="&lt;mxfile host=&quot;app.diagrams.net&quot; agent=&quot;Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.1 Safari/605.1.15&quot; version=&quot;24.7.5&quot;&gt;&#10;  &lt;diagram name=&quot;Page-1&quot; id=&quot;nS3I_m_30D9WI8xmdrnn&quot;&gt;&#10;    &lt;mxGraphModel dx=&quot;1434&quot; dy=&quot;924&quot; grid=&quot;1&quot; gridSize=&quot;10&quot; guides=&quot;1&quot; tooltips=&quot;1&quot; connect=&quot;1&quot; arrows=&quot;1&quot; fold=&quot;1&quot; page=&quot;1&quot; pageScale=&quot;1&quot; pageWidth=&quot;827&quot; pageHeight=&quot;1169&quot; math=&quot;0&quot; shadow=&quot;0&quot;&gt;&#10;      &lt;root&gt;&#10;        &lt;mxCell id=&quot;0&quot; /&gt;&#10;        &lt;mxCell id=&quot;1&quot; parent=&quot;0&quot; /&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-5&quot; value=&quot;&quot; style=&quot;fillColor=none;strokeColor=#5A6C86;dashed=1;verticalAlign=top;fontStyle=0;fontColor=#5A6C86;whiteSpace=wrap;html=1;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;510&quot; y=&quot;180&quot; width=&quot;340&quot; height=&quot;370&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-13&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;&quot; edge=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;280&quot; y=&quot;350&quot; as=&quot;targetPoint&quot; /&gt;&#10;            &lt;mxPoint x=&quot;397&quot; y=&quot;300&quot; as=&quot;sourcePoint&quot; /&gt;&#10;            &lt;Array as=&quot;points&quot;&gt;&#10;              &lt;mxPoint x=&quot;397&quot; y=&quot;350&quot; /&gt;&#10;              &lt;mxPoint x=&quot;280&quot; y=&quot;350&quot; /&gt;&#10;            &lt;/Array&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-14&quot; value=&quot;Cache hit&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-13&quot;&gt;&#10;          &lt;mxGeometry x=&quot;-0.4745&quot; y=&quot;1&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;-51&quot; y=&quot;-11&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-2&quot; value=&quot;CloudFront&quot; style=&quot;sketch=0;points=[[0,0,0],[0.25,0,0],[0.5,0,0],[0.75,0,0],[1,0,0],[0,1,0],[0.25,1,0],[0.5,1,0],[0.75,1,0],[1,1,0],[0,0.25,0],[0,0.5,0],[0,0.75,0],[1,0.25,0],[1,0.5,0],[1,0.75,0]];outlineConnect=0;fontColor=#232F3E;fillColor=#8C4FFF;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.cloudfront;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;358&quot; y=&quot;200&quot; width=&quot;78&quot; height=&quot;78&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-37&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;RVEh1LkQJVqgOGLHUIks-3&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;280&quot; y=&quot;519&quot; as=&quot;targetPoint&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-38&quot; value=&quot;Return response&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-37&quot;&gt;&#10;          &lt;mxGeometry x=&quot;0.0689&quot; y=&quot;4&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;-86&quot; y=&quot;-4&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-3&quot; value=&quot;CloudFront&quot; style=&quot;sketch=0;points=[[0,0,0],[0.25,0,0],[0.5,0,0],[0.75,0,0],[1,0,0],[0,1,0],[0.25,1,0],[0.5,1,0],[0.75,1,0],[1,1,0],[0,0.25,0],[0,0.5,0],[0,0.75,0],[1,0.25,0],[1,0.5,0],[1,0.75,0]];outlineConnect=0;fontColor=#232F3E;fillColor=#8C4FFF;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.cloudfront;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;358&quot; y=&quot;480&quot; width=&quot;78&quot; height=&quot;78&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-34&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;RVEh1LkQJVqgOGLHUIks-4&quot; target=&quot;RVEh1LkQJVqgOGLHUIks-27&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-4&quot; value=&quot;&quot; style=&quot;sketch=0;points=[[0,0,0],[0.25,0,0],[0.5,0,0],[0.75,0,0],[1,0,0],[0,1,0],[0.25,1,0],[0.5,1,0],[0.75,1,0],[1,1,0],[0,0.25,0],[0,0.5,0],[0,0.75,0],[1,0.25,0],[1,0.5,0],[1,0.75,0]];outlineConnect=0;fontColor=#232F3E;fillColor=#ED7100;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.lambda;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;540&quot; y=&quot;200&quot; width=&quot;78&quot; height=&quot;78&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-32&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=1;entryY=0.5;entryDx=0;entryDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; target=&quot;RVEh1LkQJVqgOGLHUIks-31&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;949&quot; y=&quot;430&quot; as=&quot;sourcePoint&quot; /&gt;&#10;            &lt;Array as=&quot;points&quot;&gt;&#10;              &lt;mxPoint x=&quot;949&quot; y=&quot;450&quot; /&gt;&#10;              &lt;mxPoint x=&quot;820&quot; y=&quot;450&quot; /&gt;&#10;            &lt;/Array&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-40&quot; value=&quot;Bytes&amp;lt;div&amp;gt;returned&amp;lt;/div&amp;gt;&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-32&quot;&gt;&#10;          &lt;mxGeometry x=&quot;0.3377&quot; y=&quot;4&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;31&quot; y=&quot;16&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-6&quot; value=&quot;S3&quot; style=&quot;sketch=0;points=[[0,0,0],[0.25,0,0],[0.5,0,0],[0.75,0,0],[1,0,0],[0,1,0],[0.25,1,0],[0.5,1,0],[0.75,1,0],[1,1,0],[0,0.25,0],[0,0.5,0],[0,0.75,0],[1,0.25,0],[1,0.5,0],[1,0.75,0]];outlineConnect=0;fontColor=#232F3E;fillColor=#7AA116;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.s3;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;910&quot; y=&quot;330&quot; width=&quot;78&quot; height=&quot;78&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-7&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;entryPerimeter=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;RVEh1LkQJVqgOGLHUIks-2&quot; target=&quot;RVEh1LkQJVqgOGLHUIks-4&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-8&quot; value=&quot;Cache&amp;lt;div&amp;gt;miss&amp;lt;/div&amp;gt;&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-7&quot;&gt;&#10;          &lt;mxGeometry x=&quot;-0.3269&quot; y=&quot;1&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;-1&quot; y=&quot;-18&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-11&quot; value=&quot;&quot; style=&quot;endArrow=classic;html=1;rounded=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;entryPerimeter=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; target=&quot;RVEh1LkQJVqgOGLHUIks-2&quot;&gt;&#10;          &lt;mxGeometry width=&quot;50&quot; height=&quot;50&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;280&quot; y=&quot;239&quot; as=&quot;sourcePoint&quot; /&gt;&#10;            &lt;mxPoint x=&quot;230&quot; y=&quot;240&quot; as=&quot;targetPoint&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-12&quot; value=&quot;Client request&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-11&quot;&gt;&#10;          &lt;mxGeometry x=&quot;-0.4103&quot; y=&quot;3&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;-73&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-15&quot; value=&quot;Return cached&amp;lt;div&amp;gt;response&amp;lt;/div&amp;gt;&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;230&quot; y=&quot;350&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-27&quot; value=&quot;Calculate response&amp;lt;div&amp;gt;offset in file&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&quot; style=&quot;rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;680&quot; y=&quot;278&quot; width=&quot;140&quot; height=&quot;60&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-30&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;entryPerimeter=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;RVEh1LkQJVqgOGLHUIks-27&quot; target=&quot;RVEh1LkQJVqgOGLHUIks-6&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-41&quot; value=&quot;Bytes-range&amp;lt;div&amp;gt;request&amp;lt;/div&amp;gt;&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-30&quot;&gt;&#10;          &lt;mxGeometry x=&quot;-0.5629&quot; y=&quot;1&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;47&quot; y=&quot;-17&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-36&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;RVEh1LkQJVqgOGLHUIks-31&quot; target=&quot;RVEh1LkQJVqgOGLHUIks-3&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-39&quot; value=&quot;Return response to CloudFront&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-36&quot;&gt;&#10;          &lt;mxGeometry x=&quot;0.4844&quot; y=&quot;-1&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;83&quot; y=&quot;-18&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-31&quot; value=&quot;&amp;lt;div&amp;gt;Trims any padding&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&quot; style=&quot;rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;680&quot; y=&quot;420&quot; width=&quot;140&quot; height=&quot;60&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-44&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;RVEh1LkQJVqgOGLHUIks-43&quot; target=&quot;RVEh1LkQJVqgOGLHUIks-6&quot;&gt;&#10;          &lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-45&quot; value=&quot;Write files&quot; style=&quot;edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];&quot; vertex=&quot;1&quot; connectable=&quot;0&quot; parent=&quot;RVEh1LkQJVqgOGLHUIks-44&quot;&gt;&#10;          &lt;mxGeometry x=&quot;-0.2674&quot; y=&quot;1&quot; relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&#10;            &lt;mxPoint x=&quot;-14&quot; y=&quot;-15&quot; as=&quot;offset&quot; /&gt;&#10;          &lt;/mxGeometry&gt;&#10;        &lt;/mxCell&gt;&#10;        &lt;mxCell id=&quot;RVEh1LkQJVqgOGLHUIks-43&quot; value=&quot;Glue Job&quot; style=&quot;sketch=0;points=[[0,0,0],[0.25,0,0],[0.5,0,0],[0.75,0,0],[1,0,0],[0,1,0],[0.25,1,0],[0.5,1,0],[0.75,1,0],[1,1,0],[0,0.25,0],[0,0.5,0],[0,0.75,0],[1,0.25,0],[1,0.5,0],[1,0.75,0]];outlineConnect=0;fontColor=#232F3E;fillColor=#8C4FFF;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.glue;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&#10;          &lt;mxGeometry x=&quot;1120&quot; y=&quot;330&quot; width=&quot;78&quot; height=&quot;78&quot; as=&quot;geometry&quot; /&gt;&#10;        &lt;/mxCell&gt;&#10;      &lt;/root&gt;&#10;    &lt;/mxGraphModel&gt;&#10;  &lt;/diagram&gt;&#10;&lt;/mxfile&gt;&#10;"><defs/><g><g data-cell-id="0"><g data-cell-id="1"><g data-cell-id="RVEh1LkQJVqgOGLHUIks-5"><g><rect x="319" y="0" width="340" height="370" fill="none" stroke="#5a6c86" stroke-dasharray="3 3" pointer-events="all"/></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-13"><g><path d="M 206 120 L 206 170 L 95.37 170" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 90.12 170 L 97.12 166.5 L 95.37 170 L 97.12 173.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-14"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 154px; margin-left: 157px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Cache hit</div></div></div></foreignObject><text x="157" y="157" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Cache hit</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-2"><g><path d="M 167 20 L 245 20 L 245 98 L 167 98 Z" fill="#8c4fff" stroke="none" pointer-events="all"/><path d="M 222.85 72.51 C 222.85 70.67 221.35 69.17 219.51 69.17 C 217.66 69.17 216.16 70.67 216.16 72.51 C 216.16 74.35 217.66 75.85 219.51 75.85 C 221.35 75.85 222.85 74.35 222.85 72.51 Z M 225.08 72.51 C 225.08 75.58 222.58 78.08 219.51 78.08 C 216.44 78.08 213.94 75.58 213.94 72.51 C 213.94 69.44 216.44 66.94 219.51 66.94 C 222.58 66.94 225.08 69.44 225.08 72.51 Z M 195.29 56.43 C 195.29 54.59 193.79 53.09 191.95 53.09 C 190.11 53.09 188.61 54.59 188.61 56.43 C 188.61 58.27 190.11 59.77 191.95 59.77 C 193.79 59.77 195.29 58.27 195.29 56.43 Z M 197.52 56.43 C 197.52 59.5 195.02 62 191.95 62 C 188.88 62 186.38 59.5 186.38 56.43 C 186.38 53.36 188.88 50.86 191.95 50.86 C 195.02 50.86 197.52 53.36 197.52 56.43 Z M 205.7 38.95 C 205.7 40.79 207.2 42.29 209.04 42.29 C 210.89 42.29 212.38 40.79 212.38 38.95 C 212.38 37.11 210.89 35.61 209.04 35.61 C 207.2 35.61 205.7 37.11 205.7 38.95 Z M 203.47 38.95 C 203.47 35.88 205.97 33.38 209.04 33.38 C 212.11 33.38 214.61 35.88 214.61 38.95 C 214.61 42.02 212.11 44.52 209.04 44.52 C 205.97 44.52 203.47 42.02 203.47 38.95 Z M 234.97 59 C 234.97 48.67 229.43 39.11 220.5 33.94 C 218.9 34.26 217.35 34.7 215.42 35.4 L 214.66 33.31 C 215.67 32.94 216.57 32.65 217.44 32.4 C 213.85 30.85 209.95 30.03 206 30.03 C 204.12 30.03 202.27 30.22 200.46 30.58 C 201.77 31.34 202.93 32.11 204.06 32.96 L 202.71 34.74 C 201.12 33.53 199.47 32.51 197.31 31.38 C 186.36 34.82 178.48 44.47 177.23 55.8 C 179.53 55.34 181.73 55.08 184.18 55.02 L 184.23 57.25 C 181.67 57.31 179.47 57.58 177.06 58.11 C 177.05 58.41 177.03 58.71 177.03 59 C 177.03 68.65 181.8 77.54 189.65 82.9 C 188.25 78.74 187.56 74.81 187.56 70.97 C 187.56 68.79 187.94 66.99 188.33 65.09 C 188.43 64.65 188.52 64.2 188.61 63.73 L 190.8 64.16 C 190.7 64.64 190.61 65.1 190.51 65.54 C 190.12 67.41 189.79 69.02 189.79 70.97 C 189.79 75.33 190.74 79.83 192.69 84.72 C 196.83 86.87 201.31 87.97 206 87.97 C 209.07 87.97 212.07 87.48 214.93 86.54 C 216.05 84.32 216.88 82.24 217.58 79.84 L 219.72 80.46 C 219.21 82.21 218.63 83.8 217.93 85.38 C 219.73 84.57 221.43 83.57 223.03 82.41 C 222.65 81.46 222.23 80.53 221.78 79.61 L 223.77 78.62 C 224.16 79.39 224.51 80.18 224.85 80.98 C 231.3 75.46 234.97 67.54 234.97 59 Z M 237.2 59 C 237.2 68.72 232.78 77.72 225.08 83.68 C 223.17 85.16 221.1 86.39 218.91 87.38 C 217.99 87.8 217.04 88.19 216.07 88.52 C 212.86 89.64 209.47 90.2 206 90.2 C 200.87 90.2 195.78 88.92 191.28 86.51 C 181.11 81.07 174.8 70.53 174.8 59 C 174.8 58.24 174.82 57.65 174.87 57.12 C 175.62 44.1 184.56 32.83 197.09 29.1 C 199.94 28.24 202.94 27.8 206 27.8 C 211.36 27.8 216.63 29.18 221.25 31.79 C 231.09 37.3 237.2 47.73 237.2 59 Z M 203.01 42.8 L 201.54 41.12 C 199.04 43.3 197.1 45.62 194.83 49.11 L 196.7 50.33 C 198.85 47.01 200.68 44.84 203.01 42.8 Z M 199.62 57.25 L 198.9 59.35 C 204.01 61.11 208.48 63.92 212.95 68.18 L 214.49 66.56 C 209.78 62.07 205.05 59.11 199.62 57.25 Z M 214.52 44.14 C 218.7 50.52 221.06 57.52 221.52 64.95 L 219.29 65.09 C 218.85 58.05 216.62 51.41 212.66 45.37 Z" fill="#ffffff" stroke="none" pointer-events="all"/></g><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe flex-start; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 105px; margin-left: 206px;"><div data-drawio-colors="color: #232F3E; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(35, 47, 62); line-height: 1.2; pointer-events: all; white-space: nowrap;">CloudFront</div></div></div></foreignObject><text x="206" y="117" fill="#232F3E" font-family="&quot;Helvetica&quot;" font-size="12px" text-anchor="middle">CloudFront</text></switch></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-37"><g><path d="M 167 339 L 128.5 339 L 95.37 339" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 90.12 339 L 97.12 335.5 L 95.37 339 L 97.12 342.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-38"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 340px; margin-left: 40px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Return response</div></div></div></foreignObject><text x="40" y="343" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Return response</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-3"><g><path d="M 167 300 L 245 300 L 245 378 L 167 378 Z" fill="#8c4fff" stroke="none" pointer-events="all"/><path d="M 222.85 352.51 C 222.85 350.67 221.35 349.17 219.51 349.17 C 217.66 349.17 216.16 350.67 216.16 352.51 C 216.16 354.35 217.66 355.85 219.51 355.85 C 221.35 355.85 222.85 354.35 222.85 352.51 Z M 225.08 352.51 C 225.08 355.58 222.58 358.08 219.51 358.08 C 216.44 358.08 213.94 355.58 213.94 352.51 C 213.94 349.44 216.44 346.94 219.51 346.94 C 222.58 346.94 225.08 349.44 225.08 352.51 Z M 195.29 336.43 C 195.29 334.59 193.79 333.09 191.95 333.09 C 190.11 333.09 188.61 334.59 188.61 336.43 C 188.61 338.27 190.11 339.77 191.95 339.77 C 193.79 339.77 195.29 338.27 195.29 336.43 Z M 197.52 336.43 C 197.52 339.5 195.02 342 191.95 342 C 188.88 342 186.38 339.5 186.38 336.43 C 186.38 333.36 188.88 330.86 191.95 330.86 C 195.02 330.86 197.52 333.36 197.52 336.43 Z M 205.7 318.95 C 205.7 320.79 207.2 322.29 209.04 322.29 C 210.89 322.29 212.38 320.79 212.38 318.95 C 212.38 317.11 210.89 315.61 209.04 315.61 C 207.2 315.61 205.7 317.11 205.7 318.95 Z M 203.47 318.95 C 203.47 315.88 205.97 313.38 209.04 313.38 C 212.11 313.38 214.61 315.88 214.61 318.95 C 214.61 322.02 212.11 324.52 209.04 324.52 C 205.97 324.52 203.47 322.02 203.47 318.95 Z M 234.97 339 C 234.97 328.67 229.43 319.11 220.5 313.94 C 218.9 314.26 217.35 314.7 215.42 315.4 L 214.66 313.31 C 215.67 312.94 216.57 312.65 217.44 312.4 C 213.85 310.85 209.95 310.03 206 310.03 C 204.12 310.03 202.27 310.22 200.46 310.58 C 201.77 311.34 202.93 312.11 204.06 312.96 L 202.71 314.74 C 201.12 313.53 199.47 312.51 197.31 311.38 C 186.36 314.82 178.48 324.47 177.23 335.8 C 179.53 335.34 181.73 335.08 184.18 335.02 L 184.23 337.25 C 181.67 337.31 179.47 337.58 177.06 338.11 C 177.05 338.41 177.03 338.71 177.03 339 C 177.03 348.65 181.8 357.54 189.65 362.9 C 188.25 358.74 187.56 354.81 187.56 350.97 C 187.56 348.79 187.94 346.99 188.33 345.09 C 188.43 344.65 188.52 344.2 188.61 343.73 L 190.8 344.16 C 190.7 344.64 190.61 345.1 190.51 345.54 C 190.12 347.41 189.79 349.02 189.79 350.97 C 189.79 355.33 190.74 359.83 192.69 364.72 C 196.83 366.87 201.31 367.97 206 367.97 C 209.07 367.97 212.07 367.48 214.93 366.54 C 216.05 364.32 216.88 362.24 217.58 359.84 L 219.72 360.46 C 219.21 362.21 218.63 363.8 217.93 365.38 C 219.73 364.57 221.43 363.57 223.03 362.41 C 222.65 361.46 222.23 360.53 221.78 359.61 L 223.77 358.62 C 224.16 359.39 224.51 360.18 224.85 360.98 C 231.3 355.46 234.97 347.54 234.97 339 Z M 237.2 339 C 237.2 348.72 232.78 357.72 225.08 363.68 C 223.17 365.16 221.1 366.39 218.91 367.38 C 217.99 367.8 217.04 368.19 216.07 368.52 C 212.86 369.64 209.47 370.2 206 370.2 C 200.87 370.2 195.78 368.92 191.28 366.51 C 181.11 361.07 174.8 350.53 174.8 339 C 174.8 338.24 174.82 337.65 174.87 337.12 C 175.62 324.1 184.56 312.83 197.09 309.1 C 199.94 308.24 202.94 307.8 206 307.8 C 211.36 307.8 216.63 309.18 221.25 311.79 C 231.09 317.3 237.2 327.73 237.2 339 Z M 203.01 322.8 L 201.54 321.12 C 199.04 323.3 197.1 325.62 194.83 329.11 L 196.7 330.33 C 198.85 327.01 200.68 324.84 203.01 322.8 Z M 199.62 337.25 L 198.9 339.35 C 204.01 341.11 208.48 343.92 212.95 348.18 L 214.49 346.56 C 209.78 342.07 205.05 339.11 199.62 337.25 Z M 214.52 324.14 C 218.7 330.52 221.06 337.52 221.52 344.95 L 219.29 345.09 C 218.85 338.05 216.62 331.41 212.66 325.37 Z" fill="#ffffff" stroke="none" pointer-events="all"/></g><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe flex-start; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 385px; margin-left: 206px;"><div data-drawio-colors="color: #232F3E; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(35, 47, 62); line-height: 1.2; pointer-events: all; white-space: nowrap;">CloudFront</div></div></div></foreignObject><text x="206" y="397" fill="#232F3E" font-family="&quot;Helvetica&quot;" font-size="12px" text-anchor="middle">CloudFront</text></switch></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-34"><g><path d="M 427 59 L 559 59 L 559 91.63" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 559 96.88 L 555.5 89.88 L 559 91.63 L 562.5 89.88 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-4"><g><path d="M 349 20 L 427 20 L 427 98 L 349 98 Z" fill="#ed7100" stroke="none" pointer-events="all"/><path d="M 374.67 87.97 L 360.83 87.97 L 376.13 55.99 L 383.06 70.26 Z M 377.13 52.93 C 376.94 52.55 376.55 52.3 376.13 52.3 L 376.12 52.3 C 375.69 52.3 375.3 52.55 375.12 52.94 L 358.05 88.61 C 357.89 88.95 357.91 89.36 358.11 89.68 C 358.32 90 358.68 90.2 359.06 90.2 L 375.37 90.2 C 375.8 90.2 376.2 89.95 376.38 89.56 L 385.31 70.73 C 385.46 70.43 385.46 70.07 385.31 69.77 Z M 415.88 87.97 L 402.13 87.97 L 380.08 41.81 C 379.9 41.42 379.5 41.17 379.07 41.17 L 370.08 41.17 L 370.09 30.03 L 387.72 30.03 L 409.67 76.19 C 409.85 76.58 410.25 76.83 410.68 76.83 L 415.88 76.83 Z M 417 74.6 L 411.38 74.6 L 389.43 28.44 C 389.25 28.05 388.85 27.8 388.42 27.8 L 368.97 27.8 C 368.36 27.8 367.86 28.3 367.85 28.91 L 367.84 42.28 C 367.84 42.58 367.96 42.86 368.17 43.07 C 368.38 43.28 368.66 43.4 368.96 43.4 L 378.37 43.4 L 400.42 89.56 C 400.61 89.95 401 90.2 401.43 90.2 L 417 90.2 C 417.61 90.2 418.11 89.7 418.11 89.09 L 418.11 75.71 C 418.11 75.1 417.61 74.6 417 74.6 Z" fill="#ffffff" stroke="none" pointer-events="all"/></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-32"><g><path d="M 758 250 L 758 270 L 635.37 270" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 630.12 270 L 637.12 266.5 L 635.37 270 L 637.12 273.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-40"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 291px; margin-left: 710px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Bytes<div>returned</div></div></div></div></foreignObject><text x="710" y="294" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Bytes...</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-6"><g><path d="M 719 150 L 797 150 L 797 228 L 719 228 Z" fill="#7aa116" stroke="none" pointer-events="all"/><path d="M 781.28 192.22 L 781.71 189.21 C 785.66 191.57 785.71 192.55 785.71 192.58 C 785.7 192.58 785.03 193.14 781.28 192.22 Z M 779.12 191.62 C 772.3 189.56 762.8 185.2 758.96 183.39 C 758.96 183.37 758.96 183.36 758.96 183.34 C 758.96 181.86 757.76 180.66 756.28 180.66 C 754.81 180.66 753.61 181.86 753.61 183.34 C 753.61 184.82 754.81 186.02 756.28 186.02 C 756.93 186.02 757.52 185.78 757.98 185.39 C 762.51 187.53 771.93 191.82 778.8 193.85 L 776.08 213.03 C 776.07 213.08 776.07 213.13 776.07 213.18 C 776.07 214.87 768.6 217.97 756.39 217.97 C 744.05 217.97 736.49 214.87 736.49 213.18 C 736.49 213.13 736.49 213.08 736.48 213.03 L 730.81 171.57 C 735.72 174.95 746.29 176.74 756.39 176.74 C 766.48 176.74 777.03 174.96 781.96 171.59 Z M 730.21 167.25 C 730.29 165.78 738.71 160.03 756.39 160.03 C 774.07 160.03 782.5 165.78 782.58 167.25 L 782.58 167.75 C 781.61 171.04 770.69 174.51 756.39 174.51 C 742.07 174.51 731.15 171.02 730.21 167.73 Z M 784.81 167.27 C 784.81 163.41 773.74 157.8 756.39 157.8 C 739.05 157.8 727.98 163.41 727.98 167.27 L 728.08 168.11 L 734.27 213.27 C 734.41 218.32 747.88 220.2 756.39 220.2 C 766.94 220.2 778.15 217.77 778.3 213.27 L 780.97 194.44 C 782.45 194.8 783.68 194.98 784.66 194.98 C 785.98 194.98 786.87 194.66 787.41 194.01 C 787.85 193.49 788.02 192.85 787.89 192.16 C 787.61 190.62 785.77 188.96 782.05 186.83 L 784.69 168.16 Z" fill="#ffffff" stroke="none" pointer-events="all"/></g><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe flex-start; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 235px; margin-left: 758px;"><div data-drawio-colors="color: #232F3E; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(35, 47, 62); line-height: 1.2; pointer-events: all; white-space: nowrap;">S3</div></div></div></foreignObject><text x="758" y="247" fill="#232F3E" font-family="&quot;Helvetica&quot;" font-size="12px" text-anchor="middle">S3</text></switch></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-7"><g><path d="M 245 59 L 342.63 59" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 347.88 59 L 340.88 62.5 L 342.63 59 L 340.88 55.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-8"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 41px; margin-left: 280px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Cache<div>miss</div></div></div></div></foreignObject><text x="280" y="44" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Cache...</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-11"><g><path d="M 89 59 L 160.63 59" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 165.88 59 L 158.88 62.5 L 160.63 59 L 158.88 55.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-12"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 57px; margin-left: 40px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Client request</div></div></div></foreignObject><text x="40" y="60" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Client request</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-15"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 171px; margin-left: 40px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Return cached<div>response</div></div></div></div></foreignObject><text x="40" y="174" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Return cached...</text></switch></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-27"><g><rect x="489" y="98" width="140" height="60" rx="9" ry="9" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/></g><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 138px; height: 1px; padding-top: 128px; margin-left: 490px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">Calculate response<div>offset in file<br /></div></div></div></div></foreignObject><text x="559" y="132" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="12px" text-anchor="middle">Calculate response...</text></switch></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-30"><g><path d="M 629 128 L 758 128 L 758 143.63" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 758 148.88 L 754.5 141.88 L 758 143.63 L 761.5 141.88 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-41"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 111px; margin-left: 710px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Bytes-range<div>request</div></div></div></div></foreignObject><text x="710" y="114" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Bytes-range...</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-36"><g><path d="M 559 300 L 559 339 L 251.37 339" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 246.12 339 L 253.12 335.5 L 251.37 339 L 253.12 342.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-39"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 321px; margin-left: 420px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Return response to CloudFront</div></div></div></foreignObject><text x="420" y="324" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Return response to CloudFront</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-31"><g><rect x="489" y="240" width="140" height="60" rx="9" ry="9" fill="#dae8fc" stroke="#6c8ebf" pointer-events="all"/></g><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 138px; height: 1px; padding-top: 270px; margin-left: 490px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><div>Trims any padding<br /></div></div></div></div></foreignObject><text x="559" y="274" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="12px" text-anchor="middle">Trims any padding&#xa;</text></switch></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-44"><g><path d="M 929 189 L 803.37 189" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 798.12 189 L 805.12 185.5 L 803.37 189 L 805.12 192.5 Z" fill="rgb(0, 0, 0)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-45"><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 176px; margin-left: 868px;"><div data-drawio-colors="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 11px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; background-color: rgb(255, 255, 255); white-space: nowrap;">Write files</div></div></div></foreignObject><text x="868" y="179" fill="rgb(0, 0, 0)" font-family="&quot;Helvetica&quot;" font-size="11px" text-anchor="middle">Write files</text></switch></g></g></g></g><g data-cell-id="RVEh1LkQJVqgOGLHUIks-43"><g><path d="M 929 150 L 1007 150 L 1007 228 L 929 228 Z" fill="#8c4fff" stroke="none" pointer-events="all"/><path d="M 959.91 169.36 L 962.22 169.36 L 962.22 167.04 L 959.91 167.04 Z M 957.6 170.51 L 957.6 165.89 C 957.6 165.25 958.12 164.73 958.76 164.73 L 963.38 164.73 C 964.02 164.73 964.53 165.25 964.53 165.89 L 964.53 170.51 C 964.53 171.15 964.02 171.67 963.38 171.67 L 958.76 171.67 C 958.12 171.67 957.6 171.15 957.6 170.51 Z M 969.16 162.42 L 971.47 162.42 L 971.47 160.11 L 969.16 160.11 Z M 966.84 163.58 L 966.84 158.96 C 966.84 158.32 967.36 157.8 968 157.8 L 972.62 157.8 C 973.26 157.8 973.78 158.32 973.78 158.96 L 973.78 163.58 C 973.78 164.22 973.26 164.73 972.62 164.73 L 968 164.73 C 967.36 164.73 966.84 164.22 966.84 163.58 Z M 971.47 173.98 L 973.78 173.98 L 973.78 171.67 L 971.47 171.67 Z M 970.31 169.36 L 974.93 169.36 C 975.57 169.36 976.09 169.87 976.09 170.51 L 976.09 175.13 C 976.09 175.77 975.57 176.29 974.93 176.29 L 970.31 176.29 C 969.67 176.29 969.16 175.77 969.16 175.13 L 969.16 170.51 C 969.16 169.87 969.67 169.36 970.31 169.36 Z M 968 181.49 C 965.97 181.49 958.59 181.4 953.79 180.22 L 965.51 198.78 C 965.63 198.97 965.69 199.18 965.69 199.4 L 965.69 202.03 C 967.31 202.5 968.81 202.51 970.31 202.05 L 970.31 199.4 C 970.31 199.18 970.37 198.97 970.49 198.78 L 982.21 180.22 C 977.41 181.4 970.03 181.49 968 181.49 Z M 949.8 178.24 C 949.62 177.98 949.51 177.69 949.51 177.37 C 949.51 174.08 960.35 173.48 966.82 173.41 L 966.84 175.72 C 958.9 175.8 954.05 176.71 952.34 177.38 C 954.15 178.14 959.5 179.18 968 179.18 C 976.48 179.18 981.82 178.15 983.65 177.39 C 982.9 177.09 981.35 176.64 978.26 176.27 L 978.54 173.97 C 985.54 174.83 986.49 176.17 986.49 177.37 C 986.49 177.69 986.38 177.98 986.2 178.24 L 972.62 199.74 L 972.62 202.87 C 972.62 203.34 972.34 203.76 971.9 203.94 C 970.64 204.45 969.37 204.7 968.08 204.7 C 966.79 204.7 965.49 204.45 964.13 203.95 C 963.68 203.78 963.38 203.35 963.38 202.87 L 963.38 199.74 Z M 976 211.67 C 976.18 212.1 976.08 212.6 975.75 212.93 L 968.82 219.86 C 968.59 220.09 968.3 220.2 968 220.2 C 967.7 220.2 967.41 220.09 967.18 219.86 L 960.25 212.93 C 959.92 212.6 959.82 212.1 960 211.67 C 960.18 211.24 960.6 210.96 961.07 210.96 L 964.53 210.96 L 964.53 207.49 L 966.84 207.49 L 966.84 212.11 C 966.84 212.75 966.33 213.27 965.69 213.27 L 963.86 213.27 L 968 217.41 L 972.14 213.27 L 970.31 213.27 C 969.67 213.27 969.16 212.75 969.16 212.11 L 969.16 207.49 L 971.47 207.49 L 971.47 210.96 L 974.93 210.96 C 975.4 210.96 975.82 211.24 976 211.67 Z" fill="#ffffff" stroke="none" pointer-events="all"/></g><g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe flex-start; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 235px; margin-left: 968px;"><div data-drawio-colors="color: #232F3E; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: rgb(35, 47, 62); line-height: 1.2; pointer-events: all; white-space: nowrap;">Glue Job</div></div></div></foreignObject><text x="968" y="247" fill="#232F3E" font-family="&quot;Helvetica&quot;" font-size="12px" text-anchor="middle">Glue Job</text></switch></g></g></g></g></g></g><switch><g requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"/><a transform="translate(0,-5)" xlink:href="https://www.drawio.com/doc/faq/svg-export-text-problems" target="_blank"><text text-anchor="middle" font-size="10px" x="50%" y="100%">Text is not SVG - cannot display</text></a></switch></svg>
@diagram.svg

Usage

Setup

In order to deploy the solution in your AWS account, you need to create a new stage file in the infrastructure/stages directory. Name the file with the name of the environment you want to deploy to. For example, if you want to deploy to the dev environment, create a file named dev.yml. After creating the file, you can deploy the solution by running the following command:

make clean build && make deploy environment=<env-name> profile=<profile-name>

Where <env-name> is the name of the environment you want to deploy to and <profile-name> is the name of the AWS CLI profile you want to use. If you don't provide a profile name, the default profile will be used.

Generate test data

To generate test data, you can use the following command:

python3 tests/generate.py --frac 0.1 sample-0.1.csv

Configure

To modify the final file structure, you can update transform section. The following options are available:

• dimensions.path - list of dimensions that will be included in a path of the final file.
• dimensions.file - list of dimensions that will be used to define position offset in the final file. Here we should use dimensions with low cardinality and high density between them.
• dimensions.body - list of dimensions that will be included in the body of the response.
• metrics - list of metrics that will be included in the body of the response.

Cleanup

To remove the solution from your AWS account, you need to remove all files in S3 bucket and then delete the CloudFormation stack. As we use Lambda@Edge function, removing stack will fail on the first attempt. You need to wait for one hour and then delete the stack again.