How to run Typesense search in AWS
Get Typesense search running within minutes in AWS using infrastructure as code.
Searching datasets and retrieving relevant results blazing-fast is a challenge, no matter if the underlying database of our application is RDBMS or NoSQL.
If you're serious about search and need typo tolerance, result rankings, synonyms, filtering, faceting, geo search, and whatnot, you better forget about those sub-optimal DB queries and bring a dedicated search service into the mix.
There are multiple cloud offerings for hosted search such as Elasticsearch, Algolia, Meilisearch, or Typesense cloud. But due to infrastructure architecture requirements, regulatory reasons, cost optimization, or just cost consolidation it often makes more sense to run our search service in AWS.
Note: AWS has its own search service, OpenSearch which is a fork of ElasticSearch. At the time of writing AWS just announced OpenSearch Serverless, which is in beta/preview release.
Typesense is my favorite, it's simple, fast, typo-tolerant, and open-source. I've been using it on multiple production projects handling searches on datasets of millions of entries.
For production use, it is recommended to run a multi-node Typesense cluster, however, there are plenty of scenarios when a single-node service will do the job.
In this article, I will explain how to get Typesense running as a single node using infrastructure as code and make it as resilient and robust as it can be in a single Availability Zone and how to size your instance depending on your data set.
The CloudFormation template will create the following AWS resources:
EC2 Auto Scaling Group & Launch Configuration
Instance Profile, IAM Roles & Security Group
EBS Volume
Elastic IP
Lambda Function & CloudFormation Custom Resource
A self-healing search service
We're going to launch the instance using EC2 Auto Scaling, there are no additional fees for it, and the service can automatically determine the health status of an instance using EC2 status checks.
If the search service instance is unhealthy, it will be terminated and replaced by a new one with minimum downtime.
When an instance is terminated and a new one gets created, the data stored in the EC2 instance store is lost and the new instance will have a different IP.
That's why it is critical to have persistent block storage and an elastic IP.
EC2 Auto Scaling will make sure to have 1 healthy instance running at all times.
# ASG
AutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
LaunchConfigurationName: !Ref LaunchConfig
DesiredCapacity: 1
MinSize: 0
MaxSize: 1
AvailabilityZones:
- !Select [0, !GetAZs ""]
Tags:
- Key: Name
PropagateAtLaunch: true
Value: !Sub ${AWS::StackName}-${EnvironmentType}
EC2 Instance Type
CPU capacity is important to handle concurrent search traffic and indexing operations, so Typesense requires at least 2 vCPUs of compute capacity to operate.
All the next-gen EC2 T4g instances have at least 2 vCPUs, they're powered by Arm-based AWS Graviton2 processors and are ideal for running applications with moderate CPU usage that experience temporary spikes in usage.
The amount of RAM required is completely dependent on the size of the data you index, but for Typesense to hold the whole index in memory, the instance memory should be 2-3X the size of the data set.
As the Typesense process, itself is quite lightweight (20MB RAM with empty dataset) a 2GB memory instance (t4g.small) will likely handle a data set of up to 1GB.
Data Storage
Amazon EC2 instance store is temporary, if the instance stops, hibernates, or terminates the data from the instance store is lost, so we need to use a durable data storage such as EBS.
Typesense stores a copy of the raw data on disk and then builds the in-memory index with the data. Then at search time, after determining the final set of documents to return in the API response, it fetches these documents (only) from the disk and puts them in the API response.
We'll need a fast and durable storage, hence we'll go for EBS gp3 volume type (next-gen General Purpose SSD).
The size of the EBS volume should be at least the size of the raw dataset.
Launch Configuration
The Launch Configuration will be used by the Auto Scaling group to configure our EC2 instance.
LaunchConfig:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
InstanceType: !Ref InstanceType
ImageId: !Ref LatestLinuxAmiId
IamInstanceProfile: !GetAtt EC2InstanceProfile.Arn
SecurityGroups:
- !Ref EC2SecurityGroup
UserData:
Fn::Base64: !Sub |
# [...] removed for brevity plese check below
Launch Configuration UserData
This is where most of the auto-healing magic happens, UserData allows us to run commands on our instance at launch. When an unhealthy instance is terminated and a new one is created it will be empty.
UserData will run the commands to ensure the search service is reconfigured back to its running state.
1. Associate the IP & attach the EBS volume
#!/usr/bin/env bash
# get instance-id
EC2_INSTANCE_ID=$(curl -s http://instance-data/latest/meta-data/instance-id)
# associate the Elastic IP address to the instance
aws ec2 associate-address --instance-id $EC2_INSTANCE_ID --allocation-id ${ElasticIP.AllocationId} --allow-reassociation --region ${AWS::Region}
# attach volume
aws ec2 attach-volume --device ${VolumeDevice} --instance-id $EC2_INSTANCE_ID --volume-id ${EBSVolume} --region ${AWS::Region}
# wait for volume to be properly attached
DATA_STATE="unknown"
until [ "${!DATA_STATE}" == "attached" ]; do
DATA_STATE=$(aws ec2 describe-volumes \
--region ${AWS::Region} \
--filters \
Name=attachment.instance-id,Values=${!EC2_INSTANCE_ID} \
Name=attachment.device,Values=${VolumeDevice} \
--query Volumes[].Attachments[].State \
--output text)
sleep 5
done
# format the volume if doesn't have file system (on stack creation)
blkid --match-token TYPE=ext4 ${VolumeDevice} || mkfs.ext4 -m0 ${VolumeDevice}
# create mount root folder
mkdir -p ${VolumeMountPath}
# mount the volume to folder
mount ${VolumeDevice} ${VolumeMountPath}
# persist the volume on restart
echo "${VolumeDevice} ${VolumeMountPath} ext4 defaults,nofail 0 2" >> /etc/fstab
2. Configure the Typesense Server
If you're using the CloudFormation template defaults the VolumeMountPath is set to /mnt/ebs hence:
The config file will be at
/mnt/ebs/typesense.iniLogs will be under
/mnt/ebs/log/Data will be stored under
/mnt/ebs/data/
# create typesense server config
if [ ! -f ${VolumeMountPath}/typesense.ini ]
then
echo "[server]
api-key = ${TypesenseApiKey.Value}
data-dir = ${VolumeMountPath}/data
log-dir = ${VolumeMountPath}/log
api-port = ${TypesensePort}" > ${VolumeMountPath}/typesense.ini
chown ec2-user:ec2-user ${VolumeMountPath}/typesense.ini
fi
# create typesense data folder if not exists
if [ ! -d ${VolumeMountPath}/data ]
then
mkdir -p ${VolumeMountPath}/data
chown ec2-user:ec2-user ${VolumeMountPath}/data
fi
# create typesense logs folder if not exists
if [ ! -d ${VolumeMountPath}/log ]
then
mkdir -p ${VolumeMountPath}/log
chown ec2-user:ec2-user ${VolumeMountPath}/log
fi
3. Download & Install Typesense Server
As we're running the service on a T4g instance powered by Graviton2 processors we have to download the arm64 Binary.
# download & unarchive typesense
curl -O https://dl.typesense.org/releases/${TypesenseVersion}/typesense-server-${TypesenseVersion}-linux-arm64.tar.gz
tar -xzf typesense-server-${TypesenseVersion}-linux-arm64.tar.gz -C /home/ec2-user
# remove archive
rm typesense-server-${TypesenseVersion}-linux-arm64.tar.gz
4. Create systemd service & enable the daemon
As we installed it from a binary we have to create a systemd service for the Typesense server. This will make the Typesense server service always available.
# create typesense service
echo "[Unit]
Description=Typesense service
After=network.target
[Service]
Type=simple
Restart=always
RestartSec=5
User=ec2-user
ExecStart=/home/ec2-user/typesense-server --config=${VolumeMountPath}/typesense.ini
[Install]
WantedBy=default.target" > /etc/systemd/system/typesense.service
# start typesense service
systemctl start typesense
# enable typesense daemon
systemctl enable typesense
EBS & Elastic IP
Important: the instance needs to be launched in the same Availability Zone as the EBS volume.
Tip: to save costs on dev instances, you can create EC2 Auto Scaling scheduled actions that set autoscaling to 0 at night and back to 1 during work hours.
EBSVolume:
Type: AWS::EC2::Volume
Properties:
Size: !Ref VolumeSize
VolumeType: gp3
AvailabilityZone: !Select [0, !GetAZs ""]
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-${EnvironmentType}
DeletionPolicy: Snapshot
ElasticIP:
Type: AWS::EC2::EIP
Properties:
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-${EnvironmentType}
EC2 IAM Role & Security Groups
The template will create an Instance Profile, an IAM Role, and a Security Group.
The IAM Role will be assumed by the EC2 instance so it can attach the EBS volume and self-associate the Elastic IP. These commands are run using AWS CLI in the Launch Configuration UserData .
The Instance Profile allows passing the IAM role to the EC2 instance.
The Security Group permits the instance to receive traffic on the port used by the Typesense server (8108) and on port 22 for SSH access.
Note: a LambdaExecutionRole is also created to allow the API Key generation function to write logs to AWS CloudWatch, but it's omitted for brevity.
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
InstanceProfileName: !Sub ${AWS::StackName}-${EnvironmentType}-instance-profile
Path: /
Roles:
- !Ref EC2InstanceRole
EC2InstanceRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub ${AWS::StackName}-${EnvironmentType}-instance-role
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: !Sub ${AWS::StackName}-${EnvironmentType}-instance-policy
PolicyDocument:
Version: "2012-10-17"
Statement:
# allow attaching EBS volumes
- Effect: Allow
Action:
- ec2:AttachVolume
- ec2:DescribeVolumes
Resource: "*"
# allow Elastic IP association
- Effect: Allow
Action:
- ec2:AssociateAddress
Resource: "*"
# SG
EC2SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: allow connections on HTTP & SSH
SecurityGroupIngress:
# Allow HTTP traffic on Typesense port
- IpProtocol: tcp
FromPort: !Ref TypesensePort
ToPort: !Ref TypesensePort
CidrIp: 0.0.0.0/0
# Allow SSH - !! narrow CidrIp down, too broad
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
Typesense API Key Generation
The CloudFormation stack will generate a Typesense admin API key using a CustomResource that references a RandomStringGenerator Lambda function.
The TypesenseApiKeyis exported in the Output so it can be referenced in other stacks. Optionally it can be stored in a AWS::SSM::Parameter to be used by other AWS services.
The admin API key provides full control over the Typesense API, make sure you create Scoped API Keys to be used in your application.
TypesenseApiKey:
Type: AWS::CloudFormation::CustomResource
Properties:
Length: 32
ServiceToken: !GetAtt RandomStringGenerator.Arn
The random string generator Lambda function is deployed using inline code, this works for simple functions when the code length is up to 4096 chars.
RandomStringGenerator:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Sub ${AWS::StackName}-${EnvironmentType}-random-string
Code:
ZipFile: >
const response = require("cfn-response"), crypto = require("crypto");
exports.handler = (event, context) =>{
response.send(event, context, response.SUCCESS, {Value:crypto.randomBytes(parseInt(event['ResourceProperties']['Length'])).toString('hex')});
};
Handler: index.handler
Role: !GetAtt LambdaExecutionRole.Arn
Runtime: nodejs16.x
MemorySize: 128
Timeout: 10
One-click Deployment to get going quickly, or click to view the complete self-contained Cloudformation template that can be deployed into any AWS account.
For complete details about Typesense API check API Reference
Summary
Although this CloudFormation stack template is made especially to get a Typesense search service up and running in minutes, it can be repurposed easily, e.g. for installing WordPress instead of Typesense.
For educational purposes, it features some common real-life scenarios and pattern examples such as:
Create a single self-healing instance with EC2 Auto Scaling
Use Launch Configuration and UserData to run commands on the instance at launch
Persistent auto-mounting EBS storage + creating a filesystem on new volumes
Auto-install a service from a Linux Binary
Creating a Linux systemd service & daemon
Use of Metadata to group and sort CloudFormation parameters
How to deploy a Lambda function with inline code
Use of Custom Resource to trigger a Lambda and reference the output
Use of CloudFormation Intrinsic Functions (!Ref, !GetAtt, !Sub, !Select, !GetAZs)
Cover photo by Marten Newhall on Unsplash