IoT development is an ever growing endeavour that has incorporated transmission of different data formats since its inception. The goal of this experiment is to understand the requirements of transmitting images from an IoT sensor and how it would scale as the number of subscribers increase over time.
I believe that the development of IoT sensors and actuators brings us closer to a fully autonomous world run by intelligent software. It’s use is prevalent in many different industries and is only expected to grow over the years.
In essence, the research question of our experiment is as follows:
Does the streaming of multimedia data from an IoT sensor, via AWS IoT Core, to it’s subscribers scale well as the number of subscribers are increased?
Streaming of multimedia data from an IoT sensor, via AWS IoT Core, to it’s subscribers scales efficiently as it increases to a maximum of 500. The latency incurred for each subscriber shall be ≤ 3 seconds.
The primary measurement of this experiment shall be the latency incurred for each subscriber. In the context of this experiment, latency is described as the number of seconds it takes for the subscriber to receive an image upon publishing it.
As depicted from the above image, the architecture shall include the following entities:
The subscriber and publisher entities shall be built out with Ruby scripts while all infrastructure, including AWS IoT Core, shall be deployed via Terraform.
For context, Terraform is an infrastructure as code (IaC) tool that enables efficient management of cloud infrastructure. With a simple terraform apply from our local machine, we can deploy all the declared infrastructure. After conducting the experiment, destroying the deployed resources is as simple as terraform destroy.
By using Terraform in this project, deployment of infrastructure is heavily abstracted away and speed of deployment is high. Therefore allowing for a more efficient execution of the experiment as a whole.
One of the common methods of data transmission for IoT devices is the MQTT protocol. This protocol models the publisher/subscriber pattern to efficiently transmit data from an IoT device (publisher) to the entities (subscribers) that are listening to it’s updates [1].
To route published messages to it’s appropriate subscriber, the MQTT protocol enforces the need for a topic and an additional entity, the MQTT broker. A topic essentially serves as an identifier for the type of messages that are being transmitted. Subscribers may then subscribe to a given topic to listen to messages that are published to that topic. Both the subscriber and publisher entities are denoted as MQTT clients where the former listens for messages whilst the latter publishes them [1].
When an IoT device wishes to make an update to it’s subscribers, it connects to it’s MQTT broker as an MQTT client and publishes a message to it’s allotted topic. Upon receipt of this message by the MQTT broker, it then forwards it to all subscribers of the corresponding topic. Where a subscriber is an MQTT client that has been connected to the MQTT broker [1].
The environment in which the publisher and subscriber exists is the same. Hence implying:
The size of the images transmitted shall be capped at 128 KBs.
The MQTT client library used for both the publisher and subscriber shall be the same.
The language in which the publisher and subscriber script is written in, shall be the same, Ruby.
The experiment shall be conducted as follows:
iot-core-topic"The network environment in which the experiment is conducted in, may have a higher bandwidth and throughput than the environment of an IoT sensor. Moreover, an IoT sensor’s environment may experience a lot more variability in it’s bandwidth and throughput.
The experiment is conducted with images that shall not exceed 128 KBs in size. However, in reality, images larger than this may be transmitted by the IoT sensor. Said larger size may even imply an exponentially higher latency as the number of subscribers increase.
Upon conducting the experiment as per the procedure denoted above, it can be observed that the average latency did increase as the number of subscribers were increased. Intuitively, this makes sense as the MQTT broker has more subscribers to forward packets to.
Overall, these results support our hypothesis and allow us to reject the null hypothesis. For reference, the null hypothesis in this context is as follows:
Streaming multimedia data from an IoT sensor, via AWS IoT Core, to it’s subscribers does not scale well as the number of subscribers are increased.
During the setup phase of the experiment, I was unable to establish a connection with the MQTT broker from my local machine. Due to this, I was unable to publish images and subscribe to topics.
A certificate was required to authorize all interactions with AWS IoT Core’s MQTT broker. While the certificate had been created, it lacked the permissions to perform any actions. Therefore rendering it useless with respect to it’s purpose.
To resolve this issue, the certificate had to be associated with a policy that included the correct permissions.
resource "aws_iot_policy" "iot_certificate_policy" {
name = "iot-certificate-policy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect",
"iot:Publish",
"iot:Receive",
"iot:RetainPublish",
"iot:Subscribe"
],
"Resource": "*"
}
]
}
EOF
}
resource "aws_iot_policy_attachment" "att" {
policy = aws_iot_policy.iot_certificate_policy.name
target = aws_iot_certificate.iot_certificate.arn
}
Only after doing so, the connection between the MQTT client and AWS IoT Core’s MQTT broker could be established. Therefore allowing for messages to be published and topics to be listened to.
As mentioned earlier, in order to establish a connection with AWS IoT Core’s MQTT broker, the client requires a certificate. However, I lacked a locally persisted certificate to establish a connection.
Upon creation of a certificate, during deployment, it’s associated PEM file and private key are briefly available for persistence before being destroyed due to it’s sensitivity. Since deployment is automated via Terraform, one cannot retrieve the certificate during deployment and persist it locally. Therefore implying the absence of a certificate to establish a connection.
To persist the certificate, AWS SSM’s Parameter store was used. Upon creation of the PEM file and private key, it is then stored in the Parameter store with a specified name.
resource "aws_ssm_parameter" "certificate_pem" {
name = "certificate-pem"
type = "String"
value = aws_iot_certificate.iot_certificate.certificate_pem
}
resource "aws_ssm_parameter" "private_key" {
name = "private-key"
type = "String"
value = aws_iot_certificate.iot_certificate.private_key
}
When a client wishes to establish a connection with the MQTT broker, it retrieves the PEM file and the private key from the Parameter store.
class SSMRetriever
class << self
def retrieve(name)
secrets_manager.get_parameter(name: name, with_decryption: true).parameter.value
end
def secrets_manager
@secrets_manager ||= Aws::SSM::Client.new(
region: ENV['AWS_REGION'],
access_key_id: ENV["AWS_ACCESS_KEY_ID"],
secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"]
)
end
end
end
Message publishes failed due to an inappropriate message format and size.
Upon a message publish, said message would be persisted into AWS Timestream database by default. The problem with this is that AWS Timestream’s supported data types and maximum message size did not meet the requirements of our experiment setup [2].
Since AWS Timestream did not meet the requirements of our experiment, a different datastore had to be used. The datastore that met all our requirements was AWS S3. Using S3, we are now able to transmit images of sizes ≤ 128 KBs to our topic.
resource "aws_iot_topic_rule" "iot_rule" {
name = "iot_rule"
sql = "SELECT * FROM 'iot-core-topic'"
sql_version = "2016-03-23"
enabled = true
description = "Send data from the AWS IoT Core to S3"
s3 {
bucket_name = aws_s3_bucket.iot_core_bucket.bucket
key = "iot-core-topic/${timestamp()}"
role_arn = aws_iam_role.iot_role.arn
}
}
In the context of the experiment, all the subscribers existed in the same location, on the machine in which the experiment is conducted on. To mimic real life scenarios, it’s best that the subscribers are remote instances located in different locations.
To deploy said remote instances, a combination of AWS EC2, Docker and Terraform may be used.
The network environment in the experiment was fairly ideal when compared to that of typical IoT sensors. This may have impacted the results of our experiment, hence it has been identified as a confounding variable in the Threats to Validity section.
To fix this, latency that matches that of typical IoT sensors could have been emulated in the local environment. Therefore rendering more accurate results.
The original intent of the experiment was to start with image transmission and transition to video streaming. However, AWS IoT Core is not built for both image and video transmission. It is primarily built for small updates at high frequency. Moreover, the maximum payload size that AWS IoT Core facilitates is 128 KBs [3]. Therefore implying that larger images of higher resolution may not be transmitted.
Given these constraints, the experiment would have been better conducted with AWS Kinesis Video instead [4], a service better equipped for video streaming at a large scale.