In this guide, we'll explore how to manage access to buckets in Amazon Simple Storage Service (S3), and how to connect them to Accurate.Video. We'll explore two options for configuring access, either using AWS Identity and Access Management (IAM) policies or through S3 bucket policies.
Adding storages to Accurate.Video is a task that is performed by an administrator or someone with access to the AWS infrastructure. Due to the nature of this task, some AWS knowledge is required. You'll need access to an AWS account where you can modify roles in IAM and buckets in S3.
Ensure your user has the sufficient permissions required for adding storages in Accurate.Video. Refer to the roles available in the guide Authentication - Roles and permissions configurations.
If you cannot see the storage tab, or the Add storage button is missing, your user lacks the required permissions.
Storages are accessed through the Storages tab in the navigation bar at the top.
Clicking the Add storage button opens up a modal popup where you enter the required information about storage, shown below.
There are three options for supplying credentials when adding storage:
The server credentials option makes use of the IAM role that is assigned to each running service inside our Amazon Elastic Container Service (ECS) cluster. This is the default and recommended option for controlling access to S3 buckets.
Using custom credentials, you can directly enter the access key and secret key of an IAM user. This can be a good option during testing, but the server credentials option is much more flexible.
Anonymous access can be used if the bucket is public and you can read from it openly. Not recommended unless you're aware of the risks.
The recommended best practice is to differentiate between input and output buckets. Input buckets are storages where files are being read and ingested from, with read-only permission. Output buckets are dedicated buckets where artifacts are stored, such as thumbnails, waveform data, audio & video proxy files, and exported metadata.
You have full control of how you map your buckets, and you can have multiple input and output buckets.
After installing Accurate.Video, you'll find that a number of IAM roles have been created in your AWS account. These roles control what permissions the different parts of the system are allowed to access. You'll see there are roles for the frontend (UI), the adapter (REST & middleware), and the jobs system.
Accessing buckets is done by the jobs system, so this is the role we'll need to find and note down the ARN for later.
The name will look something like name-of-stack-JobsTaskStack-<hash>-ECSTaskRole-<hash>. It's a long name, but there will be only one matching this exact name. Once you've found it, you can look at the attached policies. There will be an inline policy attached, with access to ECS, IAM, MediaConvert, Rekognition, and S3.
From here there are two options for controlling access to S3 buckets, either using an IAM policy or through individual S3 bucket policies.
The first option is to write and attach IAM policies directly to the role itself. Refer to the AWS guide below for more details on how to write IAM policies.
AWS Security Blog: Writing IAM Policies: How to Grant Access to an Amazon S3 Bucket
Note: for an input bucket, you'll need to include the actions s3:GetObject and s3:ListBucket, and for an output bucket, add s3:PutObject.
The second option is to restrict access to the S3 bucket directly using a bucket policy. Note that this needs to be done for every bucket you connect as storage. Refer to the AWS guide below for more details on how to restrict access using bucket policies.
AWS Security Blog: How to Restrict Amazon S3 Bucket Access to a Specific IAM Role
Note: To give access to the IAM role, you'll use the principal section to refer to the unique ARN of the IAM role of the jobs task, as noted down earlier.
In order to access media files from a browser, a CORS policy must be added to the bucket. The CORS policy is found at the bottom of the bucket Permission page.
More details on CORS can be found on the AWS docs page below.
Amazon S3: Using cross-origin resource sharing (CORS)
Here is a sample CORS configuration that can be used to allow access:
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"GET",
"HEAD"
],
"AllowedOrigins": [
"*"
],
"ExposeHeaders": [
"Content-Range"
],
"MaxAgeSeconds": 3000
}
]
Note 1: If you know the exact origin of the Accurate.Video system, you should limit this instead of setting the AllowedOrigin to "*" (accept anything).
Note 2: If AllowedOrigins is not set to "*", then one of the allowed origins must be the literal string "null" (without quotation marks) in order for DASH and HLS playback to work. The reason for this is that all browsers set their origin to "null" when following redirects.
Here you can find more information about CORS and why this configuration is needed: https://en.wikipedia.org/wiki/Cross-origin_resource_sharing
All settings are now set in AWS. Now it’s time to add the storage to Accurate Video.
Go to the Accurate Video start page, click on “storages” at the top. You should see a screen similar to the above, empty without any storage. If you have already added storage, it will show up here.
Click the “add storage” button at the top.
A modal with a form should pop up. Start by giving the storage a name, this is purely for identification and can be anything you want.
The permission section decides what AV can do with the storage and here you simply click the respective checkbox to activate certain permissions:
With the above permissions, you can control and easily create read-only storages where files are ingested from, and writable, output storages where files that are generated by Accurate.Video is stored, files such as transcoded proxy files, thumbnails, audio waveform, and time-coded marker metadata.
Using tags, you can fine-tune where these files are stored. There are a number of pre-defined tags that can be used, refer to the guide on writable storage metadata fields for details.
The next step is to enter the S3 bucket name, note that this represents the name in S3 that was used before. The region should also be set, this has to be the same region as in S3.
If you are unsure about what region codes, look at the following list:
Optionally you can set a periodic refresh interval. This reflects how often the S3 bucket is queried and files within the bucket are indexed by Accurate.Video. Note that it will poll the S3 API according to this schedule, so you don’t want to set the interval too high. In case the storage is very rarely updated, the best option could be to disable it and doing manual refreshes in the UI when required.
Another option is to use Amazon SNS to automatically index new files as they appear in S3. Please refer to the Configuring SNS and S3 event notifications in Accurate.Video guide on how to set this up.
After connecting the storage, you should see a green text which says Connected, this is an indication that everything is ok. If it comes up as Broken instead, something is wrong and Accurate.Video failed to access the storage.
The next step is to start ingesting files, which can be done either through the UI or the API.
Refer to the Ingest assets using the UI guide for more details on how to ingest files through the UI.
You can also configure automatic ingest, see Auto Ingest using the UI for more details.
If you're interested in learning more about automatic ingest in general, see Automatic Ingest.
You can also connect storages and ingest files through the REST API, see REST API Examples for some examples.