Merge pull request #13 from Vualto/dev-branch

Dev branch

docs/source/ReleaseNotes.md

@@ -0,0 +1,105 @@
+# RELEASE NOTES
+
+## 1.169.2 - 07/07/2020
+
+- General release build.
+
+## 1.169.1 - 07/07/2020
+
+- Build fixes
+
+## 1.169.0 - 06/07/2020
+
+- API enhancements for UI integrations
+- Updated to a newer Ruby version
+- Support for specifying Bitmovin encoding region
+- Bitmovin encodings now have client labels assigned
+- Queue reservation functionality for priority jobs
+- Optimised thumbnail and VTT generation for timeline preview thumbnails
+
+## 1.168.1 - 17/03/2020
+
+- Added workflows for:
+  - generating gifs
+  - capturing single frames as jpgs
+  - deleting multiple assets without deleting an entire VOD.
+- Added support for multiple file formats when ingesting and encoding using Bitmovin.
+  - Full list of supported source files are:
+    - mp4
+    - mov
+    - mpg
+    - mkv
+    - avi
+- Added number of failed jobs in the health check endpoint response.
+- Bugfix: Added index to audio tracks when ingesting mp4s. This is to ensure each track name is unique.
+- Added support for USP 1.10.18
+
+## 1.167.2 - 04/11/2019
+
+- Bitmovin enhancements for re-packaging.
+
+## 1.167.1 - 29/10/2019
+
+- Bitmovin integration now supports setting the ACL permission for the mp4 outputs.
+
+## 1.167.0 - 25/10/2019
+
+- Added support for Azure Blob Storage as a source and/or destination storage option.
+  - The option needs to be specified as `azure_blob`.
+- Added the functionality to encode ingest source into multiple smaller resolutions
+  - Caveat: Currently this only works if there is a single MP4 source in the ingest folder.
+- Task Engine version is now visible within the queue page and returned as part of the health check api request.
+- Updates to the preview thumbnail (trickplay) generation to improve performance.
+- Updated VOD remix:
+  - Added support for Frame Accurate clips
+  - Added option for setting the profile (for a clip) for the remix output
+- Updated Task Engine queue and logs UI.
+  - Completed status filter will no longer load failed jobs.
+  - Search by job IDs is now supported. Multiple job IDs must be comma separated.
+  - There is no longer a requirement for both From and To date to be provided for the date filter to work.
+  - Search results will now include the Created At information for the jobs.
+  - Word wrapping has been added to logs.
+- Added `output_root` option to all workflows
+- API error codes updated.
+- CPIX support for DRM packaging
+- Completed vodremix workflow.
+
+## 1.166.8 - 04/09/2019
+
+- Hotfix: Updated DRM switch callback message.
+
+## 1.166.7 - 20/08/2019
+
+- Hotfix: supporting spaces in file inputs/outputs.
+
+## 1.166.6 - 14/08/2019
+
+- Added support for USP 1.10.12
+- Removed support for USP 1.8.5
+
+## 1.166.5 - 13/08/2019
+
+- Added visible field to log with show_all param toggling.
+- Improvements to the `build_thumbnails` workflow.
+- Complete re-write of the way storage is handled:
+  - Storage controller dictates which storage medium is used.
+  - S3 storage re-written to support this.
+  - Added local storage support.
+  - Added support for different source and destination storage types within the same job.
+    - Eg. Source for ingest can be a local MP4 and the destination for the packaged content is S3.
+- Rewrite of DRM switch.
+- Added vodcapture.json optional parameter "output_root" to specify the output root key.
+- Added vodcapture.json optional parameter "empty_target" to specify whether to delete the contents of the output folder or not.
+- New versioning system for the Task Engine.
+
+## 165 - 15/02/2019
+
+- Fixed issue with retry job call from the front end (queue page).
+- Fixed bug with job scheduler not assigning the correct run_at time.
+- Added a health check endpoint for improved Zabbix monitoring.
+- Added proper support for TransDRM (without subtitles).
+- Creating a clear manifest now requires the `clear` option to be added to the DRM list.
+- Added `build_thumbnails` workflow (builds thumbnail sprite and vtt file for VOD content).
+- Added transcoding option back to mp4 generation due to compatibility issues with Twitter.
+- Added support for applying track properties to manifest files.
+  - These can be added either through the job JSON payload or saved in Central Configuration.

docs/source/TaskEngineAdditionalFeatures.md

@@ -34,6 +34,14 @@ The `"priority"` parameter needs to be submitted within the `"job"` section of t
 
 Whenever an execution slot is available, the system will first check by priority and then check the submission time and date of the job. In the case where multiple jobs are executed with the same priority (eg. with the default priority 5), the Task Engine operates in a FIFO (First In First Out) manner.
 
+### Priority Slots
+
+To further enhance support for priority jobs, a new setting has been added to reserve job slots for default to top priority (1 - 5) jobs. This will allow a client to schedule as many low priority jobs as required while still having job slots free for default and priority jobs.
+
+The advantage of using priority slots is to stop the queue from being held up by low priority jobs. This is especially useful if long running jobs are given a lower priority than the default as they can be queued up without exhausting the setup's concurrency availability.
+
+**Important note**: This will not increase the number of max concurrent jobs but it will reserve some fo the concurrency for jobs with priority between 1 and 5. As an example, if a setup has 5 maximum concurrent jobs and the priority slots is set to 2, any job can ustilise 3 concurrency slots but only jobs with priority between 1 and 5 can use the 2 priority slots.
+
 ## Multiple Clips
 
 The Task Engine includes a feature that will allow multiple clips to be stitched together into a single clip, in a single job. This can be done by defining multiple objects within the `"clips"` parameter in the json payload for `vodcapture`. This also allows a mixture of live and VoD sources to be captured and stitched together into a new clip. The example below shows how the `"clips"` parameter would need to be provided to achieve this.
@@ -198,7 +206,7 @@ If a capture is submitted with a clip end time that is in the future, it will be
 
 ## Track Properties
 
-There are instances when track properties need to be added to specific tracks within the VOD manifest. This usually occurs when custom track descriptions or track roles need to be set. The Task Engine supports adding track properties to audio and subtitle tracks. Filtering of tracks is based on type (`audio` or `textStream`) and a combination of language and/or track role. The filters and values can be set in Vualto's Central Configuration so they can easily be applied to all VODs being captured or ingested. They can also be defined as part of the job submission. The value set will overwrite the existing value for the property, if it already exists. Below are some samples of how the filters can be defined.
+There are instances when track properties need to be added to specific tracks within the VOD manifest. This usually occurs when custom track descriptions or track roles need to be set. The Task Engine supports adding track properties to audio and subtitle tracks. Filtering of tracks is based on type (`audio` or `textstream`) and a combination of language and/or track role. The filters and values can be set in Vualto's Central Configuration so they can easily be applied to all VODs being captured or ingested. They can also be defined as part of the job submission. The value set will overwrite the existing value for the property, if it already exists. Below are some samples of how the filters can be defined.
 
 Setting the defined track description where the audio language is not set or set to `und` (undefined).
 
@@ -229,7 +237,7 @@ Setting the defined track role and description to the subtitle track where the l
 
 ```json
 "track_properties": {
-  "textStream": {
+  "textstream": {
     "eng": {
       "track_role" : "caption",
       "track_description": "English CC"
@@ -254,7 +262,7 @@ Setting a combination of properties to both audio and subtitle tracks.
       "track_role" : "main",
     }
   },
-  "textStream": {
+  "textstream": {
     "eng": {
       "track_role" : "caption",
       "track_description": "English CC"

docs/source/TaskEngineWorkflows.md

@@ -38,7 +38,7 @@ This workflow will generate a VOD asset from an offline source (eg. MP4). A serv
 | encode_source     |No | This boolean indicates whether the source is to be encoded into multiple bitrates/resolutions. | false |
 | encoding_profile  |No | This is used to indicate which encoding profiles are used when encoding the source. | "H264" |
 | encoding_mode     |No | This is used to indicate which Bitmovin encoding mode is used (See [here](https://bitmovin.com/bitmovin-video-encoding-v2/) for more details). | "STANDARD" |
-
+| encoding_region   |No | This is used to indicate in which region Bitmovin's encoding process should be executed. |  |
 
 ### Vodstream: JSON Payload example
 
@@ -785,6 +785,7 @@ Task callbacks are triggered after each task within a workflow is completed. Bel
 | event             | This will identify the event that caused the callback to be triggered. It can be one of `start`, `complete` or `fail`. |
 | content_id        | Content ID provided when the job was submitted. |
 | message           | Any message associated with the event. This will usually contain exception messages. |
+| files   |         | List of assets deleted |
 
 #### Job Callback
 

docs/source/conf.py

@@ -38,7 +38,7 @@ def setup(app):
 html_context = {
     'home_link': 'https://vualto-public-documentation.readthedocs-hosted.com',
     'product_logo': '',
-    'base_breadcrumb_label': 'Task-Engine'
+    'base_breadcrumb_label': 'Task Engine'
 }
 
 # The short X.Y version

docs/source/index.rst

@@ -1,4 +1,4 @@
-Welcome to the Task-Engine's documentation!
+Welcome to the VUALTO Task Engine documentation!
 ===========================================
 
 .. toctree::
@@ -16,3 +16,9 @@ Contents:
    TaskEngine.md
    TaskEngineWorkflows.md
    TaskEngineAdditionalFeatures.md
+
+.. toctree::
+   :maxdepth: 2
+   :hidden:
+
+   ReleaseNotes.md