Merge pull request #18 from Vualto/te-new-template-changes

Te new template changes

docs/source/DeveloperDocumentation.rst

@@ -0,0 +1,15 @@
+DEVELOPER DOCUMENTATION
+=============================================
+
+As mentioned in the index page the Task Engine facilitates the creation of VOD from online and offline sources. This supports products such as VCH, Clip2VU and Media Syndication.
+
+.. toctree::
+   :maxdepth: 2
+
+   DeveloperDocumentation/Integration.md
+   DeveloperDocumentation/TaskEngineAPI.md
+   DeveloperDocumentation/TaskEngineWorkflowFeatures.md
+   DeveloperDocumentation/TaskEngineWorkflows.md
+   VCH API<https://docs.vualto.com/projects/VIS/en/latest/>
+   CLIP2VU <https://docs.vualto.com/projects/clip2vu/en/latest/>
+   MEDIA SYNDICATION<https://docs.vualto.com/projects/media-syndication/en/latest/>

docs/source/DeveloperDocumentation/Integration.md

@@ -0,0 +1,51 @@
+# INTEGRATION
+
+There are two main integration points for the Task Engine:
+
+**API** – how jobs are submitted to the Task Engine.
+
+**Callbacks** – how the Task Engine notifies client systems of job progress.
+
+### API
+
+The API is mainly used to trigger [workflows](TaskEngineWorkflows.md) within the Task Engine but additional API endpoints are available for job management. Full API documentation can be found [here](TaskEngineAPI.html)
+
+### CALLBACKS
+
+Callbacks are used to notify a integrated services with workflow execution updates. Callback URLs are submitted as part of the payloads and the Task Engine will send callbacks when:
+
+1. A task starts
+2. A task ends (success or fail)
+3. A job ends (success or fail)
+
+All default task callbacks will the contain the same JSON body structure:
+
+```json
+"job_id": "<job id>",
+"task_id": "<task id>",
+"task_name": "<task name>",
+"workflow": "<workflow name>",
+"event": "<task event>",
+"content_id": "<content_id>",
+"message": "<task exception message>"
+```
+
+Job callbacks vary depending on the workflow being executed but the following are common in all workflows.
+
+```json
+"job_id": "<job id>",
+"status": "<job status>",
+"workflow": "<workflow name>",
+"content_id": "<content id>",
+"custom_data": "<client custom data>"
+```
+
+More information of the callbacks for each workflow can be found [here](TaskEngineWorkflows.md).
+
+Specifying the Vualto Control Hub Video Information Service web-hook (`https://vis.controlhub.[client].vualto.com/api/event/vuflow/taskenginecallback`) as a callback url, will be add the asset as a VOD event within the Vualto Control Hub CMS. A second CMS web-hook (`https://admin.controlhub.[client].vualto.com/vod/PublishVuflowData`) can also be included for realtime updates on the status of the job.
+
+As Task Engine is an integration product that can be customised, any specific requirements are easily catered for (eg. setting authentication headers) and the body of a callbacks may also be modified.
+
+## AUTHENTICATION
+
+All Task Engine API calls that require authentication currently use the client name and API key provided by Vualto. The credentials should be supplied as `client` and `api-key` headers respectively.

docs/source/TaskEngineAPI.md → docs/source/DeveloperDocumentation/TaskEngineAPI.md

@@ -2,7 +2,7 @@
 
 The Task Engine endpoints will always return a JSON response unless explicitly indicated otherwise.
 
-## Status Endpoints
+## STATUS ENDPOINTS
 
 ### GET: `/`
 
@@ -199,7 +199,7 @@ The dashboard endpoint returns information about the current Task Engine queue s
 
 </details><br /><br />
 
-## Job Endpoints
+## JOB ENDPOINTS
 
 ### POST: `/job`
 
@@ -466,7 +466,7 @@ Successful Response:
 
 </details><br /><br />
 
-## Log Endpoints
+## LOG ENDPOINTS
 
 ### GET: `/logs/<job id>`
 
@@ -564,7 +564,7 @@ Successful Response:
 
 </details><br /><br />
 
-## Scheduler Endpoints
+## SCHEDULER ENDPOINTS
 
 ### GET: `/schedules`
 
@@ -664,7 +664,7 @@ Successful Response:
 
 </details><br /><br />
 
-## Settings Endpoints
+## SETTINGS ENDPOINTS
 
 ### POST: `/settings`
 

docs/source/TaskEngineWorkflowFeatures.md → docs/source/DeveloperDocumentation/TaskEngineWorkflowFeatures.md

@@ -1,6 +1,6 @@
 # TASK ENGINE WORKFLOW FEATURES
 
-## Priority
+## PRIORITY
 
 The Task Engine supports ordering of jobs by priority. The priority parameter can be submitted as part of the json payload being submitted. The priority is in ascending order as follows:
 
@@ -42,7 +42,7 @@ The advantage of using priority slots is to stop the queue from being held up by
 
 **Important note**: This will not increase the number of max concurrent jobs but it will reserve some fo the concurrency for jobs with priority between 1 and 5. As an example, if a setup has 5 maximum concurrent jobs and the priority slots is set to 2, any job can ustilise 3 concurrency slots but only jobs with priority between 1 and 5 can use the 2 priority slots.
 
-## Stitching Clips
+## STITCHING CLIPS
 
 The Task Engine includes a feature that will allow multiple clips to be stitched together into a single clip, in a single job. This can be done by defining multiple objects within the `"clips"` parameter in the json payload for [VOD Capture](TaskEngineWorkflows.md#vod-capture). This also allows a mixture of live and VoD sources to be captured and stitched together into a new clip. The example below shows how the `"clips"` parameter would need to be provided to achieve this.
 
@@ -79,7 +79,7 @@ The Task Engine includes a feature that will allow multiple clips to be stitched
 }
 ```
 
-## Multiple Sources
+## MULTIPLE SOURCES
 
 In some cases, a live stream could have multiple origins setup (eg. for load balancing the origin servers). The Task Engine, allows for both streams to be defined as the source for a capture. It is smart enough to find which live stream will provide the best output capture and use that stream as the source. If the Task Engine discovers discontinuities within the streams, it will use segments from both streams to try and generate a clip with the least number of missing fragments.
 
@@ -144,13 +144,13 @@ In this case, `"sources"`  replaces the `"source"` parameter, however; it can st
 }
 ```
 
-## Generate Download Clips
+## GENERATE DOWNLOAD CLIPS
 
 The Task Engine [VOD Capture](TaskEngineWorkflows.md#vod-capture) workflow supports generating download clips without creating VoD assets. This is done by setting the property `"generate_vod"` to false and `"generate_mp4"` to true. It is important that if `"generate_vod"` is set to false, to not manually override the `"create_dref"` parameter. Setting `"create_dref"` to true will lead to a failed workflow as this requires VoD assets to generate DREF mp4s.
 
 The resulting download will be an MP4 containing all the video, audio and caption tracks defined using the clip's `"filter"` parameter. If no filter is defined, the resulting MP4 will contain all the tracks available in the stream.
 
-## Scheduler
+## SCHEDULER
 
 The Task Engine supports scheduling of jobs via the `run_at` and `sempahore_url` job attributes. Jobs are moved from a queue_state of `scheduled` to a queue_state of `queued` via a scheduler-worker. The interval at which this runs is pulled from the database settings table (schedule_interval, default: 1 hour).
 
@@ -203,7 +203,7 @@ In the above example the job will be queued at 10:00AM on the 6th of June 2040 a
 
 If a capture is submitted with a clip end time that is in the future, it will be automatically scheduled to run at the end time of the clip which is furthest in the future. The exception to this is if the `run_at` time is specified and is further in the future than the end time, then the `run_at` time will be used.
 
-## Track Properties
+## TRACK PROPERTIES
 
 There are instances when track properties need to be added to specific tracks within the VOD manifest. This usually occurs when custom track descriptions or track roles need to be set. The Task Engine supports adding track properties to audio and subtitle tracks. Filtering of tracks is based on type (`audio` or `textstream`) and a combination of language and/or track role. The filters and values can be set in Vualto's Central Configuration so they can easily be applied to all VODs being captured or ingested. They can also be defined as part of the job submission. The value set will overwrite the existing value for the property, if it already exists. Below are some samples of how the filters can be defined.
 
@@ -272,7 +272,7 @@ Setting a combination of properties to both audio and subtitle tracks.
 
 Support is confirmed for `track_description`, `track_role` and `track_name` properties, but other properties may be supported.
 
-## Storage Support
+## STORAGE SUPPORT
 
 The Task Engine supports multiple storage types for ingesting content and saving VOD. Support has also been added so a combination of storage types can be used for the same job. This can be done by setting the `source_storage` and `destination_storage` in the job payload (for supported workflows). Eg. Ingesting content from local storage and save VOD assets on S3 would require `source_storage` to be set to `local` and `destination_storage` to be set to `S3`.
 
@@ -285,7 +285,7 @@ Natively supported storage types:
 - On premises infrastructure (`local`)
 
 
-## Preview Thumbnails
+## PREVIEW THUMBNAILS
 
 Preview thumbnails refers to the thumbnails that appear on the a video player's timeline as the user hovers over the progress bar. These can be generated on 3 different occasions:
 

docs/source/TaskEngineWorkflows.md → docs/source/DeveloperDocumentation/TaskEngineWorkflows.md

@@ -1,6 +1,6 @@
 # TASK ENGINE WORKFLOWS
 
-## VOD Stream
+## VOD STREAM
 
 This workflow will generate a VOD asset from an offline source (eg. MP4). A server side manifest is created, with and/or without DRM, that can be used for on the fly delivery of VOD content via the Unified Streaming Platform.
 
@@ -112,7 +112,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | files             | List of files (manifests, content files, thumbnail, etc...) that have been copied to the final destination. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## VOD Capture
+## VOD CAPTURE
 
 This workflow allows you to create a frame accurate VOD clip by passing in a start and end time. If the source stream contains time stamps, UTC time stamps can be used for the start and end times. The result will be a new VOD asset and/or a downloadable MP4.
 
@@ -230,7 +230,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | files             | List of files (manifests, content files, thumbnail, etc...) that have been copied to the final destination. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## VOD Delete
+## VOD DELETE
 
 This workflow allows you to a delete VOD asset from storage.
 
@@ -293,7 +293,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | message           | Name of the folder deleted from storage. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## DRM Switch
+## DRM SWITCH
 
 This workflow allows you to toggle DRM on and off for a VOD asset. Missing manifests will be generated when required. If the VOD asset does not have a DRM manifest and DRM is being enabled, a list of DRM systems needs to be provided as part of the payload.
 
@@ -362,7 +362,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | message           | Full path of the active manifest, for the generated content. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## Create MP4
+## CREATE MP4
 
 This workflow allows you to create an MP4 from a VOD asset.
 
@@ -433,7 +433,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | files             | List of files uploaded to the destination storage. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## Build Thumbnails
+## BUILD THUMBNAILS
 
 This workflow allows you to generate thumbnail assets which can then be used for video timeline previews.
 
@@ -502,7 +502,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | message           | List of thumbnail assets uploaded to the destination storage. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## VOD Remix
+## VOD REMIX
 
 This workflow allows you to create a virtual VOD asset that is just a playlist referencing other VOD streams or video files.
 
@@ -659,7 +659,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | output            | List of files (manifests, content files, thumbnail, etc...) that have been copied to the final destination. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## Generate GIF
+## GENERATE GIF
 
 This workflow allows you to create animated GIFs from a VOD stream.
 
@@ -742,7 +742,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | files             | List of files uploaded to the destination storage. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## Capture Frame
+## CAPTURE FRAME
 
 This workflow allows you to capture a single frame from a stream.
 
@@ -812,7 +812,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | files             | List of files uploaded to the destination storage. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## Asset Delete
+## ASSET DELETE
 
 This workflow allows you to delete individual assets without deleting an entire VOD directory.
 
@@ -982,7 +982,7 @@ Job callbacks are triggered when the entire job has completed. Below is a list o
 | segments          | Segments used for the VOD asset. |
 | custom_data       | Returns the custom data submitted to the workflow. |
 
-## Workflow Trigger Example
+## WORKFLOW TRIGGER EXAMPLE
 
 Example of a curl command to trigger ingest for the [VOD Stream](#vod-stream) workflow: