Metrics

fl-analytics is an add-on library that serves as a framework for recording comprehensive data metrics related to application, video playback, video downloads, and user events. The library can also be used to collect custom events and metadata. The library's data model is based on the Data Dictionary which is the standard schema used to normalize every piece of data captured in order to generate many popular video streaming quality of experience metrics.

Setup

Datazoom SDK

Quickplay's Player QOE metrics are powered by Datazoom, a video data platform which continually gathers data from endpoints, like a CDN or a video player, through an ecosystem of collectors. Once the data is gathered, it is normalized using standardized data definitions. This data is then sent through available connectors to analytics platforms like Google BigQuery, Google Analytics, and Splunk and can be visualized in tools such as Looker and Superset. Datazoom is your key to a more effective and efficient data pipeline.

1. Fetch the Datazoom Roku SDK.

   git clone https://gitlab.com/datazoom/roku/dz_collector_roku_release.git

2. Extract the SDK files from latest version of the cloned repository

3. Copy the SDK files into your channel project. here is a directory tree for reference: DzLib

├── README.md
├── components
│   ├── Common.brs
│   ├── DzLib
│   │   ├── dz_adapter_roku_native
│   │   └── dz_collector_roku
├── images
├── ...

4. Through FLAnalyticsServiceFactory available with fl-analytics package, create a service instance by filling in the client specific infomation.

library file structure:

├── fl-analytics
│   └── source
│       └── library
│           ├── ApplicationSession.brs
│           ├── FLAnalyticsFactory.brs
│           ├── FLAnalyticsReporter.brs
│           ├── FLReportingData.brs
│           ├── PlaybackSession.brs
│           └── UserSession.brs

dzConfig = FLAnalyticsServiceFactory().datazoomConfig(<datazoom_key>,  "https://platform.datazoom.io",<custom_event_endpoint>)
appInfo = FLAnalyticsServiceFactory().applicationInfo(<appName>, <version>, <buildVersion>)
deviceInfo = FLAnalyticsServiceFactory().deviceInfo("Roku", "Roku", "Roku_ultra")
' The following are sample values
subInfo = FLAnalyticsServiceFactory().subscriptionInfo("subId-456","active", "subPlan-monthly", "subplanId-8989" )
userInfo = FLAnalyticsServiceFactory().userInfo("userId-123",  "userType_freeTrial", subInfo,  "profileId-123-4567-09876")
globalData = FLAnalyticsServiceFactory().globalData(appInfo, userInfo, deviceInfo)

analyticsService = FLAnalyticsServiceFactory().analyticsService(dzConfig, globalData)

note

Custom events reporting to the Analytics server can be toggled on/off remotely. Passing the custom_event_endpoint, where the black-listed items are hosted to restricts the reporting of black-listed custom events

A reference template is located at: FLAnalyticsReportService.brs with sample app:

.
├── README.md
├── components
│   ├── ...
└── source
    ├── ...
    ├── service
    │   ├── FLAnalyticsReportService.brs

FLAnalyticsServiceFactory provides the entry point to the library analytics protocols. It is a factory classs use to create all analytic sessions (ApplicationSeesion, PlaybackSession, UserSession)

Application Session

ApplicationSession provides APIs for reporting app start and errors.

The ApplicationSession instance can be accessed by:

applicationSession = analyticsService.getApplicationSession()

Playback Session

PlaybackSession provides APIs for reporting playback states and errors.

The PlaybackSession instance can be accessed by:

playbackSession = analyticsService.getPlaybackSession()

User Session

UserSession provides APIs for reporting user actions, states, and errors.

The UserSession instance can be accessed by:

userSession = analyticsService.getUserSession()

The library can support multiple analytics providers (supports Datazoom at present) by creating a configuration to a specific analytics provider. The following code snippet provides an example for Datazoom configuration:

Datazoom Configuration

Create configuration to initialize datazoom.

Name	Required	Description
Id	true	The unique Datazoom configuration key.
url	true	The URL of the Datazoom server endpoint.
configUrl	false	The URL of the Custom Datazoom events endpoint. Custom events reporting to the Analytics server can be toggled on/off remotely by passing the url where black-listed items are hosted to restricts the reporting of custom events.

dzConfig = {
           id: "40c04d29-3e15-4d26-970c-cb2daab214ce"
           url: "https://platform.datazoom.io"
           configUrl: "<custom_event_endpoint>"
           }

CommonReportingData

The Metadata that is expected to remain constant for the entire duration of a session should be provided using CommonReportingData. It encapsulates all relevant optional and mandatory metadata regarding different aspects of the AnalyticSession.

note

Use API to update any of this common reporting data.

Application

Create application reporting data payload.

Name	Type	Required	Description
name	String	true	The name of the application.
version	String	true	The version of the application.
buildVersion	String	true	The build version of the application.
product	String	false	The customer product.

User

Create user's reporting data payload.

Name	Type	Required	Description
type	UserType	true	Indicates appropriate user type.
id	String	false	The unique user identifier (eg: Customer Id)
subscription	Subscription	false	User subscription details.
profileID	String	false	The unique identifier for the user's profile

Subscription

Create Subscription reporting data payload which will be reported as part of above user's payload.

Name	Type	Required	Description
subscriptionId	String	true	The unique subscription identifier for the user.
subscriptionStatus	SubscriptionStatus	false	Indicates appropriate subscription status.
subscriptionPlan	String	false	The subscription plan type name.
subscriptionPlanID	String	false	The subscription plan type id

Device

Create user device's reporting data payload.

Name	Type	Required	Description
platformType	PlatformType	true	Represents how user consuming service (eg: APP, WEB)
customDeviceManufacturer	String	false	The manufacturer of the user's device if available.
customDeviceName	String	false	The name of the device model if available.

Create CommonReportingData that has to be reported across all the events for the entire application lifecycle.

Name	Type	Required	Description
app	Application	true	The Application instance
user	User	true	The User instance.
device	Device	true	The Device instance.

appInfo = Application("testAppName", "1.0.0", "54321")

subInfo = {
               subscriptionId: "subId-456"
               subscriptionStatus:  "active"
               subscriptionPlan: "subPlan-monthly"
               subscriptionPlanID: "subplanId-8989"
           }
 userInfo = User("userId-123",  "userType_freeTrial", subInfo,  "profileId-123-4567-09876")

 deviceInfo = Device("OTT_Roku", "Roku", "Roku_ultra")

 globalData  = CommonReportingData(appInfo, userInfo, deviceInfo)

Standard Playback Events & Metrics

Create PlaybackSession to report player related events.

playbackSession = applicationSession.createPlaybackSession()

Attach Player

PlaybackSession provides and API called attachPlayer which takes Player.rawplayer video node as parameter. This API enables the underlying analytics collection tool to access platform native player instance, therefore auto-reporting most of the playback related events.

playbackSession.resolvePlayer(Player.rawplayer)

note

Call above resolvePlayer API once player is in loading state, Listen for PlaybackState.LOADING state (refer Here).

Playback events that are auto-captured by library itself

Name	Description
playback_start	This event is reported when the video starts playing for the user.
heartbeat	This event is reported at periodic intervals (1 minute) during content playback.
media_request	This event is reported after the user invokes playback.
player_ready	Signifies when the player has been initialized and is ready for playback.
pause	This event is reported when the player is paused.
resume	This event is reported when the user begins playing after pausing the video.
buffer_start	Buffer Start identifies anytime the player has to wait for the video buffer to fill with video segments.
buffer_end	Event is reported when video starts playing again after a buffer is completed.
stall_start	Stall Start event is reported when playback of video stops because the buffer has been depleted causing an unexpected interruption for the user.
stall_end	Event reported when video starts playing again after a stall and the buffer has been replenished and playback resumes.
error	Thrown if an error occurs during content playback or retrieval of the video.
playback_complete	This event signifies that the video player has reached the end of the currently playing content.
stop	This event is reported when the player has entered a stopped state.
milestone	Reported when the playheadPosition passes a predetermined percentile milestone of the video's duration. Current milestones are fired at the 10, 25, 50, 75, 90 & 95 percentiles.
seek_start	This event records when the user interacts with time controls within the player to move forward or backward in the video timeline.
see_end	Event is reported when the player stops moving the playhead position to jump to a specific point on the timeline.
playing	The media is no longer blocked from playback, and has started playing. Reported when playback resumes from Stall, Buffering or Seek.

Report Playback Start

PlaybackSession provides API to report playback_request event.

PlaybackRequest

Content Specific metadata

Create data payload for specific type of contents as below.

SportContent

Create SportContent reporting data payload that is specific to sport event.

Name	Type	Required	Description
id	String	true	The unique identifier for content item.
type	String	true	The type of content to report.
name	String	true	The content name from CMS.
providerId	String	false	The content's provider identifier.
genre	String	false	The genre of the content.
licenseWindowStartDate	String	false	The license window start date. The clear content may not have this parameter.
sportName	String	true	The name of the sport.
leagueName	String	true	The name of the sport league.
team1Name	String	true	The name of the first sport team.
team2Name	String	true	The name of the second sport team.
leagueID	String	false	The unique identifier of the sport league.
leagueExternalID	String	false	The external unique identifier of the sport league.
leagueShortName	String	false	The short name of the sport league.
gameID	String	false	The unique identifier of the sport game.
gameExternalId	String	false	The external unique identifier of the sport game.
venue	SportVenue	false	The SportVenue instance.
team1ID	String	false	The unique identifier of the first sport team.
team1ExternalID	String	false	The external unique identifier of the first sport team
team1ShortName	String	false	The short name of the first sport team.
team2ID	String	false	The unique identifier of the second sport team.
team2ExternalID	String	false	he external unique identifier of the second sport team.
team2ShortName	String	false	The short name of the second sport team.

SportVenue

Create SportVenue reporting data payload that is specific to sport event.

Name	Type	Required	Description
venueName	String	true	The name of the sport event venue.
venueCity	String	true	The city where the sport event takes place.
venueCountry	String	true	The country where the sport event takes place.

TVShow

Create TVShow reporting data payload that is specific to TV show.

Name	Type	Required	Description
id	String	true	The unique identifier for content item.
type	String	true	The type of content to report.
name	String	true	The content name from CMS.
providerId	String	false	The content's provider identifier.
genre	String	false	The genre of the content.
licenseWindowStartDate	String	false	The license window start date. The clear content may not have this parameter.
seriesID	String	true	The unique identifier of the TV show series.
seriesName	String	true	The name of the TV show series.
seriesNumber	String	true	The season number of the TV show series.
episodeNumber	String	true	The number of one of the episodes in the TV show series.

MiscellaneousContent

Create MiscellaneousContent video content reporting data payload that belongs to categories other than SportContent or TVShow.

Name	Type	Required	Description
id	String	true	The unique identifier for content item.
type	String	true	The type of content to report.
name	String	true	The content name from CMS.
providerId	String	false	The content's provider identifier.
genre	String	false	The genre of the content.
licenseWindowStartDate	String	false	The license window start date. The clear content may not have this parameter.

note

If the content type do not fall under any of the categories like SportContent and TVShow then create Content as MiscellaneousContent.

ApplicationContainer

Create ApplicationContainer reporting data payload related to application container.

Name	Type	Required	Description
id	String	true	The unique identifier of the carousel where the content or link to the collection is located.
name	String	true	The name of the carousel where the content or link to the collection is located.

Create PlaybackRequest reporting data payloads related to video playback request event.

Name	Type	Required	Description
content	Content	true	The Content details.
playbackSource	PlaybackSource	true	The PlaybackSource details.
container	ApplicationContainer	false	The ApplicationContainer details.

container = ApplicationContainer("456", "Movies")
playbackSource = PlaybackSource(PlaybackSourceType.CAROUSEL)

// For MiscellaneousContent
contentInfo = MiscellaneousContent("12345", "movie", "Sample Content")

// For SportContent
contentInfo = SportContent("Base ball", "MBL", "Strikers", "Blasters")

// For TVShow
contentInfo = TVShow("2345", "Magic Show", "1", "5")

playbackRequest = PlaybackRequest(contentInfo, playbackSource, container)
playbackSession.start(playbackRequest)

Handling Playback Request Interruptions

Any interruptions prior to starting the playback (before creating a player) can be reported using PlaybackSession API. Example: Checks related to content authorization (content authentication, parental controls etc)

// Report Interruption start
event = "content_authorization_started"
reportingData = {"key" : "value"}

customEvent = CustomEvent(event, reportingData)

playbackSession.interruptStart(customEvent)

// Report Interruption end
event = "content_authorization_success"
reportingData = {"key" : "value"}
customEvent = CustomEvent(event, reportingData)

playbackSession.interruptEnd(customEvent)

Configure Playback Metadata

Add Playback custom metadata

PlaybackSession provides an API that inserts common metadata to be reported across all playback related events.

playbackSession.addMetadata({
  "key" : "value"
  })

Update Playback custom metadata

PlaybackSession provides an API to update common metadata to be reported across all playback related events.

playbackSession.updateMetadataRecord({
  "cc_enabled" : true
  })

Report Playback Error

To report any error caused like content-auth failure, concurrent streams max limit reached (failure from server but not from player layer as fl-analytics auto-reports player errors) can be reported using stop API in PlaybackSession. Send Error as a parameter(optional) to report playback stop on any error.

playbackSession.stop(error)

Value Added Events and Metrics

Application Events

Report Application start

ApplicationSession provides an API to report app_start event along with app start time.

ApplicationMetrics

Create ApplicationMetrics reporting data payload related to application performance metrics.

Name	Type	Required	Description
startUpTimeMs	Long	true	The time, in milliseconds, it takes to start the application.

appMetrics = ApplicationMetrics(createObject("roDateTime").asSeconds())
applicationSession.start( appMetrics )

Report Application End

ApplicationSession provides an API to report app_end event.

 applicationSession.stop()

User Events

UserSession provides the following API to report user action based events.

// Called when [User] executes "signup" flow.
userSession.signup(user: User)

// Called when [User] executes "login" flow.
userSession.login(user: User)

// Called when [User] executes "logout" flow.
userSession.logout(user: User)

// Called when [User] executes "create profile" flow.
userSession.createProfile(user: User)

// Called when [User] executes "update profile" flow.
userSession.updateProfile(user: User)

// Called when [User] executes "start subscription" flow.
userSession.startSubscription(user: User)

// Called when [User] executes "purchase subscription" flow.
userSession.purchaseSubscription(user: User)

// Called when [User] executes "change subscription" flow.
userSession.changeSubscription(user: User)

// Called when [User] executes "complete payment" flow.
userSession.completePayment(user: User)

note

It is client application responsibility to ensure that each user action is triggered in the correct sequence.

Report Custom Events

ApplicationSession provides an API to report custom events. The library provides MiscellaneousEvent names for the client application to reuse.

customEvent = CustomEvent("sample_event", mapOf("key" to "value"))
applicationSession.addEvent(customEvent)

note

1. In order to conform to the existing reporting scheme the event name should be provided in the lower case format and words should be separated by the underscore character e.g."playback_request"
2. In datazoom, any custom event will be reported as event name with custom_ prefix. (eg: playback_start as custom_playback_start)

caution

When trying to report an custom event with a name that is already reserved (the list of reserved event names is captured by QP_ANALYTICS_EVENTS), library will throw an error.

Configure Application Metadata

Add Application custom metadata

Any additional metadata that is to be attached to all application / playback / user events can be done using the the below API. All subsequent events will report this additional data once it is defined.

customMedata = {
  key1: "value1"
  key2: "value2"
}
applicationSession.addMetadata(customMedata)

Update Application custom metadata

ApplicationSession provides an API if the application wants to change any application level attribute at any point of time during the application session.

Example: If the application wants to report the changeable language that it is rendering in, a key - value pair denoting the information can be added using this method. If the language is updated at any point of time during the session, this change can be reflected using below API.

addMetadata = {
  key1: "value1"
  key2: "value2"
}
applicationSession.updateMetadata(addMetadata)

Application Errors

ApplicationSession provides APIs to report both fatal and non-fatal errors.

Report Fatal errors

Any application fatal error can be reported using below API. Send Error as a parameter(optional) to report playback stop on any error.

applicationSession.stop(error)

Report Non-fatal errors

Any application non-fatal error can be reported using below API. Send Error to report playback stop on any error.

applicationSession.addErrorEvent(error, null)

User Action Errors

Any User error can be reported using below API. Create UserError reporting data payload in case where one of user's action fails and User attribute along with it.

Create User Error

Name	Type	Required	Description
code	String	true	The unique error code.
message	String	true	The message describing the error.
description	String	false	The contextual description of the error.

userError = UserError("<error-code>", "user action failure")
user = User(UserType.SUBSCRIBED)

userSession.addErrorEvent(userError, user)

Out-of-box Player Metrics

The QPPlayer provides number of key metrics to analyze the playback performance. The FLPlayer's StreamInfo provides the player metrics info. Information can be obtained upon observing this field.

Key Player Metrics

• Playback Startup Time - The time in milliseconds from playback being started until the video actually began playing. The minimum valid value is 1 millisecond.

• Stream Bitrate - The bitrate of the stream measured in bps.

• Streaming Segment - Information about the video segment that is currently streaming. This is only meaningful for segmented video transports, such as DASH and HLS.

• Downloaded Segment - Information about the video segment that was just downloaded. This is only meaningful for segmented video transports, such as DASH and HLS.

VideoPlayerScene.brs

Observing stream info change

sub initializeFLPlayer()
    ...

    m.flPlayer.observeField(m.flPlayerFields.STREAM_INFO, "onStreamInfoChanged")
    
    ...
end sub

sub onStreamInfoChanged(event as Object)
    ? "Stream Info: ", event.getData()
end sub