Agora : Highlighting the Active Speaker using the Agora Flutter SDK

Satyam Parasa is a web and mobile application developer from India. He loves learning new Technology. The tech enthusiasm led him to make a blog on Flutter named flutterant.com.

For a meeting or video chat to work well, several factors must come into play. The Internet connection must be stable, the speaker's words must be clearly heard, and everyone should be aware of who is speaking. We often get confused about who is speaking, which may lead to miscommunication. So highlighting a speaker is important for every video calling program.

When you build a video calling app with the Agora SDK, the development becomes easy. In this Nicely organized tutorial, we are going to implement a feature that highlights a speaker in a call with the Agora Flutter SDK.

Prerequisites

• Basic knowledge of Flutter
• Any IDE (for example, Android Studio or Visual Studio Code)
• Agora Flutter SDK
• Agora Developer Account- Sign up here

Project Setup

1. Create a Flutter project in your favorite IDE, and remove the boilerplate code.
2. Add the following required dependencies in the pubspec.yaml file, and install and install those dependencies by running pub get:

3. Create the file structure in the lib folder as shown below:

Project structure Implementing the Video Calling Interface

As we know the Flutter application is initialized in the main.dart file and calls the HomePage(),which is defined in the home_page.dart file:

main.dart Building the Home Page

Let's create a dart file with the name home_page.dart. This page will take the channel name as input from the user.

When the user clicks the join button, the app will ask for the camera and mic permissions from the user. If permissions are granted, then the channel name will be passed to call_page.dart:

The onJoin() method is triggered when the user clicks the join button on the home page. It validates the user input and calls the _handleCameraAndMic(..) method to get the camera and mic permissions. After getting the required permissions, the channel name will be passed to call_page.dart:

Here we used the permission_handler plug-in to request the permissions and check their status:

Building the Calling Page

Create a dart file with the name call_page.dart, Our entire functionality will be developed on this page only, This includes showing the surface views in grid view and highlighting the speaker in the call, as well as user interaction buttons such as call end, mic mute/unmute, and camera switching:

Let's have a look at the above code for understanding the variables that are used.

Our CallPage constructor holds the value of the channel name which comes from the home page:

• _engine is a reference to the RtcEngine class.
• _userMap maintains the data of the users who joined the channel.
• _muted is to gives the Boolean status of whether or not the mic is muted.
• _localUid holds the local user Uid.

As we know, the initState() method is called only once when the stateful widget is added to the widget tree. Here, we call the initialize() method:

If we look at the above code snippet, the initialize() method is responsible for calling the following methods:

• _initAgoraRtcEngine() is responsible for initializing the Rtc Engine.
• _addAgoraEventHandlers() is responsible for event handling.
• joinChannel() is responsible for joining into a specific channel.

create(..): The AgoraRtc Engine will be initialized by the create(..) method with the parameter of App ID, Our app will connect to the Agora engine by the App ID that we got from the Agora developer console (this App ID is kept in the settings.dart file).

settings.dart

enableVideo(): Enables the video module.You can call this method either before joining a channel or during a call. If you call this method before joining a channel, the service starts in video mode. If you call this method during an audio call, the audio mode switches to the video mode.

setChannelProfile(..): We are using communication as the channel profile which is the default profile in one-on-one calls or group calls.

enableAudioVolumeIndication(..): Enables the RtcEngineEventHandler.audioVolumeIndication callback at a set time interval to report on which users are speaking and the speakers' volume. This method has three parameters: interval, smooth, and report_vad:

• int interval: Sets the time interval between two consecutive volume indications. Agora recommends setting an interval that should be greater than or equal to 200 ms. If the interval is ≤ 0, the volume indication is disabled.
• int smooth: The smoothing factor sets the sensitivity of the audio volume indicator. The value ranges between 0 and 10. The greater the value, the more sensitive the indicator. Agora's recommended value is 3.
• bool report_vad: If the value is true, the RtcEngineEventHandler.audioVolumeIndication callback reports the voice activity status of the local user, Otherwise, it won't detect the voice activity status of the local user. The default value of report_vad is false.

_addAgoraEventHandler(): is responsible for the RtcEngine event callback methods, which set the event handler by calling setEventHandler(). After setting the engine event handler, we can listen to engine events and receive the statistics of the corresponding RtcEngine.

Before going to callback methods, create a dart file with the name user.dart. This is for capturing the speaking status of the user.

The event handler callback methods include:

• error() callback is for reporting an error during the SDK runtime.
• joinChannelSuccess() triggers when the local user joins a specific channel. This method returns the channel name, the uid for the user, and the time elapsed (in ms) for the local user to join the channel:

_localUid = uid;
_userMap.addAll({uid: User(uid, false)});

A local user uid is assigned to _localUid and the user data is added into a userMap with uid as the key and user object as the value, which has uid and false as the status.

• leaveChannel() initiates whenever the user leaves the channel. It returns the RtcStats, which includes call duration, number of bytes transmitted and received, latency, and so on. Here we clear the _userMap when the user leaves the channel.
• userJoined() triggers when a remote user joins a specific channel. It returns the uid for the user and the time elapsed (in ms) for the remote user to join a channel:

_userMap.addAll({uid: User(uid, false)});

Remote user data is added into a _userMap with uid as the key and user object as the value, which has uid and false as status.

• userOffline() occurs when the remote user leaves the channel. It returns the uid of the user and the reason why the user went offline. Here we are removing the user from _userMap with uid
• audioVolumeIndication() This callback is triggered at the set interval, regardless of whether a user speaks or not. It reports which users are speaking and the speakers' volume, and whether the local user is speaking. By default, this callback is disabled. We can enable it by calling the RtcEngine.enableAudioVolumeIndication method:

The local user will be detected only when the report_vad value should be set as true in theRtcEngine.enableAudioVolumeIndication method.

If the local user calls the RtcEngine.muteLocalAudioStream method, the SDK stops triggering the local user's callback. And, 20 seconds after a remote speaker calls the RtcEngine.muteLocalAudioStream method, the remote speakers' callback does not include information of this remote user. Then 20 seconds after all remote users call the RtcEngine.muteLocalAudioStreammethod, the SDK stops triggering the remote speakers' callback.

The audioVolumeIndication() method is an effective method for highlighting all active speakers. We can use theactiveSpeaker() callback to highlight one of the active speakers. Basically, the callback returns the loudest speaker out of all active speakers.

AudioVolumeCallback includes the two parameters:

• Listspeakers:An array containing the user ID and volume information for each speaker. The uid of the local user is 0.
• int totalVolume: Total volume after audio mixing. The value ranges between 0 (lowest volume) and 255 (highest volume):

The above code is for updating the _userMap based on the condition validation. If the speaking person's uid matches any uid in _userMap, that corresponding user object is updated as true and vice versa, for example,

_userMap.update(key, (value) => User(key, true))

In this way, _userMap will be updated every time and maintain the user data with the speaker's status, whether they are speaking or not in the call. So we can update the UI based on the _userMap data:

We are using GridView for showing SurfaceViews based on the _userMap data and highlighting the speaking person in the call. If the isSpeaking value true represents that the respected user is saying something in the call, then the container color changes to blue.

Widget _toolbar() is for adding the user interactive buttons at the bottom of the screen for ending the call, mic mute/unmute, and switching the camera:

• _onToggleMute is used to toggle the local audio stream by calling the _engine.muteLocalAudioStream() method, which takes a Boolean value:

• _onCallEnd is used to disconnect the call and navigate back to the home page:

• _onSwitchCamera() is used to toggle between the front and back cameras by calling the _engine.switchCamera() method:

At the end, we call the _engine.destroy() method to destroy the channel. The _engine.leaveChannel() method allows a user to leave the channel. And the dispose() method clears the users:

Testing:

Once we are done with coding, we can test the app on the device. For that, we need to run the project in our IDE:

Conclusion:

We have successfully used the Agora Flutter SDK to implement the Flutter video calling app with a feature highlighting the speaking person in a call.

Click this link for the complete source code.

Other Resources

For more about the Agora Flutter SDK, see the developer's guide here.

For more about the methods discussed in the article, click this link.

You're welcome to join the Agora Developer Slack community.

Want to build Real-Time Engagement apps?

Get started with 10,000 free minutes today!

If you have questions, please call us at 408-879-5885. We'd be happy to help you add voice or video chat, streaming and messaging into your apps.