Oct 7, 2022
Building a Twitter Spaces clone in Flutter
A walkthrough of how we built SpeakUp, a clone of Twitter Spaces we created for conducting spaces of our internal communities.
Ashish Rajesh GourSoftware Engineer - II
Introduction
Audio streaming has garnered a lot of attention recently. You’ve heard of audio streaming platforms such as Clubhouse and Twitter Spaces. As part of our flutter community initiatives at Geekyants, we started experimenting with 100ms.live and built a clone around Twitter Spaces.
Twitter Spaces is a new feature that allows users to have live audio conversations on the platform. Users can host these conversations in an audio chat room called a "Space" and invite other users to participate.
This article will demonstrate how to build a Twitter Spaces clone with the 100ms Flutter SDK.
Why 100ms?
100ms is a cloud-based live video infrastructure software that allows developers to create video and audio conferencing applications on the web, iOS, and Android platforms using the 100ms software development kit (SDK), REST APIs, and account dashboard. The platform provides an easy and flexible kit to enable developers to create awesome, feature-rich, and custom-built video and audio conferencing applications without breaking much sweat.
Prerequisites
To follow this tutorial, you must have a basic understanding of
- Flutter with state management libraries such as Bloc.
- Firebase.
- 100ms server-side APIs ( 100ms.live/docs/server-side/v2/introduction.. )
Basic Terms to Know in the 100 ms SDK
Basic Terms to Know in the 100 ms SDK
Before starting, let’s get acquainted with some common terms that will be frequently used throughout this article :
Room: A room is a virtual space within which the audio-video interaction between peers occurs. To allow users to join a 100ms audio/video conferencing session inside your app, you must first create a room. The room is the basic object that 100ms SDKs return on a successful connection. You can create a room using either the dashboard or via API.
Peer: A peer is an object returned by 100ms SDKs, containing all information about a user ( name, role, audio/video tracks, etc ).
Role: A role is a collection of permissions that allows you to perform a specific set of operations while being part of a room. An audio room can have roles such as speaker, moderator, or listener, while a video conference can have roles such as host and guest.
Track: A track represents either the audio or video published by a peer.
Setting up the project
Create A Template : To get started on this project, create an account on the 100ms dashboard. Moving forward, we would create a template where we will define the roles and settings, simply select “Video Conferencing”. The roles to be defined for this project are the host and guest. Each role has its permissions and settings with the host having the highest set of permissions.
Create A Room : Now, we have to create a room based on the template created above, as earlier mentioned, a room is the object returned when a connection to the SDK is successful, it defines the place where you and other participants meet and interact in the app. To create a room, go to the Rooms tab and click on “Create A Room”, and input the room name. On successful creation of a room, a room id is automatically generated which will be useful while rendering the rooms on UI.
Integrate the SDK
To get started, we’ll need to add the Flutter package for 100ms SDK. You can find it on pub.dev or run the following command in the terminal:
flutter pub add hmssdk_flutter
We’ll also need to add a few other packages to pubspec.yaml file namely:
Setting up services
Create a new folder services with a file sdk_initializer.dart in it.
Add the following code to initialize the SDK:
Add Permissions
As discussed in the above terms a track represents either the audio or video that a peer is publishing.
You will require Recording Audio and Internet permission in this project as we are focused on the audio track in this tutorial.
Android Permissions
Add the following permissions in your AndroidManifest file (android/app/src/main/AndroidManifest.xml):
iOS Permissions
Add the following permissions to your Info.plist file:
Getting the permissions from users
To use the camera or microphone in our app, we need to get the required permissions from the users.
You can use the permission_handler package to do that easily.
Call the getPermissions() function inside the initState() of your Stateful HomeScreen.
Implement Listener
Now, we have to implement a listener class over the current SDK which will help us to interact with the SDK easily.
HMSUpdateListener listens to all the updates happening inside the room. 100ms SDK provides callbacks to the client app about any change or update happening in the room after a user( peer) has joined by implementing HMSUpdateListener.
When someone requests a track change for video or audio this will be triggered. Where [hmsTrackChangeRequest] request instance consists of all the required info about track changes.
This will be called on a successful joining of the room by the user.
This is called when there is a new broadcast message from any other peer in the room. This can be used to implement chat in the room.
This will be called whenever there is an update on an existing peer, a new peer got added or an existing peer is removed. This callback can be used to track all the peers in the room.
This method is called when a peer is back in the room after reconnection.
This method is called when a network or some other error happens.
This method is called when the host removes you or when someone ends the room at that time, it gets triggered.
This method is called when someone requests a change of role.
This is called when there is a change in any property of the Room.
This is called when there are updates on an existing track or a new track got added/an existing track is removed. This callback can be used to render the video on the screen whenever a track gets added.
Note: An HMSSpeaker object contains -
- peerId: The peer identifier of HMSPeer who is speaking.
- trackId: The track identifier of HMSTrack which is emitting audio.
- audioLevel: a number within range 1-100 indicating the audio volume.
This will be called when there is an error in the system and SDK has already retried to fix the error.
Joining the Room
Now, to join an ongoing space, we can call the onJoin() method on HMSSDK with the config settings. This would require an authentication token and a room id.
To get an auth token, we need to send an HTTP post request to the Token endpoint which can be obtained from the dashboard manually.
Go to Developer -> Copy Token endpoint (under Access Credentials)
A successful HTTP post would return us a token that can be passed to the join method of HMSSDK as config.
On success, JoinRoomState() will be fired by the bloc which will render the updated UI.
✅ If successful, the function onJoin(room: HMSRoom) method of HMSUpdateListener will be invoked with details about the room containing in the HMSRoom object.
❌ If failure, the function onError(error: HMSException) method will be invoked with failure reason.
Creating a Room
For creating a room add the following lines of code in room_repository.dart file.
When an user presses the create room button, we will send the CreateRoomEvent to the RoomBloc to handle it and emit the RoomCreated State.
✅ On success, RoomCreatedState() will get triggered and a new room card will be rendered on the home screen.
Render the Peers
A peer is an object returned by 100ms SDKs that hold the information about a user in room ( name, role, track, raise hand, message, etc.)
There a two types of peers available:
- local peer
- remote peer
You can access the localPeer and remotePeers from hmssdk as shown in the code below:
Whenever a user (peer) rejoins, leaves or send message to another peer in a room OnJoinRoomEvent gets triggered which emits the UpdateRoomUI state which updates the UI.
End Room
- When the user (remotePeers) clicks on the End room button EndRoomEvent gets triggered and they will be redirected back to the home screen.
- When the host (localPeer) clicks on the End room button EndRoomByRemoteHostEvent gets triggered which will remove the room from the home screen for all the peers.
For sending the request to the host we are using sendDirectMessage() method provided by hmssdk as show in the code below:
The host can see the list of all the peers who have raised their hands to become a speaker in an ongoing space. The host has the ability to accept or reject the request.
When the host accepts the request of any peer _approveSpeakerRequest() method is invoked. For sending the approved request to the peer we are using sendBroadcastMessage() method provided by hmssdk as show in the code below:
Mute/ Unmute Audio
To mute or unmute the mic feature use the below code:
Chat Feature
The final feature we would be implementing is the Chat feature, which enables users to send messages to everyone present in the room, as well as get a reply from anyone. 100ms supports chat feature for every video/audio room you create.
100ms provides three ways of addressing messages :
Broadcast messages are sent to Everyone in the chat ( hmsSDK.sendBroadcastMessage).
Direct messages are sent to a specific person( hmsSDK.sendDirectMessage).
Group messages are sent to everyone with a particular HMSRole (hmsSDK.sendGroupMessage).
For more detailed information on implementing chat feature refer to the following link: docs
We are using sendBroadcastMessage() method for implementing chat feature for all the participants in the room.
Report a Speaker
The users can also report a speaker by clicking on the Report Speaker button on the room details screen. The data of the reported speaker will be stored in our backend (Firestore database).
Conclusion
Congratulations for making it till the end of the article. In this article, you learned about 100ms and how you can easily integrate 100ms to build an audio room application such as "SpeekUp". However, this is only the beginning, you can learn more about changing roles, changing tracks, adding peers, direct chat with another peer, and much more from 100 ms official docs .
My overall experience working with 100ms SDK in flutter was really great as it allows us to add live audio/video, chat and many more features in our app with very less lines of code and handles all the functionality in the SDK. It supports Web, Android and iOS platforms which makes it more convincing to use.
I would recommend you guys check it out and build your own live apps!
Future Aspects
- As discussed above, 100ms also provides us with live video conferencing as well as screen sharing features which can also be integrated into our application.
- We can also implement a feature of scheduling spaces and the promote/demote roles of peers.