Working with microphones, cameras and speakers

SignalWire Video SDK provides a simple way to deal with cameras, webcams and microphones that the user has. In this part, we will find out about getting the list of connected devices, switching between the devices and updating the list of devices as they are plugged in or out.

The Video SDK library for JS provides two different namespaces: Video and WebRTC. Video gives you access to the classes and methods that let you interface with the backend Video APIs. WebRTC contains several functions that are useful for interacting with the hardware of the user's device.

We will be using some functions from the WebRTC namespace to find out about user’s devices.

Getting the list of available devices

The SDK provides symmetrical methods to get the list of microphones, cameras and speakers. They are all available in the WebRTC namespace:

  1. SignalWire.WebRTC.getMicrophoneDevicesWithPermissions(): Promise
  2. SignalWire.WebRTC.getCameraDevicesWithPermissions(): Promise
  3. SignalWire.WebRTC.getSpeakerDevicesWithPermissions(): Promise

These functions return asynchronously so don't forget to use await.

let speakers = await SignalWire.WebRTC.getSpeakerDevicesWithPermissions();
console.log(speakers);

These functions return an array of devices in this format:

[
  {
    "deviceId": "ADciLf...NYgF8=",
    "kind": "audiooutput",
    "label": "External Speaker",
    "groupId": "rgZgKM...NW1hU="
  }
]

The deviceId attribute here will be used to uniquely identify and select between the devices internally, and the label attribute will be used as display names for the user.

Changing the active device

The SDK also provides methods to change the I/O device in use. These methods are available from the RoomSession object.

  1. RoomSession.updateMicrophone({deviceId: <device_id>})
  2. RoomSession.updateCamera({deviceId: <device_id>})
  3. RoomSession.updateSpeaker({deviceId: <device_id>})

Note that this method is called from the current room session object, not the SignalWire.WebRTC namespace.

Updating the list of devices

The user might decide to plug in headphones halfway through the meeting. More generally, the availability of devices might change during the call. It might be necessary to fall back to another device when the active device is unplugged, or update the UI to include newly plugged devices.

The SDK provides a way to register events that fire when the list of devices is updated via the createDeviceWatcher in the WebRTC namespace. Use it as follows:

let inputWatcher = await SignalWire.WebRTC.createDeviceWatcher({
  // targets can be a subset of microphone, camera and speaker
  targets: ["microphone", "camera"]
})

inputWatcher.on("added", (e)=>{
  console.log("device added", e);
})
inputWatcher.on("removed", (e)=>{
  console.log("device removed", e);
})
inputWatcher.on("updated", (e)=>{
  console.log("device updated", e);
})
inputWatcher.on("changed", (e)=>{
  console.log("device changed (fires when any of the above happens)", e);
})

The callback function is called with an object as its only parameter. For example if user disconnects their headphone, the event will fire with the following object as parameter:

{
  "changes": {
    "updated": [
      {
        "type": "updated",
        "payload": {
          "deviceId": "f6f6a63616ef07213ef4dad5aff9b97b5ef43065a8ab0282160229d09a9d2421",
          "kind": "audiooutput",
          "label": "Internal Speakers (Built-in)",
          "groupId": "b2a56e423764229dd41501a7eac393f0020b54bc37caa9fcf94528c337baf1e4"
        }
      }
    ],
    "removed": [],
    "added": []
  },
    
  "devices": [
    {
      "deviceId": "67d33f...fcea95",
      "kind": "audioinput",
      "label": "Internal Microphone (Built-in)",
      "groupId": "b2a56e...baf1e4"
    },
    {
      "deviceId": "f6f6a6...9d2421",
      "kind": "audiooutput",
      "label": "Internal Speakers (Built-in)",
      "groupId": "b2a56e...baf1e4"
    }
  ]
}

As you'll notice, the event parameter has two objects: changes and devices. changes describes the nature and the specifics of the change, whereas the devices objects has an up-to-date list of devices from the targets array. You can either patch the changes object into the current state or replace the current state with the devices object.

Integrating with React

Now that the basics have been laid, let us start to integrate this into our component.

As discussed earlier, we want to use the component as the wrapper for SignalWire Video SDK. So we will try to localise listing and updating I/O devices inside this component. To do so, we will introduce a new function prop to the component, called onRoomInit(). We will call this function from the component to notify the parent component that the room has been initialised and send the starting configuration.

// inside the <Video /> component's initializing useEffect();

let room = new SignalWire.Video.RoomSession({
  token,
  rootElement: document.getElementById('temp'), // an html element to display the video
  video: true,
})

let cameras =
    await SignalWire.WebRTC.getCameraDevicesWithPermissions();
let microphones =
    await SignalWire.WebRTC.getMicrophoneDevicesWithPermissions();
let speakers =
    await SignalWire.WebRTC.getSpeakerDevicesWithPermissions();

onRoomInit(room, cameras, microphones, speakers);

Similarly, to update the list of devices once they change, we'll introduce onRoomUpdate() prop.

let camChangeWatcher = await SignalWire.WebRTC.createDeviceWatcher({
    targets: ["camera"],
});
camChangeWatcher.on("changed", (changes) => {
    eventLogger("The list of camera devices has changed");
    onRoomUpdate({
        cameras: changes.devices
    });
});

let micChangeWatcher = await SignalWire.WebRTC.createDeviceWatcher({
    targets: ["microphone"],
});
micChangeWatcher.on("changed", (changes) => {
    eventLogger("The list of microphone devices has changed");
    onRoomUpdate({
        microphones: changes.devices
    });
});

let speakerChangeWatcher =
    await SignalWire.WebRTC.createDeviceWatcher({
        targets: ["speaker"],
    });
speakerChangeWatcher.on("changed", (changes) => {
    eventLogger("The list of speakers has changed");
    onRoomUpdate({
        speakers: changes.devices
    });
});

On the parent component of the component, we use onRoomInit and onRoomUpdate events to update crucial UI elements. The code below is for component. This is the parent component, and includes the component as well as other components. We will also start using React Bootstrap here to make things look neater, but you should be able to see that beneath the fluff, the actual actions we are performing follow naturally from above.

import React, { useCallback, useState } from "react";
import Button from "react-bootstrap/Button";
import "bootstrap/dist/css/bootstrap.min.css";
import Video from "../components/Video";
import Container from "react-bootstrap/Container";
import Row from "react-bootstrap/Row";
import Col from "react-bootstrap/Col";
import NavBar from "react-bootstrap/Navbar";

export default function InCall({ roomDetails }) {
  let [cameras, setCameras] = useState([]);
  let [microphones, setMicrophones] = useState([]);
  let [speakers, setSpeakers] = useState([]);

  let [room, setRoom] = useState({});

  let logEvent = useCallback((msg, title, variant) => {
    // a simple event logger that we will later replace
    // for a bootstrap toast
    console.log(msg, title, variant);
  }, []);

  let onRoomInit = useCallback(
    (room, cameras, microphones, speakers) => {
      setCameras(cameras);
      setMicrophones(microphones);
      setSpeakers(speakers);
      setRoom(room);
    },
    []
  );

  let onRoomUpdate = useCallback((updatedValues) => {
    if (updatedValues.cameras !== undefined) setCameras(updatedValues.cameras);
    if (updatedValues.speakers !== undefined)
      setSpeakers(updatedValues.speakers);
    if (updatedValues.microphones !== undefined)
      setMicrophones(updatedValues.microphones);
  }, []);

  function DeviceSelect({
    devices = [],
    onChange = (value) => {},
    deviceName = "device",
  }) {
    return (
      <select
        onChange={async (e) => {
          if (e.target.value !== "") onChange(e.target.value);
        }}
        defaultValue=""
      >
        <option value="" disabled hidden>
          Change {deviceName}
        </option>
        {devices.map((device) => (
          <option key={device.deviceId} value={device.deviceId}>
            {device.label}
          </option>
        ))}
      </select>
    );
  }

  return (
    <>
      <Container fluid>
        <Row className="mt-3">
          <Col
            style={{ backgroundColor: "black" }}
            className="justify-content-md-center"
          >
            {roomDetails.mod ? "Moderator" : "normal uwer"}
            <Video
              onRoomInit={onRoomInit}
              onRoomUpdate={onRoomUpdate}
              joinDetails={roomDetails}
              width={800}
              eventLogger={logEvent}
            />
          </Col>
        </Row>
      </Container>

      <NavBar fixed="bottom">
        <Container fluid className="justify-content-md-center">
          <Row>
            <Col md="auto">
              <DeviceSelect
                onChange={(id) => {
                  room.updateCamera({ deviceId: id });
                }}
                deviceName="Camera"
                devices={cameras}
              />
            </Col>

            <Col md="auto">
              <DeviceSelect
                onChange={(id) => {
                  room.updateMicrophone({ deviceId: id });
                }}
                deviceName="Microphone"
                devices={microphones}
              />
            </Col>

            <Col md="auto">
              <DeviceSelect
                onChange={(id) => {
                  room.updateSpeaker({ deviceId: id });
                }}
                deviceName="Speaker"
                devices={speakers}
              />
            </Col>

            <Col md="auto">
              <Button
                onClick={async () => {
                  await room.leave();
                }}
                variant="danger"
              >
                Leave
              </Button>
            </Col>
          </Row>
        </Container>
      </NavBar>
    </>
  );
}
import React, {
    useEffect,
    useRef,
    useState
} from "react";
import axios from "axios";
import * as SignalWire from "@signalwire/js";

export default function Video({
    onRoomInit = () => {},
    width = 400,
    joinDetails: roomDetails = {
        room: "signalwire",
        name: "JohnDoe",
        mod: false,
    },
    eventLogger = (msg) => {
        console.log("Event:", msg);
    },
}) {
    let [setupDone, setSetupDone] = useState(false);
    let thisMemberId = useRef(null);

    useEffect(() => {
        if (setupDone) return;
        setup_room();
        async function setup_room() {
            setSetupDone(true);
            let token, room;
            try {
                token = await axios.post("/get_token", {
                    user_name: roomDetails.name,
                    room_name: roomDetails.room,
                    mod: roomDetails.mod,
                });
                console.log(token.data);
                token = token.data.token;

                try {
                    try {
                        room = await SignalWire.Video.createRoomObject({
                            token,
                            rootElementId: "stream",
                            video: true,
                        });
                    } catch (e) {
                        console.log(e);
                    }
                    room.on("room.joined", async (e) => {
                        thisMemberId.current = e.member_id;
                        eventLogger("You have joined the room.");
                    });
                    room.on("room.updated", async (e) => {
                        eventLogger("Room has been updated");
                    });
                    room.on("member.joined", async (e) => {
                        eventLogger(e.member.name + " has joined the room.");
                    });
                    room.on("layout.changed", async (e) => {
                        eventLogger("The layout was changed to", e.layout.name);
                    });
                    room.on("member.left", async (e) => {
                        let memberList = await room.getMembers();
                        let member = memberList.current.filter((m) => m.id === e.member.id);
                        if (member.length === 0) {
                            return;
                        }
                        eventLogger(member[0]?.name + " has left the room.");
                    });

                    await room.join();

                    let cameras =
                        await SignalWire.WebRTC.getCameraDevicesWithPermissions();
                    let microphones =
                        await SignalWire.WebRTC.getMicrophoneDevicesWithPermissions();
                    let speakers =
                        await SignalWire.WebRTC.getSpeakerDevicesWithPermissions();

                    onRoomInit(room, cameras, microphones, speakers);

                    let camChangeWatcher = await SignalWire.WebRTC.createDeviceWatcher({
                        targets: ["camera"],
                    });
                    camChangeWatcher.on("changed", (changes) => {
                        eventLogger("The list of camera devices has changed");
                        onRoomUpdate({
                            cameras: changes.devices
                        });
                    });
                    let micChangeWatcher = await SignalWire.WebRTC.createDeviceWatcher({
                        targets: ["microphone"],
                    });
                    micChangeWatcher.on("changed", (changes) => {
                        eventLogger("The list of microphone devices has changed");
                        onRoomUpdate({
                            microphones: changes.devices
                        });
                    });
                    let speakerChangeWatcher =
                        await SignalWire.WebRTC.createDeviceWatcher({
                            targets: ["speaker"],
                        });
                    speakerChangeWatcher.on("changed", (changes) => {
                        eventLogger("The list of speakers has changed");
                        onRoomUpdate({
                            speakers: changes.devices
                        });
                    })

                } catch (error) {
                    console.error("Something went wrong", error);
                }
            } catch (e) {
                console.log(e);
                alert("Error encountered. Please try again.");
            }
        }
    }, [roomDetails, eventLogger, onRoomInit, onRoomUpdate, setupDone]);
    return (<div id = "stream" style = {{width}}/>);
}

Did this page help you?