View as Markdown Open in ChatGPT Open in Claude

useAvatar

Control the avatar and read its live status from your own components.

useAvatar gives you everything you need to build custom controls: methods to start, stop, and send speech, plus live status flags like isSpeaking and isConnected that update as the session changes.

Usage

Call it inside any component nested under <AvatarProvider>.

tsx

import { Text, Button } from "react-native";
import { useAvatar } from "@avatarfactory/react-native";

function Controls() {
  const { start, stop, isIdle, isReady } = useAvatar();

  if (!isReady) return <Text>Loading…</Text>;

  return (
    <Button
      title={isIdle ? "Start conversation" : "End session"}
      onPress={isIdle ? start : stop}
    />
  );
}

Full return signature

tsx

const {
  // — Avatar (paint axis) —
  isReady, isLoading, isFailed,

  // — Session —
  isIdle, isConnecting, isConnected,

  // — Turn —
  isListening, isThinking, isSpeaking, isResponseSlow,

  // — Mute —
  isMicMuted, isSpeakerMuted, setMicMuted, setSpeakerMuted,

  // — Push to talk —
  isPushToTalk, isTalking, startTalking, stopTalking,

  // — Live data —
  avatarId, error, notice, health, transcript, controls,
  availableActions,

  // — Event subscription —
  on, off,

  // — Session control —
  start, stop, interrupt, stopSpeaking,

  // — Mode-specific methods —
  speakText,      // tts mode only
  speakAudio,     // audio mode only
  play,           // player mode only
  addLiveContext, // call mode
  triggerAction,  // fire a named rig gesture
} = useAvatar();

State properties

Prop	Type	Default	Description
`isReady`Optional	`boolean`	No default	The avatar is on screen and animating. Independent of the call — a loaded avatar is ready before a call starts, and stays ready after one ends.
`isLoading`Optional	`boolean`	No default	The avatar is still coming up — being resolved, or resolved and still rendering.
`isFailed`Optional	`boolean`	No default	The avatar could not be loaded, so nothing is on screen. start() tries again.
`isConnecting`Optional	`boolean`	No default	start() is in flight — true for the whole window from the tap until the call is live.
`isConnected`Optional	`boolean`	No default	True while a session is in progress — the WebSocket is open and the server has acknowledged the session start.
`isSpeaking`Optional	`boolean`	No default	The avatar is actively speaking and animating.
`isListening`Optional	`boolean`	No default	The avatar is listening for user input (call mode).
`isThinking`Optional	`boolean`	No default	The avatar is working out its reply, between listening and speaking.
`isResponseSlow`Optional	`boolean`	No default	The current turn is taking unusually long. Advisory — the turn is fine, just slow. React Native ships no built-in indicator, so render your own "still working" hint from this. Auto-clears when speech starts.
`isIdle`Optional	`boolean`	No default	No call in progress; start() is available. True while the avatar is still loading too — start() waits the load out for you.
`avatarId`Optional	`string \| undefined`	No default	The currently loaded avatar's ID.
`error`Optional	`AvatarError \| null`	No default	The most recent error, or null. Sticky — it stays until the next start(), and never clears on a timer.
`controls`Optional	`{ enabled: boolean; stopSpeaking?: boolean }`	No default	Whether the built-in control bar is shown, and whether it includes the stop-speaking button.

Mute

Prop	Type	Default	Description
`isMicMuted`Optional	`boolean`	No default	The user's mic is muted — outbound audio is dropped, so the server genuinely hears silence. Call mode.
`isSpeakerMuted`Optional	`boolean`	No default	The avatar's audio is muted. Playback is silent but lip-sync keeps animating.
`setMicMuted`Optional	`(muted: boolean) => void`	No default	Mute or unmute the user's microphone.
`setSpeakerMuted`Optional	`(muted: boolean) => void`	No default	Mute or unmute the avatar's spoken audio.

Push to talk

Set turnTaking: { mode: "push-to-talk" } on the config, then wire a Pressable. Pressing cuts the avatar off if it was speaking, which is what makes barge-in work.

Prop	Type	Default	Description
`isPushToTalk`Optional	`boolean`	No default	The session is running in push-to-talk. Static for the session.
`isTalking`Optional	`boolean`	No default	The user is holding the talk button, so their audio is reaching the server.
`startTalking`Optional	`() => void`	No default	The press: start uploading mic audio. No-op outside push-to-talk or with no live session.
`stopTalking`Optional	`() => void`	No default	The release: end the user's turn. No-op if no turn is open.

tsx

<Pressable
  onPressIn={startTalking}
  onPressOut={stopTalking}   // also fires when the touch is cancelled
  accessibilityState={{ selected: isTalking }}
>
  <Text>{isTalking ? "Listening…" : "Hold to talk"}</Text>
</Pressable>

Live data

Prop	Type	Default	Description
`notice`Optional	`SessionNotice \| null`	No default	The last server notice (plan limit, demo cap, expiring test key). Not an error — the session is ending gracefully, and notice.message is written for your users.
`health`Optional	`ConnectionHealth`	No default	Live connection health. health.concern is the blame-resolved state to render from (null when all is well).
`transcript`Optional	`TranscriptEntry[]`	No default	Conversation lines, oldest first. Empty unless transcript.enabled.
`availableActions`Optional	`string[]`	No default	Gesture names the loaded rig advertises. Pass one to triggerAction().

Methods

start()

Start the avatar session. In call mode, this opens a WebSocket connection and requests microphone permission. In tts and audio modes, the connection is established automatically upon loading; start() can be used to re-connect if the session was manually stopped.

tsx

const { start } = useAvatar();
<Button title="Begin" onPress={start} />

stop()

Close the session and disconnect from the server.

tsx

const { stop } = useAvatar();
<Button title="End" onPress={stop} />

interrupt()

Immediately stop the avatar mid-sentence. The avatar returns to listening state.

tsx

const { interrupt, isSpeaking } = useAvatar();
{isSpeaking && <Button title="Stop" onPress={interrupt} />}

speakText(sentence)

tts mode only. Send a text string — the avatar speaks it with full lip-sync.

tsx

const { speakText } = useAvatar();
speakText?.("Welcome! How can I help you today?");

speakAudio(audio)

audio mode only. Feed raw audio data. The avatar lip-syncs to it.

tsx

const { speakAudio } = useAvatar();

// Accepts: ArrayBuffer | Float32Array | Blob — raw PCM audio, 16 kHz mono
speakAudio?.(buffer);

play(job)

player mode only. Play a pre-rendered AvatarSpeech job — no server connection needed.

tsx

const { play } = useAvatar();

play?.({
  audioChunks: ["base64-audio..."],
  alignments: [{
    characters: ["H", "e", "l", "l", "o"],
    character_start_times_seconds: [0, 0.08, 0.16, 0.24, 0.32],
    character_end_times_seconds:   [0.08, 0.16, 0.24, 0.32, 0.4],
  }],
});

addLiveContext(args)

call mode (requires an open WebSocket). Inject real-time context into the AI's conversation — useful for telling the avatar what the user is currently doing in the app.

tsx

const { addLiveContext } = useAvatar();

// Append to existing context
addLiveContext?.({
  context: "User is now viewing the Premium Plan screen.",
});

// Replace the previous context entirely
addLiveContext?.({
  context: "User just opened checkout.",
  update: true,
});

Argument shape:

tsx

type LiveContextArgs = {
  context: string;   // non-empty, max 2000 characters
  update?: boolean;  // when true, replaces the previously submitted context
};

If there is no live session, the context is dropped and a non-fatal NOT_CONNECTED error is emitted. If context is empty or exceeds 2000 characters, a non-fatal INVALID_INPUT error is emitted. Neither ends the session.

Event subscription via on/off

For one-off subscriptions in a useEffect. For most cases, prefer useAvatarEvent — it cleans up automatically.

tsx

const { on, off } = useAvatar();

useEffect(() => {
  const onStart = () => console.log("Session started");
  const onError = (err) => console.error(err.code, err.message);

  on("start", onStart);
  on("error", onError);

  return () => {
    off("start", onStart);
    off("error", onError);
  };
}, [on, off]);

Mode-specific methods (speakText, speakAudio, play, addLiveContext) return undefined when called in the wrong mode. Always use optional chaining: speakText?.("hello").

Avatar

useAvatarEvent