Skip to content

Add handlers for supported synthesis events for Azure TTS - #1486

Merged
theomonnom merged 8 commits into
livekit:mainfrom
anishnag:azure-tts-callbacks
Feb 18, 2025
Merged

theomonnom merged 8 commits into
livekit:mainfrom
anishnag:azure-tts-callbacks

Conversation

@anishnag

@anishnag anishnag commented Feb 13, 2025 •

Copy link
Copy Markdown
Contributor

Adding handlers for each of the SpeechSynthesizer subscribable events according to the docs:

  • BookmarkReached
  • SynthesisCanceled
  • SynthesisCompleted
  • SynthesisStarted
  • Synthesizing
  • VisemeReceived
  • WordBoundary

@CLAassistant

CLAassistant commented Feb 13, 2025 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@anishnag anishnag changed the title Add supported callbacks for Azure TTS Add handlers for supported synthesis events for Azure TTS Feb 13, 2025
@anishnag
anishnag marked this pull request as ready for review February 13, 2025 08:53
@changeset-bot

changeset-bot Bot commented Feb 14, 2025 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 2572a17

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
livekit-plugins-azure Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@theomonnom

theomonnom commented Feb 14, 2025 •

Copy link
Copy Markdown
Member

Hey, can you fix the Ruff CI.

Also curious, why do you need those callbacks? It seems hard to correlate them with the ChunkedStream output (Since the frames will be read asynchronously)

@anishnag

Copy link
Copy Markdown
Contributor Author

Hey, can you fix the Ruff CI.

Also curious, why do you need those callbacks? It seems hard to correlate them with the ChunkedStream output (Since the frames will be read asynchronously)

@theomonnom For now, we do client-side syncing given these events. Eventually we'll want to make it work on the agent side which would probably look like monitoring the audio frames and relaying the events based on running duration.

@theomonnom
theomonnom merged commit fc9e523 into livekit:main Feb 18, 2025
@github-actions github-actions Bot mentioned this pull request Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants