Skip to content

Text to Speech - add new languages support #712

@IgorSwat

Description

@IgorSwat

Problem Description

As it stands for now, the RNE implementation of the Kokoro model supports 2 languages: 🇺🇸 American English and 🇬🇧 British English. Hovewer, the Kokoro model itself provides a support for many other languages, such as:

  • 🇪🇸 Spanish
  • 🇫🇷 French
  • 🇮🇹 Italian
  • 🇧🇷 Brazilian Portuguese
  • 🇨🇳 Mandarin Chinese
    The only significant requirement to support those languages is to provide an appropriate phonemization, similar to the one for English.

There is also a possibility of adding an external language, not supported by Kokoro originally, by finetuning the model and training a new voice.

What should be done

In case of a language already supported by Kokoro:

  1. Implement phonemization for given language in the phonemization package

In case of an external language:

  1. Implement phonemization for given language in the phonemization package
  2. Select an appropriate dataset containing speech samples for given language
  3. Finetune the model to produce a new voice for given language

Benefits to React Native ExecuTorch

Expands the Text to Speech function accessability for non-english users.

Metadata

Metadata

Assignees

No one assigned

    Labels

    3rd party packageIssue related to 3rd party packages, but not ExecuTorch, e.g. ExpofeaturePRs that implement a new featureideaNew idea to enhance the library, suggestion, etc.modelIssues related to exporting, improving, fixing ML models

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions