User Tools

Site Tools


thesis:work-journal

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Last revision Both sides next revision
thesis:work-journal [2019/03/24 10:33]
avnerus
thesis:work-journal [2019/03/24 11:11]
avnerus
Line 322: Line 322:
 ==== Toolkits ==== ==== Toolkits ====
 In the last update regarding the gstreamer saga I hinted at the possibility of using a different toolkit than uclibc, namely musl, to solve the dynamic linking issues I had with gstrpicamsrc. Since then the following developments unfolded: In the last update regarding the gstreamer saga I hinted at the possibility of using a different toolkit than uclibc, namely musl, to solve the dynamic linking issues I had with gstrpicamsrc. Since then the following developments unfolded:
 +
 +**TLDR** : I am now more involved in WPE development,​ know a lot more about toolkit possibilities,​ running the latest WPE and looking toward 64bit support.
 +
   * I switched from uclibc to musl, and indeed it solved the dynamic linking problem.   * I switched from uclibc to musl, and indeed it solved the dynamic linking problem.
   * In order to compile wpewbkit with musl, I had to migrate some patches that I found are being used in [[https://​github.com/​WebPlatformForEmbedded/​meta-wpe|here]]. My patches are now located [[https://​github.com/​Avnerus/​softbot-buildroot/​tree/​softbot/​package/​wpewebkit/​musl-patch-backup|here]].   * In order to compile wpewbkit with musl, I had to migrate some patches that I found are being used in [[https://​github.com/​WebPlatformForEmbedded/​meta-wpe|here]]. My patches are now located [[https://​github.com/​Avnerus/​softbot-buildroot/​tree/​softbot/​package/​wpewebkit/​musl-patch-backup|here]].
Line 336: Line 339:
 BR2_TARGET_OPTIMIZATION="​-mthumb -mfpu=neon-fp-armv8 -mtune=cortex-a53"​ BR2_TARGET_OPTIMIZATION="​-mthumb -mfpu=neon-fp-armv8 -mtune=cortex-a53"​
 </​code>​ </​code>​
 +
 +==== Audio / Voice ====
 +
 +I have been working on Hitodama'​s web based voice. The current strategy for implementation is as follows:
 +
 +  - Translate the controller'​s text if requested - Using Google Cloud Translate (Which I belive is the best translate API).
 +  - Generate the voice using Microsoft'​s Cloud Speech (Which supports much more langauges than Google, including Hebrew, Arabic and Finnish ).
 +  - Apply Web Audio filters on the voice to make it gender neutral and nonhuman.
 +
 +This was fairly easy to make for the desktop, currently using Granular pitch shifting for Web Audio from [[https://​github.com/​urtzurd/​html-audio|here]]. However, as what is becoming a pattern, this became much more complicated when trying to run on WPE:
 +
 +  * The first problem was that there is a bug with streaming //audio src>// elements, having the audio loop indefinitely. After discussing on #webkit I submitted a report [[https://​bugs.webkit.org/​show_bug.cgi?​id=196000|here]], ​ which was then marked as a duplicate of [[https://​bugs.webkit.org/​show_bug.cgi?​id=189471|this bug]]. Now Phillipe made a patch and I was able to apply it, and waiting to test it.
 +  * However I did realized that I probably don't need streaming anyway because it's good enough to buffer the audio first and then pass it through Web Audio. ​
 +  * Web-audio support in cog / WPE is apparently not enabled by default, even if you compile WPE with support. It is enabled using command line arguments when running Cog, and you can see my current command line arguments in [[https://​github.com/​Avnerus/​softbot-buildroot/​blob/​softbot/​softbot-scripts/​softbot-browser.sh#​L4|here]].
 +  * I have Web Audio working now in WPE, but for some reason in my latest attempt, I couldn'​t play a blob that was converted to an arraybuffer node. It was just silent. This requires further investigation.
 +  * NOTE: In order to remotely inspect the latest WPE, one needs the latest WebKitGTK based browser. Easiest way to install it in a sandboxed ([[http://​flatkill.org/​|maybe]]) environment would be using Flatpak. You can install the latest Epiphany from [[https://​webkitgtk.org/​epiphany-tech-preview|here]].
  
  
  
thesis/work-journal.txt ยท Last modified: 2019/04/13 15:25 by avnerus