Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- WS Success! TTFT dropped to 4.5s! Now moving to the final Glue Layer
- Hey again! The 2.0 motor with the threading patch and the rearm_idle logic is running much better on the ODROID-H4. Our TTFT dropped from about 10 seconds to a stable 4.5 seconds using the WebSocket client.
- We noticed a small issue during the state transitions: when moving from THINKING back to IDLE, the system often triggers a phantom wake-word immediately. It seems like the audio queue accumulates frames while the system is processing, and when it returns to IDLE, it processes that backlog.
- Here is the current chassis of our loop, including the threading fix:
- Python
- import collections, enum, queue, threading, time, os, sys
- import numpy as np
- import sounddevice as sd
- import webrtcvad
- import openwakeword
- from openwakeword.model import Model
- class State(enum.Enum):
- IDLE = "idle"
- RECORDING = "recording"
- THINKING = "thinking"
- SPEAKING = "speaking"
- class VoiceLoop:
- def __init__(self):
- self.sample_rate = 16000
- self.frame_ms = 20
- self.audio_q = queue.Queue(maxsize=256)
- pkg_path = os.path.dirname(openwakeword.__file__)
- model_dir = os.path.join(pkg_path, "resources", "models")
- alexa_file = next((os.path.join(root, f) for root, _, files in os.walk(model_dir)
- for f in files if "alexa" in f.lower() and f.endswith(".onnx")), None)
- self.oww = Model(wakeword_model_paths=[alexa_file])
- self.wake_word_key = list(self.oww.models.keys())[0]
- self.vad = webrtcvad.Vad(2)
- self.state = State.IDLE
- self.oww_batch = collections.deque(maxlen=4)
- self.preroll = collections.deque(maxlen=40)
- self.rearm_until = 0.0
- def drain_audio_q(self):
- try:
- while True: self.audio_q.get_nowait()
- except queue.Empty: pass
- def rearm_idle(self, cooldown_s=1.0):
- self.drain_audio_q()
- self.oww.reset()
- self.oww_batch.clear()
- self.preroll.clear()
- self.rearm_until = time.monotonic() + cooldown_s
- self.state = State.IDLE
- def on_utterance_ready(self, audio_bytes):
- print("Utterance ready. This is where we need the glue!")
- time.sleep(2)
- self.rearm_idle()
- def process_frame(self, frame_bytes):
- now = time.monotonic()
- if self.state in (State.THINKING, State.SPEAKING): return
- if self.state == State.IDLE and now < self.rearm_until: return
- self.preroll.append(frame_bytes)
- is_speech = self.vad.is_speech(frame_bytes, 16000)
- if self.state == State.RECORDING:
- # Silence detection and finish_recording()
- pass
- elif self.state == State.IDLE:
- # OWW batching and prediction
- pass
- We are ready for the final piece of the puzzle! Could you share the glue layer to connect this loop with faster_whisper transcription, the persistent OpenClaw WS client (using the auth key), and the Piper TTS streaming? We are especially interested in the barge-in logic to stop Piper if speech is detected during output.
Advertisement
Add Comment
Please, Sign In to add comment