[{"data":1,"prerenderedAt":1394},["ShallowReactive",2],{"work-\u002Fwork\u002Fvapi-voice-widget-webfuse":3},{"id":4,"title":5,"body":6,"date":1379,"description":1380,"extension":1381,"externalUrl":1382,"featured":841,"kind":1383,"meta":1384,"navigation":841,"path":1385,"seo":1386,"stem":1387,"tags":1388,"__hash__":1393},"work\u002Fwork\u002Fvapi-voice-widget-webfuse.md","Giving a voice AI hands: a WebRTC voice agent inside a web proxy",{"type":7,"value":8,"toc":1373},"minimark",[9,13,37,44,49,52,100,111,135,138,142,158,169,407,417,688,694,712,1022,1036,1153,1159,1279,1297,1303,1307,1362,1366,1369],[10,11,5],"h1",{"id":12},"giving-a-voice-ai-hands-a-webrtc-voice-agent-inside-a-web-proxy",[14,15,16,17,24,25,30,31,36],"p",{},"Built on ",[18,19,23],"a",{"href":20,"rel":21},"https:\u002F\u002Fwww.webfuse.com",[22],"nofollow","Webfuse",", Surfly's co-browsing platform. It's a floating orb you click and talk to, and it operates the live page for you - clicks buttons, fills forms, presses keys, picks from dropdowns, navigates. The voice and reasoning come from the ",[18,26,29],{"href":27,"rel":28},"https:\u002F\u002Fvapi.ai",[22],"Vapi"," Web SDK; the hands come from the Webfuse Automation API. A public, reusable reference extension - the ",[18,32,35],{"href":33,"rel":34},"https:\u002F\u002Fgithub.com\u002Fwebfuse-com\u002Fextension-vapi-voice-agent",[22],"source is on GitHub",".",[14,38,39],{},[40,41],"img",{"alt":42,"src":43},"The orb, connected and listening, floating over the live page","\u002Fimages\u002Fvapi-voice-widget-webfuse\u002F1.webp",[45,46,48],"h2",{"id":47},"the-problem","The problem",[14,50,51],{},"A browser voice agent needs two capability sets at the same time:",[53,54,55,76],"ol",{},[56,57,58,62,63,67,68,71,72,75],"li",{},[59,60,61],"strong",{},"Network and CSP freedom"," - the Vapi SDK runs WebRTC through Daily.co, which fetches external JS bundles, opens ",[64,65,66],"code",{},"wss:\u002F\u002F"," signaling sockets, and spins up ",[64,69,70],{},"AudioWorklet","s from ",[64,73,74],{},"blob:"," URLs.",[56,77,78,81,82,85,86,89,90,93,94,96,97,36],{},[59,79,80],{},"Browser media and DOM APIs"," - ",[64,83,84],{},"RTCPeerConnection",", ",[64,87,88],{},"getUserMedia"," for the mic, ",[64,91,92],{},"AudioContext","\u002F",[64,95,70],{},", and a real ",[64,98,99],{},"document",[14,101,102,103,106,107,110],{},"The root cause of difficulty is that ",[59,104,105],{},"Webfuse is a web proxy",", and it deliberately does not let an extension's ",[64,108,109],{},"host_permissions"," override the proxied page's Content Security Policy. That's the correct decision - overriding it would break the original document's security model - but it invalidates every obvious place to put the SDK:",[112,113,114,121,124],"ul",{},[56,115,116,117,120],{},"Inject the SDK ",[64,118,119],{},"\u003Cscript>"," into the page -> runs in page context, inherits the page's CSP -> external fetch and blob worklets blocked.",[56,122,123],{},"Import the SDK in the content script -> still tab context, still page CSP -> same blocks.",[56,125,126,127,129,130,129,132,134],{},"Run the SDK in the background service worker -> CSP is fine, but service workers have no ",[64,128,84],{},", no ",[64,131,88],{},[64,133,70],{},", no DOM -> \"WebRTC not supported or suppressed\".",[14,136,137],{},"No single naive context gives you both halves. This doesn't fail on a bug; it fails on the platform's security architecture. I tried four execution contexts before the working one.",[45,139,141],{"id":140},"the-design","The design",[14,143,144,145,148,149,151,152,154,155,157],{},"The resolution: ",[59,146,147],{},"run the SDK in the popup."," A Webfuse extension popup is an extension page with its own document and its own CSP derived from ",[64,150,109],{}," - so external fetch and blob worklets are allowed (unlike content scripts) and ",[64,153,84],{},", the mic, and ",[64,156,70],{}," all exist (unlike service workers). It is the only context that has both halves. Everything else follows from that.",[14,159,160,163,164,168],{},[59,161,162],{},"The popup is detached and styled as the floating orb."," A standard Chrome popup closes the moment it loses focus - useless for a multi-minute voice call. Webfuse popups persist across navigation and can be detached and restyled, so the popup ",[165,166,167],"em",{},"is"," the widget. No separate injected UI to keep in sync.",[170,171,176],"pre",{"className":172,"code":173,"language":174,"meta":175,"style":175},"language-ts shiki shiki-themes material-theme-lighter github-light github-dark","\u002F\u002F Popup is the only context with extension CSP + WebRTC + DOM.\nbrowser.browserAction.detachPopup();\nbrowser.browserAction.setPopupStyles({\n  backgroundColor: \"transparent\", border: \"none\", boxShadow: \"none\",\n  position: \"fixed\", bottom: \"0px\", right: \"0px\",\n  width: \"224px\", height: \"224px\", overflow: \"visible\",\n  \u002F* ... *\u002F\n});\nbrowser.browserAction.openPopup();\n","ts","",[64,177,178,187,212,232,284,328,372,378,389],{"__ignoreMap":175},[179,180,183],"span",{"class":181,"line":182},"line",1,[179,184,186],{"class":185},"sutJx","\u002F\u002F Popup is the only context with extension CSP + WebRTC + DOM.\n",[179,188,190,194,197,200,202,206,209],{"class":181,"line":189},2,[179,191,193],{"class":192},"su5hD","browser",[179,195,36],{"class":196},"sP7_E",[179,198,199],{"class":192},"browserAction",[179,201,36],{"class":196},[179,203,205],{"class":204},"sGLFI","detachPopup",[179,207,208],{"class":192},"()",[179,210,211],{"class":196},";\n",[179,213,215,217,219,221,223,226,229],{"class":181,"line":214},3,[179,216,193],{"class":192},[179,218,36],{"class":196},[179,220,199],{"class":192},[179,222,36],{"class":196},[179,224,225],{"class":204},"setPopupStyles",[179,227,228],{"class":192},"(",[179,230,231],{"class":196},"{\n",[179,233,235,239,242,246,250,253,256,259,261,263,266,268,270,273,275,277,279,281],{"class":181,"line":234},4,[179,236,238],{"class":237},"skxfh","  backgroundColor",[179,240,241],{"class":196},":",[179,243,245],{"class":244},"sjJ54"," \"",[179,247,249],{"class":248},"s_sjI","transparent",[179,251,252],{"class":244},"\"",[179,254,255],{"class":196},",",[179,257,258],{"class":237}," border",[179,260,241],{"class":196},[179,262,245],{"class":244},[179,264,265],{"class":248},"none",[179,267,252],{"class":244},[179,269,255],{"class":196},[179,271,272],{"class":237}," boxShadow",[179,274,241],{"class":196},[179,276,245],{"class":244},[179,278,265],{"class":248},[179,280,252],{"class":244},[179,282,283],{"class":196},",\n",[179,285,287,290,292,294,297,299,301,304,306,308,311,313,315,318,320,322,324,326],{"class":181,"line":286},5,[179,288,289],{"class":237},"  position",[179,291,241],{"class":196},[179,293,245],{"class":244},[179,295,296],{"class":248},"fixed",[179,298,252],{"class":244},[179,300,255],{"class":196},[179,302,303],{"class":237}," bottom",[179,305,241],{"class":196},[179,307,245],{"class":244},[179,309,310],{"class":248},"0px",[179,312,252],{"class":244},[179,314,255],{"class":196},[179,316,317],{"class":237}," right",[179,319,241],{"class":196},[179,321,245],{"class":244},[179,323,310],{"class":248},[179,325,252],{"class":244},[179,327,283],{"class":196},[179,329,331,334,336,338,341,343,345,348,350,352,354,356,358,361,363,365,368,370],{"class":181,"line":330},6,[179,332,333],{"class":237},"  width",[179,335,241],{"class":196},[179,337,245],{"class":244},[179,339,340],{"class":248},"224px",[179,342,252],{"class":244},[179,344,255],{"class":196},[179,346,347],{"class":237}," height",[179,349,241],{"class":196},[179,351,245],{"class":244},[179,353,340],{"class":248},[179,355,252],{"class":244},[179,357,255],{"class":196},[179,359,360],{"class":237}," overflow",[179,362,241],{"class":196},[179,364,245],{"class":244},[179,366,367],{"class":248},"visible",[179,369,252],{"class":244},[179,371,283],{"class":196},[179,373,375],{"class":181,"line":374},7,[179,376,377],{"class":185},"  \u002F* ... *\u002F\n",[179,379,381,384,387],{"class":181,"line":380},8,[179,382,383],{"class":196},"}",[179,385,386],{"class":192},")",[179,388,211],{"class":196},[179,390,392,394,396,398,400,403,405],{"class":181,"line":391},9,[179,393,193],{"class":192},[179,395,36],{"class":196},[179,397,199],{"class":192},[179,399,36],{"class":196},[179,401,402],{"class":204},"openPopup",[179,404,208],{"class":192},[179,406,211],{"class":196},[14,408,409,412,413,416],{},[59,410,411],{},"The assistant is configured fully inline - no dashboard."," Model, transcriber, voice, system prompt, and the entire tool schema are defined in code and passed to ",[64,414,415],{},"vapi.start()",". The tradeoff: I give up reusing dashboard-managed assistants, and in return the build is self-contained, version-controlled, and portable across accounts (it only needs a public key).",[170,418,420],{"className":172,"code":419,"language":174,"meta":175,"style":175},"vapi.start({\n  model: {\n    provider: \"openai\", model: \"gpt-4o\",\n    messages: [{ role: \"system\", content: systemPrompt }],\n    tools: vapiTools as any[],\n  },\n  transcriber: { provider: \"deepgram\", model: \"nova-2\", language: \"en\" },\n  voice: { provider: \"vapi\", voiceId: \"Elliot\" },\n  firstMessage: \"Hey! I can help you interact with this page. What would you like to do?\",\n  clientMessages: [\"tool-calls\", \"transcript\"],\n});\n",[64,421,422,436,446,476,518,541,546,598,633,649,679],{"__ignoreMap":175},[179,423,424,427,429,432,434],{"class":181,"line":182},[179,425,426],{"class":192},"vapi",[179,428,36],{"class":196},[179,430,431],{"class":204},"start",[179,433,228],{"class":192},[179,435,231],{"class":196},[179,437,438,441,443],{"class":181,"line":189},[179,439,440],{"class":237},"  model",[179,442,241],{"class":196},[179,444,445],{"class":196}," {\n",[179,447,448,451,453,455,458,460,462,465,467,469,472,474],{"class":181,"line":214},[179,449,450],{"class":237},"    provider",[179,452,241],{"class":196},[179,454,245],{"class":244},[179,456,457],{"class":248},"openai",[179,459,252],{"class":244},[179,461,255],{"class":196},[179,463,464],{"class":237}," model",[179,466,241],{"class":196},[179,468,245],{"class":244},[179,470,471],{"class":248},"gpt-4o",[179,473,252],{"class":244},[179,475,283],{"class":196},[179,477,478,481,483,486,489,492,494,496,499,501,503,506,508,511,513,516],{"class":181,"line":234},[179,479,480],{"class":237},"    messages",[179,482,241],{"class":196},[179,484,485],{"class":192}," [",[179,487,488],{"class":196},"{",[179,490,491],{"class":237}," role",[179,493,241],{"class":196},[179,495,245],{"class":244},[179,497,498],{"class":248},"system",[179,500,252],{"class":244},[179,502,255],{"class":196},[179,504,505],{"class":237}," content",[179,507,241],{"class":196},[179,509,510],{"class":192}," systemPrompt ",[179,512,383],{"class":196},[179,514,515],{"class":192},"]",[179,517,283],{"class":196},[179,519,520,523,525,528,532,536,539],{"class":181,"line":286},[179,521,522],{"class":237},"    tools",[179,524,241],{"class":196},[179,526,527],{"class":192}," vapiTools ",[179,529,531],{"class":530},"sVHd0","as",[179,533,535],{"class":534},"sZMiF"," any",[179,537,538],{"class":192},"[]",[179,540,283],{"class":196},[179,542,543],{"class":181,"line":330},[179,544,545],{"class":196},"  },\n",[179,547,548,551,553,556,559,561,563,566,568,570,572,574,576,579,581,583,586,588,590,593,595],{"class":181,"line":374},[179,549,550],{"class":237},"  transcriber",[179,552,241],{"class":196},[179,554,555],{"class":196}," {",[179,557,558],{"class":237}," provider",[179,560,241],{"class":196},[179,562,245],{"class":244},[179,564,565],{"class":248},"deepgram",[179,567,252],{"class":244},[179,569,255],{"class":196},[179,571,464],{"class":237},[179,573,241],{"class":196},[179,575,245],{"class":244},[179,577,578],{"class":248},"nova-2",[179,580,252],{"class":244},[179,582,255],{"class":196},[179,584,585],{"class":237}," language",[179,587,241],{"class":196},[179,589,245],{"class":244},[179,591,592],{"class":248},"en",[179,594,252],{"class":244},[179,596,597],{"class":196}," },\n",[179,599,600,603,605,607,609,611,613,615,617,619,622,624,626,629,631],{"class":181,"line":380},[179,601,602],{"class":237},"  voice",[179,604,241],{"class":196},[179,606,555],{"class":196},[179,608,558],{"class":237},[179,610,241],{"class":196},[179,612,245],{"class":244},[179,614,426],{"class":248},[179,616,252],{"class":244},[179,618,255],{"class":196},[179,620,621],{"class":237}," voiceId",[179,623,241],{"class":196},[179,625,245],{"class":244},[179,627,628],{"class":248},"Elliot",[179,630,252],{"class":244},[179,632,597],{"class":196},[179,634,635,638,640,642,645,647],{"class":181,"line":391},[179,636,637],{"class":237},"  firstMessage",[179,639,241],{"class":196},[179,641,245],{"class":244},[179,643,644],{"class":248},"Hey! I can help you interact with this page. What would you like to do?",[179,646,252],{"class":244},[179,648,283],{"class":196},[179,650,652,655,657,659,661,664,666,668,670,673,675,677],{"class":181,"line":651},10,[179,653,654],{"class":237},"  clientMessages",[179,656,241],{"class":196},[179,658,485],{"class":192},[179,660,252],{"class":244},[179,662,663],{"class":248},"tool-calls",[179,665,252],{"class":244},[179,667,255],{"class":196},[179,669,245],{"class":244},[179,671,672],{"class":248},"transcript",[179,674,252],{"class":244},[179,676,515],{"class":192},[179,678,283],{"class":196},[179,680,682,684,686],{"class":181,"line":681},11,[179,683,383],{"class":196},[179,685,386],{"class":192},[179,687,211],{"class":196},[14,689,690],{},[40,691],{"alt":692,"src":693},"The orb while the assistant is speaking","\u002Fimages\u002Fvapi-voice-widget-webfuse\u002F2.webp",[14,695,696,699,700,703,704,707,708,711],{},[59,697,698],{},"The content script is a thin automation relay."," Here's the catch that makes the split non-negotiable: the Webfuse Automation API is only exposed in ",[165,701,702],{},"tab"," context (the content script), but the SDK can't run there. So the two contexts each do only what they uniquely can - the popup decides ",[165,705,706],{},"what"," to do and the content script ",[165,709,710],{},"does"," it on the page.",[170,713,715],{"className":172,"code":714,"language":174,"meta":175,"style":175},"\u002F\u002F tools.ts (popup): send the call across to the page context\nfunction delegateAutomation(scope: string, method: string, ...args: any[]) {\n  return browser.tabs.sendMessage(0, { automationScope: scope, automationMethod: method, automationArgs: args });\n}\n\n\u002F\u002F content.ts (tab): the only place webfuseSession.automation is available\nbrowser.runtime.onMessage.addListener((message: any) => {\n  if (message.automationScope && message.automationMethod) {\n    const { automationScope: scope, automationMethod: method, automationArgs: args = [] } = message;\n    return (browser.webfuseSession.automation as any)[scope][method](...args);\n  }\n});\n",[64,716,717,722,770,832,837,843,848,885,916,961,1008,1013],{"__ignoreMap":175},[179,718,719],{"class":181,"line":182},[179,720,721],{"class":185},"\u002F\u002F tools.ts (popup): send the call across to the page context\n",[179,723,724,728,731,733,737,740,743,745,748,750,752,754,757,760,762,764,766,768],{"class":181,"line":189},[179,725,727],{"class":726},"sbsja","function",[179,729,730],{"class":204}," delegateAutomation",[179,732,228],{"class":196},[179,734,736],{"class":735},"s99_P","scope",[179,738,241],{"class":739},"smGrS",[179,741,742],{"class":534}," string",[179,744,255],{"class":196},[179,746,747],{"class":735}," method",[179,749,241],{"class":739},[179,751,742],{"class":534},[179,753,255],{"class":196},[179,755,756],{"class":739}," ...",[179,758,759],{"class":735},"args",[179,761,241],{"class":739},[179,763,535],{"class":534},[179,765,538],{"class":192},[179,767,386],{"class":196},[179,769,445],{"class":196},[179,771,772,775,778,780,783,785,788,790,794,796,798,801,803,806,808,811,813,815,817,820,822,825,828,830],{"class":181,"line":214},[179,773,774],{"class":530},"  return",[179,776,777],{"class":192}," browser",[179,779,36],{"class":196},[179,781,782],{"class":192},"tabs",[179,784,36],{"class":196},[179,786,787],{"class":204},"sendMessage",[179,789,228],{"class":237},[179,791,793],{"class":792},"srdBf","0",[179,795,255],{"class":196},[179,797,555],{"class":196},[179,799,800],{"class":237}," automationScope",[179,802,241],{"class":196},[179,804,805],{"class":192}," scope",[179,807,255],{"class":196},[179,809,810],{"class":237}," automationMethod",[179,812,241],{"class":196},[179,814,747],{"class":192},[179,816,255],{"class":196},[179,818,819],{"class":237}," automationArgs",[179,821,241],{"class":196},[179,823,824],{"class":192}," args",[179,826,827],{"class":196}," }",[179,829,386],{"class":237},[179,831,211],{"class":196},[179,833,834],{"class":181,"line":234},[179,835,836],{"class":196},"}\n",[179,838,839],{"class":181,"line":286},[179,840,842],{"emptyLinePlaceholder":841},true,"\n",[179,844,845],{"class":181,"line":330},[179,846,847],{"class":185},"\u002F\u002F content.ts (tab): the only place webfuseSession.automation is available\n",[179,849,850,852,854,857,859,862,864,867,869,871,874,876,878,880,883],{"class":181,"line":374},[179,851,193],{"class":192},[179,853,36],{"class":196},[179,855,856],{"class":192},"runtime",[179,858,36],{"class":196},[179,860,861],{"class":192},"onMessage",[179,863,36],{"class":196},[179,865,866],{"class":204},"addListener",[179,868,228],{"class":192},[179,870,228],{"class":196},[179,872,873],{"class":735},"message",[179,875,241],{"class":739},[179,877,535],{"class":534},[179,879,386],{"class":196},[179,881,882],{"class":726}," =>",[179,884,445],{"class":196},[179,886,887,890,893,895,897,900,903,906,908,911,914],{"class":181,"line":380},[179,888,889],{"class":530},"  if",[179,891,892],{"class":237}," (",[179,894,873],{"class":192},[179,896,36],{"class":196},[179,898,899],{"class":192},"automationScope",[179,901,902],{"class":739}," &&",[179,904,905],{"class":192}," message",[179,907,36],{"class":196},[179,909,910],{"class":192},"automationMethod",[179,912,913],{"class":237},") ",[179,915,231],{"class":196},[179,917,918,921,923,926,928,931,933,935,937,939,941,943,945,947,950,953,955,957,959],{"class":181,"line":391},[179,919,920],{"class":726},"    const",[179,922,555],{"class":196},[179,924,800],{"class":925},"sucvu",[179,927,241],{"class":196},[179,929,805],{"class":930},"s_hVV",[179,932,255],{"class":196},[179,934,810],{"class":925},[179,936,241],{"class":196},[179,938,747],{"class":930},[179,940,255],{"class":196},[179,942,819],{"class":925},[179,944,241],{"class":196},[179,946,824],{"class":930},[179,948,949],{"class":739}," =",[179,951,952],{"class":237}," [] ",[179,954,383],{"class":196},[179,956,949],{"class":739},[179,958,905],{"class":192},[179,960,211],{"class":196},[179,962,963,966,968,970,972,975,977,980,983,985,988,990,993,996,999,1002,1004,1006],{"class":181,"line":651},[179,964,965],{"class":530},"    return",[179,967,892],{"class":237},[179,969,193],{"class":192},[179,971,36],{"class":196},[179,973,974],{"class":192},"webfuseSession",[179,976,36],{"class":196},[179,978,979],{"class":192},"automation",[179,981,982],{"class":530}," as",[179,984,535],{"class":534},[179,986,987],{"class":237},")[",[179,989,736],{"class":192},[179,991,992],{"class":237},"][",[179,994,995],{"class":192},"method",[179,997,998],{"class":237},"](",[179,1000,1001],{"class":739},"...",[179,1003,759],{"class":192},[179,1005,386],{"class":237},[179,1007,211],{"class":196},[179,1009,1010],{"class":181,"line":681},[179,1011,1012],{"class":196},"  }\n",[179,1014,1016,1018,1020],{"class":181,"line":1015},12,[179,1017,383],{"class":196},[179,1019,386],{"class":192},[179,1021,211],{"class":196},[14,1023,1024,1027,1028,1031,1032,1035],{},[59,1025,1026],{},"The model sees stable targets, not raw HTML."," The ",[64,1029,1030],{},"take_dom_snapshot"," tool returns interactive elements only, tagged with ",[64,1033,1034],{},"wf-id"," attributes that survive re-renders and cross shadow boundaries. That's a smaller payload to the model and a targeting scheme that doesn't break the moment the page re-renders - instead of feeding full-page HTML and brittle CSS selectors.",[170,1037,1039],{"className":172,"code":1038,"language":174,"meta":175,"style":175},"case \"take_dom_snapshot\": {\n  const snapshot = await delegateAutomation(\"see\", \"domSnapshot\", {\n    webfuseIDs: true,      \u002F\u002F stable wf-id targets, e.g. \"2-1-54\"\n    crossShadow: true,     \u002F\u002F pierce shadow DOM\n    interactiveOnly: true, \u002F\u002F only clickable\u002Ftypeable elements -> smaller payload\n  });\n  return snapshot;\n}\n",[64,1040,1041,1057,1092,1108,1122,1136,1144,1149],{"__ignoreMap":175},[179,1042,1043,1046,1048,1050,1052,1055],{"class":181,"line":182},[179,1044,1045],{"class":530},"case",[179,1047,245],{"class":244},[179,1049,1030],{"class":248},[179,1051,252],{"class":244},[179,1053,1054],{"class":192},": ",[179,1056,231],{"class":196},[179,1058,1059,1062,1065,1068,1070,1072,1074,1077,1079,1081,1083,1086,1088,1090],{"class":181,"line":189},[179,1060,1061],{"class":192},"  const snapshot ",[179,1063,1064],{"class":739},"=",[179,1066,1067],{"class":530}," await",[179,1069,730],{"class":204},[179,1071,228],{"class":192},[179,1073,252],{"class":244},[179,1075,1076],{"class":248},"see",[179,1078,252],{"class":244},[179,1080,255],{"class":196},[179,1082,245],{"class":244},[179,1084,1085],{"class":248},"domSnapshot",[179,1087,252],{"class":244},[179,1089,255],{"class":196},[179,1091,445],{"class":196},[179,1093,1094,1097,1099,1103,1105],{"class":181,"line":214},[179,1095,1096],{"class":237},"    webfuseIDs",[179,1098,241],{"class":196},[179,1100,1102],{"class":1101},"syTEX"," true",[179,1104,255],{"class":196},[179,1106,1107],{"class":185},"      \u002F\u002F stable wf-id targets, e.g. \"2-1-54\"\n",[179,1109,1110,1113,1115,1117,1119],{"class":181,"line":234},[179,1111,1112],{"class":237},"    crossShadow",[179,1114,241],{"class":196},[179,1116,1102],{"class":1101},[179,1118,255],{"class":196},[179,1120,1121],{"class":185},"     \u002F\u002F pierce shadow DOM\n",[179,1123,1124,1127,1129,1131,1133],{"class":181,"line":286},[179,1125,1126],{"class":237},"    interactiveOnly",[179,1128,241],{"class":196},[179,1130,1102],{"class":1101},[179,1132,255],{"class":196},[179,1134,1135],{"class":185}," \u002F\u002F only clickable\u002Ftypeable elements -> smaller payload\n",[179,1137,1138,1141],{"class":181,"line":330},[179,1139,1140],{"class":196},"  }",[179,1142,1143],{"class":192},");\n",[179,1145,1146],{"class":181,"line":374},[179,1147,1148],{"class":192},"  return snapshot;\n",[179,1150,1151],{"class":181,"line":380},[179,1152,836],{"class":196},[14,1154,1155,1158],{},[59,1156,1157],{},"Tool results loop back into the live call."," Tools run client-side and async; each result (or error) is sent back to the model as a system message, so the conversation keeps going and the assistant can recover from a failed action mid-call.",[170,1160,1162],{"className":172,"code":1161,"language":174,"meta":175,"style":175},"const result = await handleToolCall(name, params);\nvapi?.send({\n  type: \"add-message\",\n  message: { role: \"system\", content: `[Tool \"${name}\" result]: ${result}` },\n});\n",[64,1163,1164,1189,1203,1219,1271],{"__ignoreMap":175},[179,1165,1166,1169,1172,1174,1176,1179,1182,1184,1187],{"class":181,"line":182},[179,1167,1168],{"class":726},"const",[179,1170,1171],{"class":930}," result",[179,1173,949],{"class":739},[179,1175,1067],{"class":530},[179,1177,1178],{"class":204}," handleToolCall",[179,1180,1181],{"class":192},"(name",[179,1183,255],{"class":196},[179,1185,1186],{"class":192}," params)",[179,1188,211],{"class":196},[179,1190,1191,1193,1196,1199,1201],{"class":181,"line":189},[179,1192,426],{"class":192},[179,1194,1195],{"class":196},"?.",[179,1197,1198],{"class":204},"send",[179,1200,228],{"class":192},[179,1202,231],{"class":196},[179,1204,1205,1208,1210,1212,1215,1217],{"class":181,"line":214},[179,1206,1207],{"class":237},"  type",[179,1209,241],{"class":196},[179,1211,245],{"class":244},[179,1213,1214],{"class":248},"add-message",[179,1216,252],{"class":244},[179,1218,283],{"class":196},[179,1220,1221,1224,1226,1228,1230,1232,1234,1236,1238,1240,1242,1244,1247,1250,1253,1256,1258,1261,1263,1266,1269],{"class":181,"line":234},[179,1222,1223],{"class":237},"  message",[179,1225,241],{"class":196},[179,1227,555],{"class":196},[179,1229,491],{"class":237},[179,1231,241],{"class":196},[179,1233,245],{"class":244},[179,1235,498],{"class":248},[179,1237,252],{"class":244},[179,1239,255],{"class":196},[179,1241,505],{"class":237},[179,1243,241],{"class":196},[179,1245,1246],{"class":244}," `",[179,1248,1249],{"class":248},"[Tool \"",[179,1251,1252],{"class":244},"${",[179,1254,1255],{"class":192},"name",[179,1257,383],{"class":244},[179,1259,1260],{"class":248},"\" result]: ",[179,1262,1252],{"class":244},[179,1264,1265],{"class":192},"result",[179,1267,1268],{"class":244},"}`",[179,1270,597],{"class":196},[179,1272,1273,1275,1277],{"class":181,"line":286},[179,1274,383],{"class":196},[179,1276,386],{"class":192},[179,1278,211],{"class":196},[14,1280,1281,1282,1285,1286,1288,1289,1292,1293,1296],{},"One smaller gotcha worth recording: Daily.co's signaling is a secure WebSocket, so ",[64,1283,1284],{},"wss:\u002F\u002F*.daily.co\u002F*"," has to be whitelisted in ",[64,1287,109],{}," ",[165,1290,1291],{},"separately"," from the ",[64,1294,1295],{},"https:\u002F\u002F"," entry, or the connection is silently blocked.",[14,1298,1299],{},[40,1300],{"alt":1301,"src":1302},"A missing-key error surfaced in the page, since config lives in extension settings","\u002Fimages\u002Fvapi-voice-widget-webfuse\u002F3.webp",[45,1304,1306],{"id":1305},"results","Results",[112,1308,1309,1315,1321,1344,1355],{},[56,1310,1311,1314],{},[59,1312,1313],{},"Four execution-context attempts"," before the working architecture: content-as-script-tag, content-as-import, background service worker, and finally the popup.",[56,1316,1317,1320],{},[59,1318,1319],{},"Three contexts, one SDK location"," - the popup carries all the WebRTC, audio, and network; the content script is a thin relay; the background is a two-line opener.",[56,1322,1323,1326,1327,85,1329,85,1332,85,1335,85,1338,85,1341,36],{},[59,1324,1325],{},"Six voice tools"," the model can call: ",[64,1328,1030],{},[64,1330,1331],{},"click_element",[64,1333,1334],{},"type_text",[64,1336,1337],{},"press_key",[64,1339,1340],{},"select_option",[64,1342,1343],{},"navigate_to",[56,1345,1346,1352,1353,36],{},[59,1347,1348,1349,1351],{},"Six ",[64,1350,109],{}," entries",", including the non-obvious standalone ",[64,1354,1284],{},[56,1356,1357,1358,1361],{},"Ships as three IIFE bundles from a single dependency (",[64,1359,1360],{},"@vapi-ai\u002Fweb","), configured entirely in code.",[45,1363,1365],{"id":1364},"the-principle","The principle",[14,1367,1368],{},"When a platform sandboxes capabilities across execution contexts, stop hunting for the one context that does everything. Place each piece of work in the only context that can do it, and relay between them. The architecture isn't a compromise around the sandbox - it's the shape the sandbox was telling you to build all along.",[1370,1371,1372],"style",{},"html pre.shiki code .sutJx, html code.shiki .sutJx{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#6A737D;--shiki-default-font-style:inherit;--shiki-dark:#6A737D;--shiki-dark-font-style:inherit}html pre.shiki code .su5hD, html code.shiki .su5hD{--shiki-light:#90A4AE;--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sP7_E, html code.shiki .sP7_E{--shiki-light:#39ADB5;--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sGLFI, html code.shiki .sGLFI{--shiki-light:#6182B8;--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .skxfh, html code.shiki .skxfh{--shiki-light:#E53935;--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sjJ54, html code.shiki .sjJ54{--shiki-light:#39ADB5;--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .s_sjI, html code.shiki .s_sjI{--shiki-light:#91B859;--shiki-default:#032F62;--shiki-dark:#9ECBFF}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sVHd0, html code.shiki .sVHd0{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#D73A49;--shiki-default-font-style:inherit;--shiki-dark:#F97583;--shiki-dark-font-style:inherit}html pre.shiki code .sZMiF, html code.shiki .sZMiF{--shiki-light:#E2931D;--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sbsja, html code.shiki .sbsja{--shiki-light:#9C3EDA;--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .s99_P, html code.shiki .s99_P{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#E36209;--shiki-default-font-style:inherit;--shiki-dark:#FFAB70;--shiki-dark-font-style:inherit}html pre.shiki code .smGrS, html code.shiki .smGrS{--shiki-light:#39ADB5;--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .srdBf, html code.shiki .srdBf{--shiki-light:#F76D47;--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sucvu, html code.shiki .sucvu{--shiki-light:#E53935;--shiki-default:#E36209;--shiki-dark:#FFAB70}html pre.shiki code .s_hVV, html code.shiki .s_hVV{--shiki-light:#90A4AE;--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .syTEX, html code.shiki .syTEX{--shiki-light:#FF5370;--shiki-default:#005CC5;--shiki-dark:#79B8FF}",{"title":175,"searchDepth":189,"depth":189,"links":1374},[1375,1376,1377,1378],{"id":47,"depth":189,"text":48},{"id":140,"depth":189,"text":141},{"id":1305,"depth":189,"text":1306},{"id":1364,"depth":189,"text":1365},"2026-06-16","A voice assistant that sees and operates any web page - solved by placing the WebRTC SDK in the one extension context that has both an unblocked CSP and full media APIs, after four failed attempts.","md",null,"case-study",{},"\u002Fwork\u002Fvapi-voice-widget-webfuse",{"title":5,"description":1380},"work\u002Fvapi-voice-widget-webfuse",[23,29,1389,1390,1391,1392],"Voice AI","WebRTC","Browser Automation","TypeScript","Z2hRGsJn4xwaz2TCY5Wkyo7O8dz4W0vp77V1_h6Ll9Y",1781635014771]