New top story on Hacker News: Show HN: Blast – Fast, multi-threaded serving engine for web browsing AI agents
Show HN: Blast – Fast, multi-threaded serving engine for web browsing AI agents
20 by calebhwin | 9 comments on Hacker News.
Hi HN! BLAST is a high-performance serving engine for browser-augmented LLMs, designed to make deploying web-browsing AI easy, fast, and cost-manageable. The goal with BLAST is to ultimately achieve google search level latencies for tasks that currently require a lot of typing and clicking around inside a browser. We're starting off with automatic parallelism, prefix caching, budgeting (memory and LLM cost), and an OpenAI-Compatible API but have a ton of ideas in the pipe! Website & Docs: https://ift.tt/YgSsNIB https://ift.tt/Gg4rpOX MIT-Licensed Open-Source: https://ift.tt/lSckMgD Hope some folks here find this useful! Please let me know what you think in the comments or ping me on Discord. — Caleb (PhD student @ Stanford CS)
20 by calebhwin | 9 comments on Hacker News.
Hi HN! BLAST is a high-performance serving engine for browser-augmented LLMs, designed to make deploying web-browsing AI easy, fast, and cost-manageable. The goal with BLAST is to ultimately achieve google search level latencies for tasks that currently require a lot of typing and clicking around inside a browser. We're starting off with automatic parallelism, prefix caching, budgeting (memory and LLM cost), and an OpenAI-Compatible API but have a ton of ideas in the pipe! Website & Docs: https://ift.tt/YgSsNIB https://ift.tt/Gg4rpOX MIT-Licensed Open-Source: https://ift.tt/lSckMgD Hope some folks here find this useful! Please let me know what you think in the comments or ping me on Discord. — Caleb (PhD student @ Stanford CS)
Comments
Post a Comment