●Stories
●Firehose
●All
●Popular
●Polls
●Software
●Thought Leadership
Submit
●
Login
●or
●
Sign up
●Topics:
●Devices
●Build
●Entertainment
●Technology
●Open Source
●Science
●YRO
●Follow us:
●RSS
●Facebook
●LinkedIn
●Twitter
●
Youtube
●
Mastodon
●Bluesky
Catch up on stories from the past week (and beyond) at the Slashdot story archive
Forgot your password?
Close
Post
Load All Comments
Full
Abbreviated
Hidden
/Sea
Score:
5
4
3
2
1
0
-1
More
| Reply
Login
Forgot your password?
Close
Close
Log In/Create an Account
●
All
●
Insightful
●
Informative
●
Interesting
●
Funny
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
byDan East ( 318230 ) writes:
Just to be clear here, Moltbot does not run AI inference locally. You connect it to your standard AI services (ChatGPT, Gemini, etc), which do the actual AI processing. What Moltbot does is connect those things to other things, like to Whatsapp.
In fact, even if you do have your own local inference engine running, like a llama model, Moltbot can't work with it currently. It ONLY works with the big AI services.
It really is just glue to connects things together, and is so lightweight it even runs on a Raspberry Pi with 2GB of ram. So I'm not sure what all the Mac Mini hubbub is about. The ability to run this on Amazon's Free Tier shows just how lightweight and little processing it does (it's just formatting and moving chat messages from one thing to another basically).
To earlier commenters saying that Peter Steinberger is missing the entire point of running locally when he recommends AWS - you aren't understanding what Moltbot is doing. If you're already committed to using online services for the fundamental AI inference itself, it doesn't matter that Moltbot is running in the cloud too.
Reply to This
twitter
facebook
Flag as Inappropriate
bydrinkypoo ( 153816 ) writes:
Just to be clear here, Moltbot does not run AI inference locally. You connect it to your standard AI services (ChatGPT, Gemini, etc), which do the actual AI processing.
From the project page, the first point of pride is:
Runs on Your Machine
Mac, Windows, or Linux. Anthropic, OpenAI, or local models. Private by default--your data stays yours.
So no, that's only an option.
Reply to This Parent
twitter
facebook
Flag as Inappropriate
byFly Swatter ( 30498 ) writes:
that 'or local models' part should be in fine print it's so small you can't see it. Few people have the hardware capable of running the big models locally; and when they see the price to buy said hardware, for most - that ain't happening.
byMoridineas ( 213502 ) writes:
Mac Minis are solid at running models locally.
bythegarbz ( 1787294 ) writes:
Few people have the hardware capable of running the big models locally
There are many "large" models that run perfectly fine on a modest gaming rig. Not every model needs to be 1T+ parameters. Even some "big" models like Llama 4 Scout or Claude 5 can run on a heafty gaming rig.
The key about this kind of thing is, if you're deploying something locally you're usually doing it for a reason. When you have a reason you don't need a general purpose attempt to do everything including generate pictures of a kitchen sink AI model. Special purpose models are small, run locally, and ofte
bydrinkypoo ( 153816 ) writes:
Any schmoe with a modern processor and a fair amount of RAM can run fairly sizable models on their CPU with decent performance. The RAM is the sticking point today, but if they bought just a few months ago, no problem.
I have a 5900X and models are only a little slower on that than my 4060 Ti 16GB. Plus I have 64GB so I can actually run larger models there than on my GPU.
A used Nvidia compute card with 24GB VRAM is available pretty reasonably on eBay, for less than I paid for this GPU.
bycayenne8 ( 626475 ) writes:
If you wanna lay out a bit of $$ for Mac Studio minis....you can link up to 5 of them high speed and run some BIG ass models.
See HERE on YouTube [youtu.be].
byEvilSS ( 557649 ) writes:
You don't need crazy hardware to run local models. gpt-oss-20b will run on a 24GB Mac Mini. Qwen3 8B 8 bit MLX will run on a base model mini. For the agenic stuff a lot of people are using Moltbot for either of those would be fine.
byDan East ( 318230 ) writes:
I was basing my comment about local model support on this:
https://github.com/moltbot/moltbot/issues/2838 [github.com]
Clawdbot currently supports model providers, but configuring local inference engines such as vLLM and Ollama is not straightforward or fully documented. Users running local LLMs (GPU / on-prem / WSL) face friction when attempting to integrate these providers reliably.
Adding official support for vLLM and Ollama as first-class providers would significantly improve local deployment, performance, and developer experience.
So it sounds like it is in the realm of possibility, but being neither documented nor straightforward sounds beyond the reach of most normal users.
bydrinkypoo ( 153816 ) writes:
It sounds like it's doable using a proxy. Most normal users won't be doing this at all.
byShades72 ( 6355170 ) writes:
I went through the installation and configuration wizard from Moltbot today. The text on the website may say that there is support for local LLMs, but there is no such option during installation or configuration. Only options for 'copy your API key from Claude/OpenAI/Brave/OpenRouter/etc here'.
So it is nice and all that their website makes those claims, but the software sure doesn't. Now I'm quite sure that it is possible to use a local LLM, but I expect it to be much more of a hassle than it is worth.
My ex
bykarmawarrior ( 311177 ) writes:
> you aren't understanding what Moltbot is doing
This is true, but that's because the summary says the exact opposite of what you're saying, and I suspect it's you, not the summary, that's right given that a "useful" spicy-autocomplete system generally requires more set up than "Just install this easy to download package on a discarded Mac mini".
Urgh! Slashdot!
bygweihir ( 88907 ) writes:
What a nice way to leak data and get attacked!
byEvilSS ( 557649 ) writes:
What are you even talking about? They have official documentation on using Ollama, LMStudio, vLLM, and others not to mention OpenRouter. Plus if it talks to OpenAI, then it can talk to most local models since many of the runtimes use the OpenAI API spec. As for why the Mac Mini, a bone-stock mini will give you about 8GB of high speed RAM (leaving 8gb for the system) for around $550. 24GB model gets you at least 16GB to play with. You can run a lot of decent local models on either one.
There may be more comments in this discussion. Without JavaScript enabled, you might want to turn on Classic Discussion System in your preferences instead.
Slashdot
●
●
Submit Story
If A = B and B = C, then A = C, except where void or prohibited by law.
-- Roy Santoro
●FAQ
●Story Archive
●Hall of Fame
●Advertising
●Terms
●Privacy Statement
●About
●Feedback
●Mobile View
●Blog
Do Not Sell or Share My Personal Information
Copyright © 2026 Slashdot Media. All Rights Reserved.
×
Close
Working...
Working...