I prefer to self host, and am in the process of setting up my entire system to run off solar (tiny 2x250 watt solar array, 3 deep cycle 100Ah marine batteries and sundry equipment).
End to end, my AI rig - a BC-250 (a cut down PS5, basically) consumes 50w standby / 200w on inference and can run Qwen 3.6-27b - Ternary Bonsai specifically (about par with Sonnet 4.5 on the AAII) at good speeds (~30 tok/s)
With a smart power plug and Raspberry pi (uses about 5w) as the pilot fish, I can run my entire system “off grid”, to say nothing of the excess solar that accumulates (several days battery back up to run home router, pi server, hard drive array and 2-3hrs daily of inference).
For $1500 AUD, “Poor man’s Sonnet 4.5 at home” ain’t nothing to sneeze at.
Outside of folks such as myself, this blog post may be of interest -
Just in defence of your point:
I prefer to self host, and am in the process of setting up my entire system to run off solar (tiny 2x250 watt solar array, 3 deep cycle 100Ah marine batteries and sundry equipment).
End to end, my AI rig - a BC-250 (a cut down PS5, basically) consumes 50w standby / 200w on inference and can run Qwen 3.6-27b - Ternary Bonsai specifically (about par with Sonnet 4.5 on the AAII) at good speeds (~30 tok/s)
With a smart power plug and Raspberry pi (uses about 5w) as the pilot fish, I can run my entire system “off grid”, to say nothing of the excess solar that accumulates (several days battery back up to run home router, pi server, hard drive array and 2-3hrs daily of inference).
For $1500 AUD, “Poor man’s Sonnet 4.5 at home” ain’t nothing to sneeze at.
Outside of folks such as myself, this blog post may be of interest -
https://blog.andymasley.com/p/individual-ai-use-is-not-bad-for