EGLA AI has developed an affordable software framework to load multi-TB data sources and your open source LLMs, or if you prefer, paid systems with the following solution, that you can locally run it or, be executed in our cloud platform.
The Hardware
We The hardware, off-the-shelf NAS and servers (e.g. Mac Mini or PC with RTX 3090,
The setup a NAS, we recommend UGREEN NAS with 4x24TB Drives and 2x8TB SSD drives.
The device provides several great options at a price point of $700 without drives, we recommd:
- 4, 24TB or 26TB Western Digital drives for
- 2, 8TB SSD NVMe drives, preferably Samsung EVO PRO or WD NVme Drive SN850X
As you will the the UGREEN NAS has 2x10GbE ports, you will need also a 10GbE swicth such as TP LINK, TL-SK1008
and some ethernet cables, including 4, 10GbE Cat8 ethernet cables:
Additionally, you may purchase a UPS system to store all your content.
NVIDIA Hardware of MLX Mac Mini or Mac Studio System
You will need an NVIDIA 24GB card such as RTX3090, RTX4090 or RTX5090 systems, one or two Mac Minis can also operate in tandem using Thunderbolt systems to interconnect the model and your embeddings. You may also use 1 or 2 DGX sparks interconnected using ConnectX interfaces
THE SOFTWARE EGLA AI’s Inference Platform
EGLA AI provide a packaged LLM , Indexing services, and processing for all content. We can load all your content from Amazon S3 and connect your system as a one-drive where you can have Amazon S3, Google Drive, Dropbox mounts.
