Finding the Right Server for Your AI

Servers today are among the most necessary servers for organizations, especially as they aim to integrate AI into their operations. Different types of servers can be used for different purposes in the context of AI. For example, while a GPU/AI server is more appropriate for running the AI model; a storage server will be needed to store the training or inference data.

Currently the AI server market is valued at USD 38.3 billion in 2023 according to Global Market Insights with a CAGR of 18% between 2024 and 2032. Comparatively, the global server market is valued at USD 97.6 billion in 2023 according to Skyquest Technology with a CAGR of 9.3%. There are many factors that drive this high growth rate in the AI server market; including the growing demand for AI applications, the continuous developments of AI-specific hardware, the growing investments in AI R&D, etc. However, the market also faces challenges that might deter potential adopters from purchasing AI servers, especially the high cost of hardware and energy consumption.

As AI servers are specifically designed and optimized for dealing with the high computational demands of AI workloads, high powered CPUs and GPUs are vital. Other features such as high memory and storage capacities, as well as redundant high-efficiency power supplies will also be needed to make sure the servers can operate at optimum levels. Leveraging the growing demand for AI hardware and solutions, many manufacturers today have expanded their server product offering to include AI servers.

For example, ASRock recently released the MECAI-GH200 that utilizes the NVIDIA GH200 Grace Hopper™ Superchip. With its 2U MGX form factor, the MECAI-GH200 is one of the most compact platforms that supports the NVIDIA GH200 Grace Hopper™ Superchip that makes it ideal for edge AI applications. It features 2x E1.S (PCIe5.0 x4) drive bays that support 9.5mm width in the front side drive bay and 2x M-key for PCIe5.0 x4 in 22110/2280 form factors.

The ASUS RS720A-E12-RS24 server is powered by AMD EPYC 9004 processors and integrates SupremeRAID™ by Graid Technology for remarkable throughput, low latency and exceptional scalability, setting new industry standards. With 128 Zen 4c cores, 12-channel, up to 4800 MHz DDR5 and support for a maximum TDP of up to 400 watts per socket; the server features a total of 24 bays in combination of Tri-Mode NVMe/SATA/SAS drives on the front panel and nine PCIe 5.0 slots for higher bandwidth and system upgrade.

As part of GIGABYTE’s range of servers, the R243-EG0 is a versatile device that can be used for AI training and inference. It features 1+1 2700W 80 PLUS Titanium redundant power supplies for higher power efficiency and supports 12 x 3.5"/2.5" SATA/SAS hot-swappable bays, 1 x M.2 slot with PCIe Gen3 x4 interface, and 4 x FHFL PCIe Gen5 x16 slots for GPUs. It also features 1 x OCP 3.0 Gen5 x16 slot for higher storage which are necessary for edge AI/ telecom applications.

Ingrasys has also released a series of servers for AI such as the SV2121A which is powered by 2 x 4th Gen AMD EPYC™ Server Processors. It features a modular design to create multiple systems and allows users to reuse and update modules without making unnecessary changes to their system. With 24 x DDR5 RDIMMs, 12-Channel per CPU, up to 4800 MT/s at 1DPC; the SV2121A can be further expanded through its 2 x PCIe 5.0 x16 FHHL Slots and 8 x PCIe 5.0 x16 LP/HHHL with Riser Card Slots. It can be used for various purposes including hyperscale data centers, enterprise application servers, HPC, or AI and Machine Learning.

Inventec's K880G6 is a fully flexible server that provides multiple configurations depending on the users’ needs. By adopting the most powerful 4th Gen Intel® Xeon® Scalable processor, it can be used for general computing, heavy storage applications or high-performance computing. It is extremely scalable with up to PCIe Gen5 x16 DW FHFLGPU card x4. Depending on the user, they might also equip the K880G6 with either forced air or liquid cooling.

The WinFast GS2050T by LEADTEK is powered by Dual Socket E (LGA-4677) 5th/4th Gen Intel® Xeon® Scalable processors and supports CPU TDP 350W up to 64 cores. With 32 DIMM slots, the WinFast GS2050T also features 2x Titanium level 1200W Redundant Power Supplies and a powerful GPU architecture that supports up to 4 double-width passive GPUs (NVIDIA® H100/L40S/A100/A40/A16) or 4 double-width active GPUs (NVIDIA® RTX A6000/A4000). These features enable it to meet the demands of AI, high-performance computing, and 3D rendering applications.

As a brand under MiTAC, the TYAN Transport HX TN85B8261 is a new dual-socket AMD EPYC 9004 barebones server for heavy storage I/O and GPU. It is a 2U 4GPU AI server with 12+12 DIMM slots for 3DS RDIMM DDR5 4800MHz / RDIMM DDR5 4800MHz. It has a maximum capacity of up 6,144 GB with 6x PCIe Gen.5 x16 expansion slots and pre-installed TYAN Riser Cards. The Transport HX TN85B8261 features 1+1 hot-swappable 80 plus Titanium power supplies and 8x 6cm hot-swap middle fans + 2x 8cm easy-swap rear fans for cooling.

The demand for AI applications will continue to drive the need for AI servers as companies aim to develop or host their own AI solutions. Depending on the users’ industry, having a personalized AI server will also be enable higher levels of security and discretion to make sure the user still follows the legal requirements they may have to follow. Manufacturers today offer various options for users who need AI servers with specific features or capabilities. As demand for AI continue to grow, so too will the demand for AI servers.

《Join the AI PC Industry Exploration Forum on May 16, 2024》

As we approach COMPUTEX2024, we will organize a number of special events to offer a glimpse of what to expect in the upcoming COMPUTEX. We will hold the AI PC Industry Exploration Forum which will discuss the experience of GenAI/ LLM integration in various applications and fields as well as the trends and potentials in the AI PC ecosystem. The event will feature speakers and panelists representing Acer, ASUS, Google, Intel, MIC, PEGATRON, and Qualcomm.

The event details are as follows:

• Event Name: AI PC Industry Exploration Forum
• Time: May 16, 13:00 – 16:00 (GMT+8)
• Location: Grand HiLai Taipei Hotel, Platinum C Hall
• Language: Chinese (English live translation available)
• For more details, visit the registration link at:
  >>>https://seminars.tca.org.tw/D17d00982.aspx

■《About COMPUTEX TAIPEI》

COMPUTEX TAIPEI was founded and named by the then Chairman of Taipei Computer Association (TCA), Stan Shih. In 1985, TCA invited TAITRA to be a co-organizer of COMPUTEX TAIPEI. In 2016, the startup focused event, InnoVEX was introduced.

●COMPUTEX TAIPEI 2024 Overview●

This year’s special highlights:
Responding to the rapid development of AI technologies and applications including GenAI and LLM as well as the continued increase in the global digital transformation demands; this year’s COMPUTEX 2024 will focus on “Connecting AI”. This year, COMPUTEX will be joined by major semiconductor and ICT manufacturers including: Acer, ADATA, AMD, ASPEED, ASRock, ASUSTeK, ATEN, BenQ, Clientron, Cooler Master, Delta, ECS, ELAN, FocalTech, GIGABYTE, G.Skill, Innodisk, Intel, Inventec, InWin, ITRI, KIOXIA, Kneron, MediaTek, Microip, Mitac, MSI, NXP, NVIDIA, PEGATRON, Phison, PNY, PSMC, QCT, Qualcomm, Quanta, Realtek, RETRONIX, Seagate, Silicon Motion, Silicon Power, SOLOMON, Supermicro, SYSGRATION, TADA, Taiwan Micro, Thermaltake, Transcend, Wiwynn, and more. In total, 1500 local and international manufacturers will join COMPUTEX 2024 in not only the exhibition, but also forum and keynote speeches as well.

Event Dates: June 4 – 7, 2024
Event Location: Taipei Nangang Exhibition Center (TaiNEX) Halls 1 & 2
Main Theme: Connecting AI
Main Topics: AI computing, advanced communications, future mobility, immersive reality, innovation, green energy, and sustainability.

【Special Events】

• Pre-show Forum: May 16, 2024 [AI PC Industry Exploration Forum] Location: Grand HiLai Taipei Hotel
• Official Award Winners Announcement: May 28, 2024 [BC Awards 2024 Award Winning Products Announcement] Location: Le Méridien Taipei

COMPUTEX Official Website: https://www.computex.biz/
Event information and pre-registration website: https://show.computex.biz/
COMPUTEX CYBERWORLD website: https://show.computex.biz/online.aspx
Facebook: https://www.facebook.com/ComputexTaipei
YouTube Channel: https://www.youtube.com/user/COMPUTEXTAIPEIshow/
LinkedIn: https://www.linkedin.com/company/computex-taipei/