In the past 40 years, chip technology has undergone dramatic changes. In the 1980s and 1990s, Very-Large-Scale Integration (VLSI) and Ultra-Large-Scale Integration (ULSI) technologies laid the foundation for chip design, driving the transistor count to double every two years, resulting in exponential growth in computing power and energy efficiency. This trend facilitated the miniaturization of chipsets, especially the rise of mobile chips, which led the semiconductor industry to focus on mobile chipsets. In the mid-2000s, the advent of smartphones further drove the development of more powerful and complex mobile chips—System-on-Chip (SoC) technologies.
However, with the increasing demand for computational functions and components and the approach of the limits of Moore’s Law, modern SoC design has refocused on the synergistic realization of performance improvement and high-energy-efficient computing. In the AI era, the scale and complexity of AI workloads continue to rise, resulting in pressures of skyrocketing energy consumption and escalating costs. As a result, the industry is pushing for high-energy-efficient computing in AI, covering all technological touchpoints from large data centers to edge devices. In necessary and relevant application scenarios, processing AI workloads at the edge can optimize network bandwidth, strengthen data privacy, and enhance user experience, but it also requires achieving more efficient AI processing capabilities on small devices with limited space and power consumption.
Recently, Arm released a report titled “New Thinking in Chips: The New Foundations for the AI Era,” which focuses on four key trends: first, the creation of specialized architectures to address specific workloads, especially in the AI domain; second, the increase in computational intensity, making energy efficiency crucial; third, the evolution of security technologies to tackle AI-driven threats; and fourth, the pivotal role of the software ecosystem in unlocking the potential of chips.
Kevork Kechichian, Executive Vice President of the Arm Solutions Engineering Division, stated that as traditional semiconductor scaling technologies driven by Moore’s Law approach their physical and economic limits, the industry is accelerating the shift toward innovative solutions such as custom chips, compute subsystems (CSS), and chiplets to break through performance and energy efficiency bottlenecks.
01
Energy Efficiency and Sustainability in AI Computing
The challenge initially comes from energy efficiency and sustainability. AI workloads, from training large models to performing complex inference tasks, present significant challenges in terms of energy efficiency, scalability, and cost-effectiveness.
AI computing demands are massive, requiring substantial power support, and this will continue to grow in the future. From the perspective of chip design, energy consumption mainly originates from computation and data transmission, as well as heat dissipation. Goldman Sachs predicts that AI will drive a 160% increase in data center power demand. To reduce energy costs, chip designs are integrating optimized memory hierarchies and collaborative communication mechanisms to reduce data transmission, while utilizing chip stacking, high-bandwidth memory, and advanced interconnect technologies to lower energy consumption. At the same time, AI frameworks and algorithms are also fine-tuning for performance per watt and unit cost performance to achieve a balance between computing power and economic sustainability.
On the other hand, custom chips designed for specific market needs are becoming an industry trend. Not only do companies like Google, Microsoft, and AWS have their own custom accelerators, but small and medium-sized enterprises are also actively developing custom chips. Arm’s Neoverse CSS, with its proven core computing functionality and flexible memory and I/O interface configurations, accelerates time-to-market while ensuring software consistency and retaining system-level optimization flexibility.
Advanced packaging technologies and processes are another important direction in the recent evolution of chips, and they have also driven the development of chiplets. These technologies allow the stacking and interconnection of multiple semiconductor dies, enhancing performance and energy efficiency while opening up possibilities for modern chip design, such as die-to-die interfaces and new 2.5D and 3D packaging solutions. The booming chiplet market has paved the way for the rise of custom chips, reducing costs, and enabling the use of existing components as building blocks for chips, accelerating development speed, and helping businesses achieve differentiated advantages.
02
Security Technology Challenges and Responses
In the AI technology field, AI-driven cybersecurity threats are becoming more severe, capable of evolving autonomously, identifying system vulnerabilities, and launching complex attacks. A report by DarkTrace in 2024 indicated that nearly 74% of respondents said AI-driven cybersecurity threats had a significant impact on their organizations, and 60% of respondents were concerned that their organizations were not adequately prepared.
Kevork Kechichian stated, “Although AI-driven cyberattacks are becoming an emerging front in cybersecurity threats, the industry is building multi-layered hardware and software defense systems, from encryption technologies embedded in chips to AI-enhanced security monitoring systems. Modern SoC architectures are continuously reinforcing their defenses to withstand both traditional attacks and next-generation threats.” Today, modern SoCs have integrated complex security mechanisms, such as Hardware Root of Trust (RoT), Trusted Execution Environments (TEE), Secure Memory Management Units (MMU), and secure enclaves.
Additionally, the rise of custom chips has introduced higher security demands. Currently, the PSA Certified certification program has become the gold standard in secure chip design, while the IoT Platform Security Evaluation Standard (SESIP), which supplements this certification program, provides a structured approach for security evaluation of custom chips. Furthermore, regulatory environments such as UNECE R155 for vehicle cybersecurity and ISO/SAE 21434 for automotive systems have set strict industry-specific requirements for custom chips.
03
Revolution in Chip Design
Driven by AI technology, the chip design field is undergoing unprecedented transformation. This revolution stems from the dual impact of the explosive growth of AI workloads and the slowdown of semiconductor scaling technology. Traditional chip technologies began facing bottlenecks at the 28-nanometer node, and as the industry pushes toward 7-nanometer and more advanced processes, the challenges have intensified. As we enter the era of fully wrapped gate transistors below the 2-nanometer node, the advantages of traditional scaling technologies are becoming increasingly difficult to maintain, and the costs are high.
A particularly noticeable trend is the slowdown of SRAM scaling: from 5 nanometers to 2 nanometers, SRAM cell area has not significantly reduced, even though there have been improvements in power consumption and performance. Additionally, what is now referred to as “scaling” is primarily achieved through structural innovations known as “scaling boosters,” such as continuous active areas/oxidation diffusion areas, contact on active gate, fully wrapped gate transistors, and backside power supply technologies, which have also become ways to increase transistor density.
As on-chip memory scaling slows down, new computing paradigms, such as advanced AI workloads, have emerged, significantly increasing their demand for memory bandwidth. This trend has driven innovations in memory architecture, including new high-bandwidth memory (HBM) integration solutions. The resulting near-memory computing architecture helps reduce the time and power consumption required for data transfer between memory and processors. Memory has thus become an indispensable component of computing architecture.
The development of AI has driven continuous transformation in chip design. AI workloads require differentiated architectures optimized for large-scale parallel processing and memory bandwidth, driving the evolution of new dedicated accelerators, memory subsystem innovations, energy-efficiency focus, and advances in packaging and integration methods.
Kevork Kechichian stated, “We are witnessing a fundamental transformation in chip design and manufacturing models. As new process nodes require closer collaboration, the traditional boundaries between chip design and manufacturing are gradually disappearing. The new era requires creativity, system-level thinking, and an unwavering pursuit of energy efficiency.”
04
Challenges in the Software Ecosystem
Driven by the rapid development of AI frameworks and the demand for software ecosystems, the AI chip ecosystem is in a continuous process of evolution.
Currently, porting AI models to custom hardware faces challenges of high cost and long timelines. CPUs, with their wide adoption and good compatibility with most AI inference workloads, are favored by developers. The consistency of CPUs effectively avoids fragmentation and inefficiency that may arise with custom hardware. Moreover, interoperability between AI frameworks is crucial, and the ubiquity of CPUs helps ensure broad compatibility.
Due to the lack of standardized practices in AI development, many AI models remain proprietary, which not only limits developers’ ability to optimize performance but also slows down the development process. Open standards have played a key role in breaking down these barriers, enabling developers to achieve seamless migration across different platforms.
To nurture a thriving developer ecosystem, companies must prioritize empowering hardware through strong backend support and actively embrace the evolving AI frameworks. Arm’s report offers four recommendations: effectively use general-purpose tools to simplify development processes and reduce fragmentation; provide pre-built backend support with “out-of-the-box” compatibility for custom chips to accelerate adoption; actively participate in the development of open-source frameworks to ensure compatibility and avoid stagnation in technological development; and, finally, keeping pace with the latest technologies is crucial to maintaining competitiveness as AI frameworks evolve rapidly.
05
Conclusion
The future of computing, especially AI, depends on continuously expanding the possibilities of chip technology, relying on collaboration between hardware and software. Kevork Kechichian emphasized that embracing open standards, achieving interoperability, and providing robust tool support will accelerate innovation and unlock the potential of chip technologies and AI frameworks.
He believes that the evolution of AI chip solutions in the future will be built on a unified, developer-centric ecosystem, focusing on consistency, accessibility, and forward adaptability. The success of future chip designs will increasingly rely on five key factors: close collaboration across IP providers, foundries, and system integrators; system-level optimization of compute-memory-power transmission; standardization of interfaces; specialized architectures for specific workloads; and a robust security framework that can flexibly address emerging threats.
Disclaimer:
- This channel does not make any representations or warranties regarding the availability, accuracy, timeliness, effectiveness, or completeness of any information posted. It hereby disclaims any liability or consequences arising from the use of the information.
- This channel is non-commercial and non-profit. The re-posted content does not signify endorsement of its views or responsibility for its authenticity. It does not intend to constitute any other guidance. This channel is not liable for any inaccuracies or errors in the re-posted or published information, directly or indirectly.
- Some data, materials, text, images, etc., used in this channel are sourced from the internet, and all reposts are duly credited to their sources. If you discover any work that infringes on your intellectual property rights or personal legal interests, please contact us, and we will promptly modify or remove it.