162

 Thank you very much for coming. We are very excited to discuss about the possibility of the IOWN data center infrastructure service. Actually, we are now doing the IOWN PoC together with NTT and NVIDIA. My name is Sugiyama. I'm a chief technology strategist in the Red Hat. I'm a director and I'm leading the IOWN infrastructure. I'm now Oguchi from Fujitsu. I'm director of infrastructure system business unit and responsible for developing OSS for our server product.

IOWN stands for the innovative optical and wireless network. IOWN global forum and I made MOU with the Linux foundation to collaborate three projects, open programmable infrastructure project, LF EDGE project, and LF networking project. In the LF EDGE project, our information model task force is now working to explore the new blueprint based on the IOWN infrastructure. In the LF networking, there's several potential opportunities to collaborate, for example, the open daylight transport PCE project, which can be one of the IOWN APN controller that IOWN member is keen to explore. I'd like to highlight the open programmable infrastructure project this time because this is the first trigger when we IOWN global forum collaborate with the Linux foundation. You can see that this diagram, we have a concept called function dedicated network, which will enable network functionality inside PCIe card or inside fight box gateway. That's why we started to work with open programmable infrastructure project. And that is one of the funding for the open programmable infrastructure project. We launched a collaboration session in the IOWN member summit last year. Since then, we are deeply starting to collaborate with each of the projects in the Linux foundation.

And I think maybe some of you might know about IOWN global forum, but we still need to explain what is IOWN global forum. And our technical committee chair, Clara Li, unfortunately she's not available here, but she made a video for you guys. So I quickly ran her video to introduce about IOWN global forum. And also for what you might need to know about what is each member company's approach to the IOWN global forum. So that's why I invited Kurebayashi san and he will talk about any approach about IOWN global forum, how they can enable the IOWN global forum technology. And then also I covered the OPI project. Unfortunately, the chair of the use case working group with OPI project is not available here, unfortunately. But I will cover his part about why we need OPI in the IOWN global forum data center infrastructure. Then one of the key features we are now adopting is composable disaggregated infrastructure. We try to adapt disaggregated computing to compose multiple types of devices like DPU, IPU, and GPU together with Intel CPU. So I invited Oguchi of Fujitsu, he will talk about current status of the composable disaggregated infrastructure development status and also he will share the current gap, what is the challenge to enable to realize IOWN technology.

 Without leaving around the quick video, that Clara is...

Hello everyone, I'm Clara Li. I work for Intel as a senior personal engineer on standards. I also serve as IOWN global forum technology senior computer. Thank you to Sugiyama-san for inviting me to the session. But unfortunately I couldn't attend in person. I hope this video can give you an overview on IOWN global forum. I'm a senior personal engineer on standards. I work for Intel as a software engineer on standards. I also serve as IOWN global forum technology senior computer. The objective of the forum is to develop reference architecture, frameworks, certifications for the next generation communication and computing infrastructure, and leverage the evolution of optical communication and photonics, electronics, and

So this page shows the overall IOWN global forum work landscape. On the use case side, the forum looks into various consumer and industrial use cases, such as area management, industry management, level entertainment. It also works on building a digital tool framework for the various use cases. On the technology side, the forum develops open all photonics network, data centric infrastructure, and IOWN security, that targets at enabling future communication computing infrastructure based on the of the methods of throughput, energy efficiency, and enable post-quantum security. Build on top of the communication computing infrastructure, the forum also looks into domain-specific technologies, such as mobile network front-haul and network functional visualization, data storage and exchange, and fiber sensing.

To date, IOWN forum has published several deliverables on 2030 vision, use cases and requirements, technology outlook, functional architecture, reference implementation model, and proof-of-concept references. All documents can be accessed in the IOWN forum website. So this concludes the brief overview on the forum, and I will hand over to Sugiyama-san to continue the session.

 So we have many documents already published, actually. So you can download the document and read, but there are lots of documents, actually. So it's not easy to read everything. So that's why we are here, and we try to have this kind of session many times to discuss with you guys, to discuss the possibility to use the IOWN forum. Actually, PoC is our external activity that any guy can try to use the IOWN infrastructure to contact PoC together, actually.

And another probably question you might have, what is IOWN member companies doing in the IOWN global forum? So I'm inviting the NTT Kurebayashi-san, and he will talk about the NTT's approach about IOWN. As you might know, the NTT announced that they are trying to launch the IOWN 1.0 service to deliver all photonics networks, but it's not only the all photonics network service. There are many things that are working in the IOWN global forum, especially the computing industry, evolution. They are working on the new computing architecture. So I will hand over to Kurebayashi-san. Okay, so here I'd like to explain the concept and also challenges of IOWN. And IOWN is our next generation of computing ICT infrastructure, aiming to achieve a countable leap of enhanced broadband, lower power, and lower latency. And this slide shows the target performance, and the target performance will be 100 to 200, better than those of the conventional technologies. So actually, the target performance is not easy to achieve, so collaboration is very important to gather many people, narrated of many people from various fields. So we have started IOWN global forum and also started discussion with relevant communities, such as the next foundation. And the key technology of IOWN is electronics to photonics.

So this slide shows the approach of IOWN for networking. Networking of IOWN is going to fully leverage optical technologies, so IOWN network is going to shift from repeat of switching to direct connection to APN. This approach can realize a direct connection with the broadband and also low latency.

And this slide shows the approach of IOWN for computing infrastructure. IOWN's computing infrastructure is trying to leverage also photonics technologies towards next generation disaggregated computing infrastructure. This shifts from a box-oriented server-based infrastructure to a fully disaggregated computing infrastructure. So component devices which compose a conventional server are disaggregated and connected with each other via a high-speed photonic network. And also forming a hardware resource pool. And this approach can improve resource usage and also power efficiency by allocating the necessary amount of computing resources to workloads at the component device level.

So this slide shows the data center infrastructure for CPS. CPS means Cyber Physical System. And to improve performance and energy consumption in massive data handling and processing in CPS, we are going to apply APN and also disaggregated computing technologies to this infrastructure. The computing sites are connected with each other via APN, we call it a data center extent. And also we apply disaggregated computing technologies to each computing site. And also hardware resource pool will help to create hardware accelerated data pipeline for efficiently handling or processing data coming from many devices.

So IOWN is a very long-term concept, but we are also taking a step-by-step approach to demonstrate the part of our IOWN concept, also to promote early adoption of IOWN technology in the business. So this shows one of the such PoC is real time video analysis. And this PoC is supported by Fujitsu, NTT, NVIDIA, and Red Hat. In this PoC, we are going to demonstrate efficient utilization of geographical distributed resources with lower latency and lower power consumption. We are trying to use two different server configurations, one is x86 plus GPU-based and the other is converged device-based for AI inference. And we are also trying to apply hardware accelerated technologies such as advanced uploading to GPU and exchange data transferring across APN based, one with GPUDirect RDMA and also converged device-based low-power AI system. And we are also trying to apply management platform OpenShift. So this PoC is integrated with our container platform as well.

So, key word for the DCI is dynamicity, I think. And the type and the number of accelerators can vary depending on the various factors. So it is necessary for the next-generation infrastructure to create, modify these logical servers with appropriate configuration on demand. There are many factors to change such appropriate configuration and one of such factor is business growth.

 And as shown in this slide, according to the business growth, we need appropriate scale up or scale out for meet the customer's demand.

So I'd like to show several examples of scaling. One is for x86 CPU plus GPU-based configuration. This is a typical configuration for AI and DCI as a service will be capable of flexible scale up and scale down and also scaling out and scaling in for the given workload.

And the next slide shows the scaling of converged device-based. converged device means accelerator card which integrates both a DPU/IPU and also GPU. In this case, we can take a more simply scaling approach because a DPU, a converged card, has both functionality of networking and also AI function. So this shows the dynamicity of what we want to support and Sugiyama-san and Oguchi-san will show the potential candidate for this solution.
 
Thank you. And let me add one more information. Because I'm also a member of this PoC team to enable the open shift on the x86 host to expand the GPU and smart lake. In addition to that, we are now implementing the micro shift which is a mini open shift, a mini Kubernetes running inside an ARM CPU in the converged GPU accelerator which in this case is NVIDIA A100X. So we are now supporting two types of patterns. Pattern A is just running opposite on top of the x86 CPU to expand the multiple GPU. And pattern B is that we enable the micro shift inside the GPU and converged GPU accelerator to scale out the multiple GPU, converged GPU accelerator. So we try to keep both patterns in order to keep the flexible deployment. It's up to the service provider how to deploy, how to expand the system. One of the current bottlenecks is that because we are now basically using the pre-configured COTS server which is not possible to scale up without changing the system. But in the ARM data center infrastructure, we can logically scale up with adding the mobile PCI device card through the composable Data Centric infrastructure. That is what I would like Oguchi-san to explain later. You can see that pattern B is similar to what our open programmable infrastructure project is doing. They are now trying to enable the Linux OS inside the GPU and the IPU. That's why we are starting to work with the open programmable infrastructure project.

Maybe I just briefly explain about what is the open programmable infrastructure project. Because we are a member of the open programmable infrastructure project, we are now targeting to enable the Linux OS inside the GPU and the IPU to increase the intelligence of each infrastructure service, network service, storage service, and managed security service. Because we see that many of the functionality so far is available. We can try to integrate the offload functionality inside the GPU and the IPU and to add more control and intelligence for the network infrastructure service, storage service, before transferring the user data to the x86 CPU. There are many solutions there. We are now exchanging ideas through the OPI use case working group.

 Actually in the OPI, there is an event held in the US, the OCP Global Summit. Their member is demonstrating one of the activities. They enabled the open system GPU and running the NGINX proxy to manage the user traffic and transfer the user data to the x86 host open shift. This is one of the scenarios they are now enabling. But there are many use cases actually we are now discussing.

I will try to cover this slide. Basically the IOWN Global Forum is starting to work with the use case working group in the OPI project to share the use case each other to explore the common goal. Actually last time we shared the network use case and they also agreed to adopt that use case. It's a UPF use case. Also this AI use case is one of the candidates to work together with the OPI. In the OPI project, they are also targeting the AI and enabling the GPU for the AI/ML Federation. In order to do this, they need more high bandwidth that the IOWN can provide. So we have many opportunities to collaborate together. And I would also highlight why OPI is needing data center infrastructure. As I mentioned a little bit about IOWN data center infrastructure, adopting this side of computing.

 So when we enable this side of computing, one of the concerns is that there might be too much transaction within the PCI fabric between the CPU host and PCI card. But OPI, if we use OPI, we enable the network functionality or stage functionality inside the GPU and the GPU. So that means that we don't need transit that may traffic over the PCI to the CPU. Because all functionality is offloaded inside the GPU and the GPU. So this is why we are now working with the OPI team on how to enable the GPU and the GPU in the composability side of the infrastructure. It's not only just a simple card, actually. It's more intelligent card we are now adopting inside the GPU and the GPU. In our case, we can try to enable the open shift inside the GPU and the GPU or micro shift into the GPU and the GPU.

This is a concept about data center infrastructure as a service. IOWN data center infrastructure system to enable to compose a large type of logical service node for each workload. It's kind of the purpose of using logical service node. Logical service node means the kind of COTS server, but dynamic configurable COTS server. We can try to integrate x86 CPU along with many GPU, many GPU, and many GPU depending on the transaction, actually. So, for example, in this case, ingestion node with AI inference, AI inference node needs many GPU cards. Not necessarily use many CPU, x86 CPU. Within the single x86 CPU, we can add many GPU PCI cards in the composability side of the infrastructure. That's why we need composability side in the infrastructure feature. So you can compare this. In this table, I just list up one of the examples. We can add 20 GPU for one CPU. So we can scale up the AI performance. Similar thing in the 5G network, when we build the unit, we can use the GPU for the layer accelerator. Similar thing in the current x86 CPU. Actually, the most of the cases, the maximum MIMO is 60, 40, 64. We can expand 20, maybe 20, 60, 40, 64. We can upgrade 20 times larger than existing network. We can upgrade many later units if we can expand the GPU with the scale-up in the single CPU. So it's a composability side in the infrastructure feature is one of the key technology.

So I'd like to handle what is the composability side in the infrastructure and what is the current challenge of the composability side in the infrastructure. Okay. I would like to explain the composable disaggregated infrastructure.
 
Composable disaggregated infrastructure, CDI, is emerging new server architecture. It disaggregates existing servers into separate components in the resource pool. And then it composes a custom-made server by software definition. We call these servers "composed bare metal."

How does it work? In resource pool, all components are connected to PCIe or CXL or photonics switches. And CDI management software controls the switches based on user demand so as to create composed bare metal.
 
CDI has some features. The most notable feature is to create custom-made servers on demand. For example, it can create GPU-rich servers or memory-rich servers. It also creates server clusters. This feature is especially helpful for IOWN and DCI node. When these servers are not used anymore, users can return those servers to resource pool. So this architecture minimizes unused resources and reduces the total cost.
 
The second feature is it enables the composed bare metals to detach or attach PCIe devices depending on the workload. Let's consider AI system for example. When the system is used AI-learning at nighttime, CDI can attach GPUs and provide higher performance for learning. And when the system is used for AI inference at daytime, CDI can detach GPUs and save power consumption. So by using CDI, users can balance high performance and power saving. To summarize, CDI has some advantages.

 It contributes to saving energy consumption and reducing total cost as described left figure. But of course there is a trade-off. CDI has advantages in total cost only when using lots of GPUs. And one more feature, it enables composed bare metals to scale up and down. From these advantages, CDI is expected to be a solution for IOWN and DCI node.

 We talked about IOWN data center infrastructure activity and talked about what is composed of two-side infrastructure. Now the discussion time. I also have some questions to cover. Why the IOWN data center infrastructure is needed? Why the CDI can fit the IOWN data center infrastructure sub-system?

 IOWN data center infrastructure has benefits for both provider side and user side. For provider side, utilization of APN relaxes constraints at distance. So now it's getting very difficult to construct a data center in city area. But by utilizing APN, we can construct data center in one sub-area. So that kind of flexibility we can get by using APN. And also component device level technology will increase usage of resource and also save power consumption. This means we can get better OPEX and also CAPEX. This means we can reduce the cost and also this means user benefit because the price also will be reduced. For users side, by using this infrastructure, we can support more wide range of use cases such as enhanced broadband and also latest use cases. We can support these kind of more advanced use cases by using data center infrastructure.

 Thank you very much. So within all photonics networks nationwide in Japan, we can design the computer infrastructure. Especially for AI, we need to exchange many data. So this is one of the potential use cases in the AI federation. At least in Japan, we have lots of fiber. We can eliminate many electric devices. That means we can reduce energy consumption. That is one challenge. That's why we set many high goals. Why the CDI can fit the data center infrastructure? Could you answer that?

I think there are two main requirements of IOWN DCI node. The first one is create DCI node, sorry, logical service node, LSN, on demand. And the second requirement is to balance higher performance and power saving. For the first requirement, CDI create composed bare metal on demand based on user demand. For the second requirement, CDI enables composed bare metal dynamically scale up and down. As a result, it contributes to not only provide higher performance but also saving power. For this reason, CDI seems to satisfy the requirements. It is expected to be the solution for IOWN DCI node.

Any other question about the composable disaggregated infrastructure? No? Another question. Sorry, I didn't listen. What about the CXL? I didn't hear much about the CXL in your presentation. Do you have any idea to adapt the CXL technology?

 Yeah. Actually, we developed right now using PCI Express switch. But next year, we will use CXL switch. CXL switch provides CXL memory. User can use larger memory space. And using CXL version 3, user can also use CPU pool. So there are lots of possibilities using CXL technologies.

 Conposable disaggregated infrastructure is now ready to deploy within the existing PCI path. When the CXL is available, CDI itself can extend to support. So you are not PCI device support at the CXL interface? 

Yes. 

Thank you very much. What is the current challenge?

As Sugiyama-san said, our use case like a CPS need to handle massive amount of data and also process such data. This task sometimes overwhelms CPU, so we need to have efficient offloading technologies. Now many vendors provide many kinds of TPU like technologies. So the key challenge will be unified management for such new technologies. This kind of diversity makes it difficult to handle new technologies. So I think OPI is a good community to address this problem.

Unified management is so important, that's why we are starting to work with OPI, Open Platform Infrastructure Project. So we can discuss how we can manage multi vendor environment. What about you, Oguchi-san?

 We can automatically attach or detach PCI devices to comporsable baremetals depending on the user workload. We can also give instructions to CDI. I think Kubernetes Dynamic Resource Allocation, DRA, can be a solution. There are two challenges. First, we need to specify the standard of communication way between DRA and CDI. Second, programmable devices such as GPU and DPU have to program on it before hot removing. We need some inter-working mechanism. So we need to work on these challenges.

 I think this topic might be interesting to bring to the OPI project. How to manage within CDI. I'd like to give you guys our next step for each.

 I'd like to conduct a PoC one by one. I'll show the current PoC, but we're going to combine CDI technologies and also converged accelerator technologies. We'd like to show the effectiveness soon.

We developed some prototype for our challenge. We plan to do some presentation in next KubeCon. If we can do presentation, it's happy.

 Thank you very much.