搜索结果：Zero-copy

In current discrete GPU systems, the penalty of data movement between host and device memory is inevitable, forcing many large-scale applications to include optimizations that amortize this cost. On systems like the AMD Instinct<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">™</sup> MI300A series accelerators, based on the accelerated processing unit (APU) architecture, host and device memories are unified into a single physical storage. On an APU, the GPU can access memory in the same way the CPU does, thus avoiding the need for additional data movement (zero-copy). To inform developers of MI300A on expected advantages and potential overheads, we follow an experimental approach to study our OpenMP implementation that leverages MI300A zero-copy. Performance results show that zero-copy is faster than the legacy "copy" implementation by a ratio of 1.2X-2.3X for a production-ready application, but that incurs up to 11% penalty for one SPECaccel 2023 benchmark.

Data centers require high-performance and efficient networking for fast and reliable communication between applications. TCP/IP-based networking still plays a dominant role in data center networking to support a wide range of Layer-4 and Layer-7 applications, such as middleboxes and cloud-based microservices. However, traditional kernel-based TCP/IP stacks face performance challenges due to overheads such as context switching, interrupts, and copying. We present Z-stack, a high-performance userspace TCP/IP stack with a zero-copy design. Utilizing DPDK's Poll Mode Driver, Z-stack bypasses the kernel and moves packets between the NIC and the protocol stack in userspace, eliminating the overhead associated with kernel-based processing. Z-stack em-ploys polling-based packet processing that improves performance under high loads, and eliminates receive livelocks compared to interrupt-driven packet processing. With its zero-copy socket design, Z-stack eliminates copies when moving data between the user application and the protocol stack, which further minimizes latency and improves throughput. In addition, Z-stack seamlessly integrates with shared memory processing within the node, eliminating duplicate protocol processing and serializationldese-rialization overheads for intra-node communication. Z-stack uses F-stack as the starting point which integrates the proven TCP/IP stack from FreeBSD, providing a versatile solution for a variety of cloud use cases and improving performance of data center networking.

The ROS2 software framework is increasingly prevalent in component-based applications for robots and other autonomous systems. Recently added ROS2 features to support zero-copy semantics may significantly reduce latency and latency variation when passing data from one component to another. Additionally, there is a growing trend of developing autonomous robotic systems on heterogeneous computing platforms to exploit hardware acceleration. However, support for portable and reusable zero-copy semantics on heterogeneous compute systems is limited. Such systems thus must either use low-level techniques to manage memory operations directly, which may be tedious and error-prone, or they may not adequately address substantial memory overheads that can arise from repeatedly copying messages and data into and out of device memory associated with GPUs and FPGAs. Towards addressing that limitation of the current state of the art, this paper introduces Hazcat, a new zero-copy framework that automatically performs device memory operations when needed, and avoids copying and other costly operations otherwise. Hazcat is integrated specifically with ROS2 but is also designed for portability to other component-based software frameworks.

Data serialization is critical for many datacenter applications, but the memory copies required to move application data into packets are costly. Recent zero-copy APIs expose NIC scatter-gather capabilities, raising the possibility of offloading this data movement to the NIC. However, as the memory coordination required for scatter-gather adds bookkeeping overhead, scatter-gather is not always useful. We describe Cornflakes, a hybrid serialization library stack that uses scatter-gather for serialization when it improves performance and falls back to memory copies otherwise. We have implemented Cornflakes within a UDP and TCP networking stack, across Mellanox and Intel NICs. On a Twitter cache trace, Cornflakes achieves 15.4% higher throughput than prior software approaches on a custom key-value store and 8.8% higher throughput than Redis serialization within Redis.

In this paper, we present a new scheme, Send Gather Receive Scatter (SGRS), to perform zero-copy datatype communication over InfiniBand. This scheme leverages the gather/scatter feature provided by InfiniBand channel semantics. It takes advantage of the capability of processing non-contiguity on both send and receive sides in the Send Gather and Receive Scatter operations. We have implemented this new design and evaluated the performance for Message Passing Interface level point-to-point microbenchmarks and collectives, on PCI-X and upcoming high performance PCI-Express systems. In our previous work we had come up with an alternate zero-copy approach using multiple RDMA Writes (Multi-W). Compared to the existing Multi-W zero-copy datatype scheme, the SGRS scheme can overcome the drawbacks of low network utilization and high startup cost. On PCI-X platforms, our experimental results show significant improvement in both point-to-point and collective datatype communication. The latency of a vector datatype can be reduced by up to 62% and the bandwidth shows improvement up to 400% as compared with the Multi-W scheme. The Alltoall collective shows up to 23% reduction in latency. Further, the SGRS scheme shows low CPU overhead with a potential promise for better computation and communication overlap. The experimental results on PCI-Express platforms demonstrate the relevance of zero-copy protocols to overcome memory bandwidth limitations. The trends we observe in PCI-X platform are magnified on PCI-Express platforms with even higher improvement for the microbenchmarks and collectives.

Zero-copy communication exchanges the messages among the buffers that are allocated and locked before the communication itself. This communication style fits into applications that the communication timings and the message sizes are known in its initialization phase. However, another application with non-deterministic messaging such as web or parallel database can not fit into the style because the sizes and timings of its messages change at every communication. This paper proposes a new zero-copy communication style for these kind of application, called active zerocopy, that receives messages without pre-allocated buffers. The performance evaluation with the active zero-copy comparing with the conventional zero-copy, when the applications with non-deterministic messaging is applied, shows its efficiency.

Zero-copy has been a hot research topic for a long history, which is an underlying technology to support many applications, including multimedia retrieval, datamining, efficient data transferring, and so on. Zero-copy means during message transmission, there is no data copy among memory segments on any network node. When a message is sent out, the data packets in user application space go through network interface directly and reach outside of the network; and when receiving a message , the same way is used, meaning the data packets are transmitted into user application space directly. In this paper we, present the design and implementation of zero-copy technology for the Linux (kernel version 2.6.11) operating system, by modifying its network device driver snull.c and improving on the COW (copy-on-write) technology,. The main method we used is the combination of MMAP and PROC procedures to implement the test program and the test strategies, and finally we successfully simulated the ARP protocol module with the VHDL language.

Embedded systems are usually resource limited in terms of processing power, memory, and power consumption, thus embedded TCP/IP should be designed to make the best use of limited resources. Applying zero-copy mechanism can reduce memory usage and CPU processing time for data transmission. Power consumption can be reduced as well. In this paper, we present the design and implementation of zero-copy mechanism in the target embedded TCP/IP component, LyraNET, which is derived from Linux TCP/IP codes and remodeled as a reusable software component that is independent from operating systems and hardware. Performance evaluation shows that TCP/IP protocol processing overhead can be significantly decreased by 23-56.22%. Besides, object code size of this network component is only 78% of the size of the original Linux TCP/IP stack. The experience of this study can serve as the reference for embedding Linux TCP/IP stack into a target system and improving the transmission efficiency of Linux TCP/IP by zero-copy implementation.

Memory copies are widely regarded as detrimental to the overall performance of applications. High-performance systems make every effort to reduce the number of memory copies, especially the copies incurred during message passing. State of the art implementations of message-passing libraries, such as MPI, utilize user-level networking protocols to reduce or eliminate memory copies. InfiniBand is an emerging user-level networking technology that is gaining rapid acceptance in several domains, including HPC. In order to eliminate message copies while transferring large messages, MPI libraries over InfiniBand employ ldquozero-copyrdquo protocols which use remote direct memory access (RDMA). RDMA is available only in the connection-oriented transports of InfiniBand, such as reliable connection (RC). However, the unreliable datagram (UD) transport of InfiniBand has been shown to scale much better than the RC transport in regard to memory usage. In an optimal design, it should be possible to perform zero-copy message transfers over scalable transports (such as UD). In this paper, we present our design of a novel zero-copy protocol which is directly based over the scalable UD transport. Thus, our protocol achieves the twin objectives of scalability and good performance. Our analysis shows that uni-directional messaging bandwidth can be within 9% of what is achievable over RC for messages of 64 KB and above. Application benchmark evaluation shows that our design delivers a 21% speedup for the in.rhodo dataset for LAMMPS over a copy-based approach, giving performance within 1% of RC.

Apache Kafka is an open-source distributed publish-subscribe system, which is widely used in data centers for messaging between applications, log aggregation, and stream processing. The existing Kafka implementation uses TCP/IP for communication, which has various inefficiencies such as a high message dispatch cost due to OS involvement and excessive memory copies. Recently, the availability of cost-effective RDMA-capable network controllers within data centers and cloud infrastructures have encouraged many modern applications to adopt RDMA networking, which offers the potential to outperform classical TCP/IP. We introduce KafkaDirect, an extension to Apache Kafka, that uses RDMA to accelerate the three most network intensive datapaths: record production, record replication, and record consumption. In this work, we explore the design choices including which RDMA operations to use to take full advantage of offloaded communication. Our RDMA design relies on one-sided RDMA requests to attain true zero-copy communication completely avoiding the need for using intermediate buffers in Kafka servers, thereby ensuring low latency and high throughput communication. KafkaDirect can offer up to 9x increase in throughput for both Kafka producers and Kafka consumers, and can provide 4x and 50x reduction in latency for Kafka producers and Kafka consumers, respectively.

RPC has established itself as one of the more powerful communication paradigms for distributed computing. In recent years, object-oriented languages have impacted RPC semantics, with a number of variants providing remote method invocation and various forms of distributed object systems. At the same time, performance has changed little with the bottleneck being the network transport, in particular the in-kernel protocol implementations. This paper describes J-RPC, an RPC architecture that leverages user-level network interfaces (UNI) to circumvent the kernel on the critical path. It describes how the wire format and the RPC system can be engineered to allow zero-copy reception of Java objects and zero-copy transmission of arrays. All objects received are fully type-checked and can be directly used by the receiving program. The design is connection-oriented for performance and leverages the JVM’s garbage collector when managing receive buffers. An implementation built from an off-the-shelf JVM and a commercial UNI is used to evaluate the architecture and the tradeoffs of type-safe, zero-copy data marshaling. 1

Zero-copy means during message transmission, there is no data copy among memory segments on any network node. When message is sent out, the data packets in user application space go through network interface directly and reach outside of network; and when message is receiving, the same way is used, means the data packets are transmitted into user application space directly. In order to implement the zero-copy, the main technology we used here is DMA data transmission technology and memory mapping technology. In this paper, we based on Linux (kernel version 2.6.11), by modifying its network device driver snull.c and improving on COW technology, we implement the zero-copy technology. The main method we used is the combination of MMAP and PROC procedures, and at the end, we also introduce the implementation of test program and the test strategies.

This paper describes a new feature in Solaris that uses virtual memory remapping combined with checksumming support from the networking hardware, to eliminate data-touching overhead from the TCP/IP protocol stack. By implementing page remapping operations at the right level of the operating system, and caching MMU mappings to take advantage of locality of reference, significant performance gain is attained on certain hardware platforms. Nevertheless, the performance improvement over CPU copying varies, depending on the host memory cache architecture, MMU design, and application behavior. We begin by comparing different zero-copy schemes, and explain our preference for page remapping and copy-on-write (COW) techniques. We then describe our implementation, and present its performance characteristics under a number of different parameters. We conclude with ideas for future improvements.

Noncontiguous I/O access is one of the main access patterns in parallel and distributed applications. An I/O architecture EXIO enables Globus, a popular run-time environment for distributed computing, on RDMA networks such as InfiniBand. In this paper, we investigate the benefits of InfiniBand zero-copy RDMA to noncontiguous I/O on Globus. Our experimental results demonstrate that, by enabling zero-copy RDMA on InfiniBand, EXIO significantly improves the performance of Globus noncontiguous I/O. Compared to the packing and unpacking, zero-copy RDMA improve the bandwidth by up to 2.7 times. Compared to both IPoIB and 10GigE, it increases the bandwidth by more than three times. While achieving efficient noncontiguous I/O, RDMA-based noncontiguous I/O on InfiniBand also leads to dramatical reduction of CPU utilization on Globus clients and servers.

We present a new Zero-Copy approach for message passing in a tightly coupled, multi - process parallel discrete event simulation. Our approach is highly scalable and is suited for large scale distributed simulations. A Zero-Copy approach never copies message content. Rather, messages are created initially in a shared - memory region, and only a smart pointer referring to the shared memory object is passed to a message recipient. The smart pointer can be dereferenced normally, and has reference count semantics allowing memory reuse when no references remain. This approach significantly reduces the amount of data copied between processes and results in considerable improvement in overall application performance as compared to more traditional shared - memory based message passing. We demonstrate the efficiency of our approach using two distributed discrete event simulators using conservative synchronization. However, we also included knowledge of the Global Virtual Time and the Current Simulation Time in our memory management algorithms, allowing rollbacks and reclaiming of memory that was freed prematurely.

Sockets direct protocol (SDP) is an industry standard pseudo sockets-like implementation to allow existing sockets applications to directly and transparently take advantage of the advanced features of current generation networks such as InfiniBand. The SDP standard supports two kinds of sockets semantics, viz., synchronous sockets (e.g., used by Linux, BSD, Windows) and asynchronous sockets (e.g., used by Windows, upcoming support in Linux). Due to the inherent benefits of asynchronous sockets, the SDP standard allows several intelligent approaches such as source-avail and sink-avail based zero-copy for these sockets. Unfortunately, most of these approaches are not beneficial for the synchronous sockets interface. Further, due to its portability, ease of use and support on a wider set of platforms, the synchronous sockets interface is the one used by most sockets applications today. Thus, a mechanism by which the approaches proposed for asynchronous sockets can be used for synchronous sockets is highly desirable. In this paper, we propose one such mechanism, termed as AZ-SDP (asynchronous zero-copy SDP), where we memory-protect application buffers and carry out communication asynchronously while maintaining the synchronous sockets semantics. We present our detailed design in this paper and evaluate the stack with an extensive set of benchmarks. The experimental results demonstrate that our approach can provide an improvement of close to 35% for medium-message unidirectional throughput and up to a factor of 2 benefit for computation-communication overlap tests and multi-connection benchmarks

Traditional UNIX - I/O interfaces are based on copy semantics, where read and write calls transfer data between the kernel and user-defined buffers. Although simple, copy semantics limit the ability of the operating system to efficiently implement data transfer operations. In this paper, we present extensions on the traditional UNIX interfaces that are based on explicit buffer exchange. Instead of transferring data between user-defined buffers and the kernel, the new extensions transfer data buffers between the user and the kernel. We study using the new interfaces in typical application programs, and compare their use to the stan-dard UNIX interfaces. The new interfaces lend themselves to an efficient zero-copy data transfer implementation. We describe such an implementation in this paper, and we examine its performance. The implementation, done in the context of the SolarisTM operating system, is very efficient: for example, on a typical file transfer benchmark, the network throughput was improved by more than 40% and the CPU utilization reduced by more than 20%.

Derived datatypes are commonly used in MPI applications to exchange non-contiguous data among processes. However, state-of-the-art MPI libraries do not offer efficient processing of derived datatypes and often rely on packing and unpacking the data at the sender and the receiver processes. This approach incurs the cost of extra copies and increases overall communication latency. While zero-copy communication schemes have been proposed for contiguous data, applying such techniques to non-contiguous data transfers bring forth several new challenges. In this work, we address these challenges and propose FALCON - Fast and Low-overhead Communication designs for intra-node MPI derived datatypes processing. We show that the memory layouts translation of derived datatypes introduce significant overheads in the communication path and propose novel solutions to mitigate such bottlenecks. We also find that the current MPI datatype routines cannot fully take advantage of the zero-copy mechanisms, and propose enhancements to the MPI standard to address these limitations. The experimental evaluations show that our proposed designs achieve up to 3 times improved intra-node communication latency and bandwidth over state-of-the-art MPI libraries. We also evaluate our designs with communication kernels of popular scientific applications such as MILC, WRF, NAS MG, and 3D-Stencil on three different multi-/many-core architectures and show up to 5.5 times improvement over state-of-the-art designs employed by production MPI libraries.

Internet services like the world-wide web and multimedia applications like news- and video-on-demand have become very popular over the last years. Since a high and rapidly increasing number of users retrieve multimedia data with high data rates, the data servers can represent a severe bottleneck. Traditional time and resource consuming operations, like memory copy operations, limit the number of concurrent streams that can be transmitted from the server, because of two reasons: (1) memory space is wasted holding identical data copies in different address spaces; and (2) a lot of CPU resources are used on copy operations. To avoid this bottleneck and make memory and CPU resources available for other tasks, i.e. more concurrent clients, we have implemented a zero-copy data path through the communication protocols to support high-speed network communication, based on UVM. In this paper, we describe the implementation and evaluation of the zero-copy protocol mechanism, and we show the potential for substantial performance improvement when moving data through the communication system without any copy operations.

The message passing interface (MPI) is the defacto standard for parallel programming. As system scales increase, application writers often try to increase the overlap of communication and computation. Unfortunately, even on offloaded hardware such as InfiniBand, performance is not improved since the underlying protocols within MPI implementation require control messages that prevent overlap without expensive threads. In this work we propose a fully-asynchronous and zero-copy design to allow full overlap of communication and computation. We design TupleQ with novel use of InfiniBand eXtended Reliable Connection (XRC) receive queues to allow zero-copy and asynchronous transfers for all message sizes. Our evaluation on 64 tasks reveals significant performance gains. By leveraging the network hardware we are able to provide fully-asynchronous progress. We show overlap of nearly 100% for all message sizes, compared to 0% for the traditional RPUT and RGET protocols. We also show a 27% improvement for NAS SP using our design over the existing designs.