文章
57[翻译] 简谈编译、汇编、链接和加载 – part IITanslated from： http://www.tenouk.com/ModuleW.html W.2 对象文件与可执行文件 源文件经历汇编后会变成对象文件（文件名通常以.o为后缀）,再经过链接后，就变成了可执行文件。 对象文件与可执行文件的格式有几种。如Linux上的ELF (Executable and Linking Format) 格式以及Windows上的 COFF (Common…58关于高可用的系统在《这多年来我一直在钻研的技术》这篇文章中，我讲述了一下，我这么多年来一直在关注的技术领域，其中我多次提到了工 […]59[翻译] 简谈编译、汇编、链接和加载 – part ITanslated from： http://www.tenouk.com/ModuleW.html 本文详细介绍了一个C/C++程序从源文件到进程的过程。但它仍然只是我的一个缓冲溢出教程的节选。本文试图解释C/C++源码如何经历预处理（pre-processed）、编译（compiled）、链接（linked）以及最后被加载（loaded）为一个运行的进程（process）的过程。文章基于GCC…60这多年来我一直在钻研的技术因为我是看到tinyfool 《那些年我赶过的时髦技术趋势》，在赞叹的时候，也让我对我有好些回忆，所以想写一篇 […]61关于量化投资、IT及教育的一点随想如果你在做股票， 你可能会意识到现在量化投资有越来越火的迹象。国内目前几个免费的量化投资平台，像米筐，优矿和聚宽等， 使用的编程语言都是Python。 我想要表达的意思是： 即使你无意从事IT行业， 你也应该学习一门编程语言， 熟练掌握一个操作系统的使用。而计算机或者是一个编程语言， 他们作为一个工具，对各行各业都有很大的影响。随着时间的推移， 这种影响会越来越大。 我也想就此再发挥一下， 聊一点…62Protecting Netflix Viewing Privacy at Scale<div dir="ltr"><span>On the Open Connect team at Netflix, we are always working to enhance the hardware and software in the purpose-built Open Connect Appliances (OCAs) that store and serve Netflix video content. As we mentioned in a </span><a href="https://media.netflix.com/en/company-blog/how-netflix-works-with-isps-around-the-globe-to-deliver-a-great-viewing-experience"><span>recent company blog post</span></a><span>, since the beginning of the Open Connect program we have significantly increased the efficiency of our OCAs - from </span><span>delivering 8 Gbps of throughput from a single server in 2012 to over 90 Gbps from a single server in 2016. We contribute to this effort on the software side by optimizing every aspect of the software for our unique use case - in particular, focusing on the open source </span><a href="https://www.freebsd.org/"><span>FreeBSD</span></a><span> operating system and the </span><a href="https://www.nginx.com/"><span>NGINX</span></a><span> web server that run on the OCAs.</span></div><b><br></b><br><div dir="ltr"><span>Members of the team will be presenting a technical session on this topic at the </span><a href="http://myeventagenda.com/sessions/0B9F4191-1C29-408A-8B61-65D7520025A8/14/5#sessionID=1362"><span>Intel Developer Forum (IDF16)</span></a><span> in San Francisco this month. This blog introduces some of the work we’ve done.</span></div><h3 dir="ltr"><span>Adding TLS to Video Streams</span></h3><b><br></b><br><div dir="ltr"><span>In the modern internet world, we have to focus not only on efficiency, but also security. There are many state-of-the-art security mechanisms in place at Netflix, including Transport Level Security (TLS) encryption of customer information, search queries, and other confidential data. We have always relied on pre-encoded Digital Rights Management (DRM) to secure our video streams. Over the past year, we’ve begun to use Secure HTTP (HTTP over TLS or HTTPS) to encrypt the transport of the video content as well. This helps protect member privacy, particularly when the network is insecure - ensuring that our members are safe from eavesdropping by anyone who might want to record their viewing habits.</span></div><b><br></b><br><div dir="ltr"><span>Netflix Open Connect serves over 125 million hours of content per day, all around the world. Given our scale, adding the overhead of TLS encryption calculations to our video stream transport had the potential to greatly reduce the efficiency of our global infrastructure. We take this </span><a href="http://techblog.netflix.com/2015/05/netflix-streaming-more-energy-efficient.html"><span>efficiency seriously</span></a><span>, so we had to find creative ways to enhance the software on our OCAs to accomplish this objective.</span></div><b><br></b><br><div dir="ltr"><span>We will describe our work in these three main areas:</span></div><div dir="ltr"><span> </span></div><ul><li dir="ltr"><div dir="ltr"><span>Determining the ideal cipher for bulk encryption</span></div></li><li dir="ltr"><div dir="ltr"><span>Finding the best implementation of the chosen cipher</span></div></li><li dir="ltr"><div dir="ltr"><span>Exploring ways to improve the data path to and from the cipher implementation</span></div></li></ul><span><br></span><span>Cipher Evaluation</span><br><b><br></b><br><div dir="ltr"><span>We evaluated available and applicable ciphers and decided to primarily use the </span><span>Advanced Encryption Standard (AES</span><span>) cipher in </span><span>Galois/Counter Mode (GCM), </span><span>available starting in TLS 1.2</span><span>. We chose AES-GCM over the Cipher Block Chaining (CBC) method, which comes at a higher computational cost. The </span><span>AES-GCM cipher algorithm encrypts and authenticates the message simultaneously - as opposed to AES-CBC, which requires an additional pass over the data to generate keyed-hash message authentication code (HMAC). </span><span>CBC can still be used as a fallback for clients that cannot support the preferre</span><span>d method.</span></div><b><br></b><br><div dir="ltr"><span>All revisions of Open Connect Appliances also have Intel CPUs that support AES-NI, the extension to the x86 instruction set designed to improve encryption and decryption performance. </span></div><div dir="ltr"><span>We needed to determine the best implementation of AES-GCM with the AES-NI instruction set, so we investigated alternatives to OpenSSL, including BoringSSL and the Intel Intelligent Storage Acceleration Library (ISA-L).</span></div><h3 dir="ltr"><span>Additional Optimizations</span></h3><b><br></b><br><div dir="ltr"><span>Netflix and NGINX had previously worked together to improve our HTTP client request and response time via the use of </span><span>sendfile</span><span> calls to perform a zero-copy data flow from storage (HDD or SSD) to network socket, keeping the data in the kernel memory address space and relieving some of the CPU burden. The Netflix team specifically added the ability to make the </span><span>sendfile</span><span> calls asynchronous - further reducing the data path and enabling more simultaneous connections. </span></div><b><br></b><br><div dir="ltr"><span><img height="246" src="https://lh5.googleusercontent.com/nacXIxfF0FlEgkCAmebsxO1SHwlhfSBhYMuV6AU5Awq4qdQAAz-3W5ummrmEFtlvCCWva4CrbjZRQKfT2ythV2FpC0xEfHlxCiHYxYLJSuHtHqJQSPKSjIY0Ww8DF1rG5RVAnw2m" width="508"></span></div><b><br></b><br><div dir="ltr"><span>However, TLS functionality, which requires the data to be passed to the application layer, was incompatible with the </span><span>sendfile</span><span> approach.</span></div><b><br></b><br><div dir="ltr"><span><img height="246" src="https://lh6.googleusercontent.com/I4Oxz8qz9iaX-N52LjeAQ1ZBpYRvPIEC8ulCZW6X3tploLNQoiIhUCOk2aXc3J6h-h9Ou_sFIK7sJfwvjcQuXhmrwyjygjBbFzZrRWaIb_MvJosoRYDqzbVSHc1s6tmrTtruCRDN" width="508"></span></div><b><br></b><br><div dir="ltr"><span>To retain the benefits of the </span><span>sendfile</span><span> model while adding TLS functionality, we designed a hybrid TLS scheme whereby session management stays in the application space, but the bulk encryption is inserted into the </span><span>sendfile</span><span> data pipeline in the kernel. This extends </span><span>sendfile</span><span> to </span><span>support encrypting data for TLS/SSL connections.</span></div><b><br></b><br><div dir="ltr"><span><img height="246" src="https://lh4.googleusercontent.com/lcFuRIeJMt22uLkwHp4yaP83fY_bwSaOGWYlxVxFaD_UZqtHpcK43LVUkJOn3TTn6aTYJoGbtrJYgV3vc-tFX5dHUjD9aFdgDRuZNS_P2j-GPCK32fmeO1oJ965Y_hLkfb9BNHm7" width="508"></span></div><b><br></b><br><div dir="ltr"><span>We also made some important fixes to our earlier data path implementation, including eliminating the need to repeatedly traverse </span><span>mbuf</span><span> linked lists to gain addresses for encryption.</span></div><h3 dir="ltr"><span>Testing and Results</span></h3><b><br></b><br><div dir="ltr"><span>We tested the BoringSSL and ISA-L AES-GCM implementations with our </span><span>sendfile</span><span> improvements against a baseline of OpenSSL (with no </span><span>sendfile </span><span>changes), under typical Netflix traffic conditions on three different OCA hardware types. Our changes in both the BoringSSL and ISA-L test situations significantly increased both CPU utilization and bandwidth over baseline - </span><span>increasing performance by up to 30%</span><span>, depending on the OCA hardware version. We chose the ISA-L cipher implementation, which had slightly better results. With these improvements in place, we can continue the process of adding TLS to our video streams for clients that support it, without suffering prohibitive performance hits.</span></div><b><br></b><br><div dir="ltr"><span>Read more details </span><a href="https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf"><span>in this paper</span></a><span> and </span><a href="https://people.freebsd.org/~rrs/asiabsd_tls_improved.pdf"><span>the follow up paper</span></a><span>. We continue to investigate new and novel approaches to making both security and performance a reality. If this kind of ground-breaking work is up your alley, check out our </span><a href="https://jobs.netflix.com/jobs?location=all&organization=all&q=open%20connect%20network%20&team=all"><span>latest job openings</span></a><span>! </span></div><br><span>By Randall Stewart, Scott Long, Drew Gallatin, Alex Gutarin, and Ellen Livengood</span>63缓存更新的套路看到好些人在写更新缓存数据代码时，先删除缓存，然后再更新数据库，而后续的操作会把数据再装载的缓存中。然而，这个 […]64为什么我不在微信公众号上写文章很多朋友问我为什么不在微信公众号上写文章。我都没有直接回答，老实说，我也是扭扭捏捏的，才去开了个个人的微信的公 […]