文章
33[转]Monitoring and Tuning the Linux Networking Stack: Receiving Data最近在网上看到了一篇详细介绍Linux系统上接收网络数据包过程的博客（英文的），觉得很不错，特地分享一下。如果你想做Linux的网络性能优化，这是一篇很好的参考文章。 The networking stack is complex and there is no one size fits all solution. If the performance and health of your ne…34技术人员的发展之路2012年的时候写过一篇叫《程序算法与人生选择》的文章，我用算法来类比如何做选择，说白了就是怎么去计算，但是并没有讲程序员可以发展的方向有哪些。 所以，就算是有...<p><a href="http://coolshell.cn/articles/17583.html"> Read More<span> Read More</span></a></p>35Netflix at RecSys 2016 – Recap<div dir="ltr"><span>By </span><a href="https://twitter.com/JustinBasilico"><span>Justin Basilico</span></a><span> and </span><a href="https://twitter.com/moustaki"><span>Yves Raimond</span></a></div><div dir="ltr"><br></div><div dir="ltr"><span><span>A key aspect of Netflix is providing our members with a personalized experience so they can easily find great stories to enjoy. A collection of recommender systems drive the main aspects of this personalized experience and we continuously work on researching and testing new ways to make them better. </span>As such, we were delighted to sponsor and participate in this year’s </span><a href="https://recsys.acm.org/recsys16/"><span>ACM Conference on Recommender Systems</span></a><span> in Boston, which marked the 10th anniversary of the conference. For those who couldn’t attend or want more information, here is a recap of our talks and papers at the conference.</span></div><div dir="ltr"><br></div><div dir="ltr"><span>Justin and Yves gave a talk titled “</span><a href="http://www.slideshare.net/justinbasilico/recommending-for-the-world-66144921"><span>Recommending for the World</span></a><span>” on how we prepared our algorithms to work world-wide ahead of our global launch earlier this year. You can also read more about it in our previous </span><a href="http://techblog.netflix.com/2016/02/recommending-for-world.html"><span>blog</span></a><span> </span><a href="https://media.netflix.com/en/company-blog/a-global-approach-to-recommendations"><span>posts</span></a><span>.</span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><div><span> </span><br><div><span><strong> <a href="https://www.slideshare.net/justinbasilico/recommending-for-the-world-66144921" target="_blank" title="Recommending for the World">Recommending for the World</a> </strong></span></div></div></div><div dir="ltr"><span><br></span></div><div dir="ltr"><br></div><div dir="ltr"><span>Justin also teamed up with </span><a href="http://xavier.amatriain.net/"><span>Xavier Amatriain</span></a><span>, formerly at Netflix and now at Quora, in the special Past, Present, and Future track to offer an </span><a href="http://www.slideshare.net/justinbasilico/past-present-future-of-recommender-systems-an-industry-perspective"><span>industry perspective</span></a><span> on what the future of recommender systems in industry may be.</span><br><span><br></span><br><div><span> </span><br><div><span><strong> <a href="https://www.slideshare.net/justinbasilico/past-present-future-of-recommender-systems-an-industry-perspective" target="_blank" title="Past, Present & Future of Recommender Systems: An Industry Perspective">Past, Present & Future of Recommender Systems: An Industry Perspective</a> </strong></span></div></div></div><div dir="ltr"><br></div><div dir="ltr"><span>Chao-Yuan Wu </span><a href="http://www.andrew.cmu.edu/user/chaoyuaw/RecSys2016_slides.pdf"><span>presented</span></a><span> a </span><a href="http://dl.acm.org/citation.cfm?id=2959174"><span>paper</span></a><span> he authored last year while at Netflix, on how to use navigation information to adapt recommendations within a session as you learn more about user intent.</span><br><span><br></span><br><div><span><a href="http://1.bp.blogspot.com/-9awzKSsgbss/WBPogZkmROI/AAAAAAAABRs/vAVhBg48Y5INO4R_i3XS_PyRaRPdUjb8gCK4B/s1600/Screen%2BShot%2B2016-10-28%2Bat%2B5.07.56%2BPM.png" imageanchor="1"><img border="0" height="225" src="https://1.bp.blogspot.com/-9awzKSsgbss/WBPogZkmROI/AAAAAAAABRs/vAVhBg48Y5INO4R_i3XS_PyRaRPdUjb8gCK4B/s400/Screen%2BShot%2B2016-10-28%2Bat%2B5.07.56%2BPM.png" width="400"></a></span><br><strong><a href="http://www.andrew.cmu.edu/user/chaoyuaw/RecSys2016_slides.pdf" target="_blank" title="Past, Present & Future of Recommender Systems: An Industry Perspective">Using Navigation to Improve Recommendations in Real-Time</a></strong> </div></div><div dir="ltr"><br></div><div dir="ltr"><span>Yves also shared some </span><a href="http://www.slideshare.net/moustaki/some-pitfalls-of-distributed-learning"><span>pitfalls of distributed learning</span></a><span> at the Large Scale Recommender Systems workshop.</span><br><span><br></span><br><div><span> </span><br><div><span><strong> <a href="https://www.slideshare.net/moustaki/some-pitfalls-of-distributed-learning" target="_blank" title="(Some) pitfalls of distributed learning">(Some) pitfalls of distributed learning</a> </strong></span></div></div></div><div dir="ltr"><br></div><div dir="ltr"><span>Hossein Taghavi gave a </span><a href="http://www.slideshare.net/MohammadHosseinTagha/balancing-discovery-and-continuation-in-recommendations/1"><span>presentation</span></a><span> at the RecSysTV workshop on trying to balance discovery and continuation in recommendations, which is also the subject of a recent </span><a href="http://techblog.netflix.com/2016/10/to-be-continued.html"><span>blog post</span></a><span>.</span><br><span><br></span><br><div><span> </span><br><div><span><strong> <a href="https://www.slideshare.net/MohammadHosseinTagha/balancing-discovery-and-continuation-in-recommendations" target="_blank" title="Balancing Discovery and Continuation in Recommendations">Balancing Discovery and Continuation in Recommendations</a> </strong></span></div></div></div><div dir="ltr"><br></div><div dir="ltr"><span>Dawen Liang </span><a href="http://www.slideshare.net/cheerz/factorization-meets-the-item-embedding-regularizing-matrix-factorization-with-item-cooccurrence"><span>presented</span></a><span> some </span><a href="http://dl.acm.org/citation.cfm?id=2959182"><span>research</span></a><span> he conducted prior to joining Netflix on combining matrix factorization and item embedding.</span><br><span><br></span><br><div><span> </span><br><div><span><strong> <a href="https://www.slideshare.net/cheerz/factorization-meets-the-item-embedding-regularizing-matrix-factorization-with-item-cooccurrence" target="_blank" title="Factorization Meets the Item Embedding: Regularizing Matrix Factorization with Item Co-occurrence">Factorization Meets the Item Embedding:</a></strong></span><br><span><strong><a href="https://www.slideshare.net/cheerz/factorization-meets-the-item-embedding-regularizing-matrix-factorization-with-item-cooccurrence" target="_blank" title="Factorization Meets the Item Embedding: Regularizing Matrix Factorization with Item Co-occurrence">Regularizing Matrix Factorization with Item Co-occurrence</a> </strong></span></div></div></div><div dir="ltr"><br></div><span><span>If you are interested in pushing the frontier forward in the recommender systems space, take a look at some of </span><span><a href="https://jobs.netflix.com/jobs?location=all&q=machine%20learning%20%22product%20engineering%20OR%20science%20and%20algorithms%22&team=all">our relevant open positions</a>!</span></span>36如何读懂并写出装逼的函数式代码今天在微博上看到了 有人分享了下面的这段函数式代码，我把代码贴到下面，不过我对原来的代码略有改动，对于函数式的版本，咋一看，的确令人非常费解，仔细看一下，你可能...<p><a href="http://coolshell.cn/articles/17524.html"> Read More<span> Read More</span></a></p>37ZFS on ubuntuUbuntu 从16.04 （AMD64架构上）开始正式支持ZFS作为数据文件系统（非root）。但是需要另外安装。过程非常简单： 验证是否ZFS是否已经安装并加载： 作为一个非常NB的文件系统，ZFS主要的特性有： 快照 写时复制（copy-on-write cloning） 数据完整性 数据压缩 自动修复 安装过程很容易，但是用起来就不会这么简单了。下面几个文档可供参考。 如果想要详细了解ZF…38IT培训，实验楼及其他可达书院建立的初衷就是为了做（在线）IT培训。自它建立以来，我也一直在思考在线培训究竟要怎么做才能有效率。就像那种面对面，一对一的培训一样的效率。 刚开始的想法（2008年左右）就是录制培训视频，通过让感兴趣者通过在线观看视频来学习。这种想法的出发点是，在IT技术学习中，尤其是偏重实战的技能，视频讲解比文字叙述更有效，更容易理解。 于是经过一年多的时间，录制了一套OpenBSD绝对新手入门的视频教…39Netflix Chaos Monkey Upgraded<div><a href="https://4.bp.blogspot.com/-hGHx6bM09ko/WAUQDVeb8WI/AAAAAAAABLU/48ffxXuVmLA_Qz1yQg7QExJ-baeevLDCACLcB/s1600/ChaosMonkey410.png" imageanchor="1"><img border="0" height="320" src="https://4.bp.blogspot.com/-hGHx6bM09ko/WAUQDVeb8WI/AAAAAAAABLU/48ffxXuVmLA_Qz1yQg7QExJ-baeevLDCACLcB/s320/ChaosMonkey410.png" width="317"></a></div><div><br></div>We are pleased to announce a significant upgrade to one of our more popular OSS projects. <a href="https://github.com/netflix/chaosmonkey" target="_blank">Chaos Monkey 2.0 is now on github!</a><br><br><a href="http://techblog.netflix.com/2011/07/netflix-simian-army.html" target="_blank">Years ago</a>, we decided to improve the resiliency of our microservice architecture. At our scale it is guaranteed that servers on our cloud platform will sometimes suddenly fail or disappear without warning. If we don’t have proper redundancy and automation, these disappearing servers could cause service problems.<br><br>The Freedom and Responsibility culture at Netflix doesn’t have a mechanism to force engineers to architect their code in any specific way. Instead, we found that we could build strong alignment around resiliency by taking the pain of disappearing servers and bringing that pain forward. We created Chaos Monkey to randomly choose servers in our production environment and turn them off during business hours. Some people thought this was crazy, but we couldn’t depend on the infrequent occurrence to impact behavior. Knowing that this would happen on a frequent basis created strong alignment among our engineers to build in the redundancy and automation to survive this type of incident without any impact to the millions of Netflix members around the world.<br><br>We value Chaos Monkey as a highly effective tool for improving the quality of our service. Now Chaos Monkey has evolved. We rewrote the service for improved maintainability and added some great new features. The evolution of Chaos Monkey is part of our commitment to keep our open source software up to date with our current environment and needs.<br><div><br></div><b>Integration with Spinnaker</b><br><br>Chaos Monkey 2.0 is fully integrated with <a href="http://techblog.netflix.com/2015/11/global-continuous-delivery-with.html" target="_blank">Spinnaker</a>, our continuous delivery platform.<br>Service owners set their Chaos Monkey configs through the Spinnaker apps, Chaos Monkey gets information about how services are deployed from Spinnaker, and Chaos Monkey terminates instances through Spinnaker.<br><br>Since Spinnaker works with multiple cloud backends, Chaos Monkey does as well. In the Netflix environment, Chaos Monkey terminates virtual machine instances running on AWS and Docker containers running on <a href="http://techblog.netflix.com/2016/07/distributed-resource-scheduling-with.html" target="_blank">Titus</a>, our container cloud.<br><br>Integration with Spinnaker gave us the opportunity to improve the UX as well. We interviewed our internal customers and came up with a more intuitive method of scheduling terminations. Service owners can now express a schedule in terms of the mean time between terminations, rather than a probability over an arbitrary period of time. We also added grouping by app, stack, or cluster, so that applications that have different redundancy architectures can schedule Chaos Monkey appropriate to their configuration. Chaos Monkey now also supports specifying exceptions so users can opt out specific clusters. Some engineers at Netflix use this feature to opt out small clusters that are used for testing.<br><br><table align="center" cellpadding="0" cellspacing="0"><tbody><tr><td><a href="https://3.bp.blogspot.com/-8wSlsvH25gM/WAVq6IJ5aqI/AAAAAAAABLk/q3PzkRci6EQGBzwF9Tr2jNhfDF6uVOW4ACLcB/s1600/SpinnakerChaosUI.png" imageanchor="1"><img border="0" src="https://3.bp.blogspot.com/-8wSlsvH25gM/WAVq6IJ5aqI/AAAAAAAABLk/q3PzkRci6EQGBzwF9Tr2jNhfDF6uVOW4ACLcB/s1600/SpinnakerChaosUI.png"></a></td></tr><tr><td><span>Chaos Monkey Spinnaker UI</span></td></tr></tbody></table><b>Tracking Terminations</b><br><br>Chaos Monkey can now be configured for specifying trackers. These external services will receive a notification when Chaos Monkey terminates an instance. Internally, we use this feature to report metrics into <a href="http://techblog.netflix.com/2014/12/introducing-atlas-netflixs-primary.html" target="_blank">Atlas</a>, our telemetry platform, and <a href="http://techblog.netflix.com/2013/03/python-at-netflix.html" target="_blank">Chronos</a>, our event tracking system. The graph below, taken from Atlas UI, shows the number of Chaos Monkey terminations for a segment of our service. We can see chaos in action. Chaos Monkey even periodically terminates itself.<br><br><table align="center" cellpadding="0" cellspacing="0"><tbody><tr><td><a href="https://3.bp.blogspot.com/-6eHVTW-EAfs/WAUAdbqGBSI/AAAAAAAABLA/VRqQlSqZKhc4x1xUBfrSB37rKDfBL2sGwCEw/s1600/AtlasChaos.png" imageanchor="1"><img border="0" src="https://3.bp.blogspot.com/-6eHVTW-EAfs/WAUAdbqGBSI/AAAAAAAABLA/VRqQlSqZKhc4x1xUBfrSB37rKDfBL2sGwCEw/s1600/AtlasChaos.png"></a></td></tr><tr><td>Chaos Monkey termination metrics in Atlas</td></tr></tbody></table><b>Termination Only</b><br><br>Netflix only uses Chaos Monkey to terminate instances. Previous versions of Chaos Monkey allowed the service to ssh into a box and perform other actions like burning up CPU, taking disks offline, etc. If you currently use one of the prior versions of Chaos Monkey to run an experiment that involves anything other than turning off an instance, you may not want to upgrade since you would lose that functionality.<br><b><br></b><b>Finale</b><br><br>We also used this opportunity to introduce many small features such as automatic opt-out for canaries, cross-account terminations, and automatic disabling during an outage. Find <a href="https://github.com/netflix/chaosmonkey" target="_blank">the code</a> on the <a href="https://github.com/Netflix" target="_blank">Netflix github account</a> and embrace the chaos!<br><br>-Chaos Engineering Team at Netflix<br><a href="https://www.linkedin.com/in/lhochstein" target="_blank">Lorin Hochstein</a>, <a href="https://www.linkedin.com/in/caseyrosenthal" target="_blank">Casey Rosenthal</a><br><br>40vmm enabled on OpenBSDAs per undeadly.org: With a small commit, OpenBSD now has a hypervisor and virtualization in-tree. This has been a lot of hard work by Mike Larkin, Reyk Flöter, and many others. VMM requires certain h…