SunisDownhttp://sunisdown.me/2019-03-06T00:00:00+08:00The Good Part of Rust2019-03-06T00:00:00+08:002019-03-06T00:00:00+08:00SunisDowntag:sunisdown.me,2019-03-06:/the-good-part-of-rust.html<p>Rust 语言中最有趣的两个特性分别是 ownership 和 borrowing,通过这两个特性,Rust 实现了他所谓的最安全的变成语言。</p>
<div class="section" id="ownership">
<h2>Ownership</h2>
<ol class="arabic simple">
<li>Each value in Rust has a variable that’s called its owner.</li>
<li>There can only be one owner at a time.</li>
<li>When the owner goes out of scope, the value will be dropped.</li>
</ol>
<p>Rust 的 ownership 往前可以追溯到 Wadler 开发的一个函数式编程语言,这个语言也跟 Rust …</p></div><p>Rust 语言中最有趣的两个特性分别是 ownership 和 borrowing,通过这两个特性,Rust 实现了他所谓的最安全的变成语言。</p>
<div class="section" id="ownership">
<h2>Ownership</h2>
<ol class="arabic simple">
<li>Each value in Rust has a variable that’s called its owner.</li>
<li>There can only be one owner at a time.</li>
<li>When the owner goes out of scope, the value will be dropped.</li>
</ol>
<p>Rust 的 ownership 往前可以追溯到 Wadler 开发的一个函数式编程语言,这个语言也跟 Rust 一样,没有 gc,但是也不需要手动管理内存。Wadler 主要是通过程序运行时,不断的复制数据做到这一点。在论文中可以这么实现,但是现实世界中做工程的时候,这种复制带来的性能损耗是无法接受的。Rust 的 ownership 简单来理解有点类似,但是 Rust 不是频繁的拷贝数据,而是复用内存里面的对象。
在 Rust 中,每个值都有他自己的 owner,传递或者返回某一个值,就意味着从原本的 owner 把值给了新的 owner。
我们通过下面的例子来看一下</p>
<div class="highlight"><pre><span></span><span class="k">fn</span> <span class="nf">make_str</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">s</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">String</span>::<span class="n">from</span><span class="p">(</span><span class="s">"hello"</span><span class="p">);</span><span class="w"> </span><span class="c1">// 字符串被创建,owner 是 s,s属于 make_str 这个作用域。</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push_str</span><span class="p">(</span><span class="s">", world!"</span><span class="p">);</span><span class="w"> </span><span class="c1">// s 是 owner</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push_str</span><span class="p">(</span><span class="s">"你好,世界!"</span><span class="p">);</span><span class="w"> </span><span class="c1">// s 是 owner</span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span><span class="w"> </span><span class="n">s</span><span class="p">);</span><span class="w"> </span><span class="c1">// s 是 owner</span>
<span class="w"> </span><span class="c1">// 作用域到此结束, s 被销毁。</span>
<span class="p">}</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">make_str</span><span class="p">();</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>这个例子中,我们创建另一个字符串,然后向他push了一些元素进去。 make_str 这个作用域中,s 一直是有效的,在这个作用域中,可以对s的值做任何操作。但是在 make_str 结束的时候,s 被自动回收。</p>
<p>当我们要把一个值作为返回值,或者传递给其他变量的时候,则是另外一种状态。这里为了方便调试,不在使用 String 类型,而是用 Vec<i32> 类型。</p>
<div class="highlight"><pre><span></span><span class="k">fn</span> <span class="nf">make_vec</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Vec</span><span class="o"><</span><span class="kt">i32</span><span class="o">></span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">s</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Vec</span>::<span class="n">new</span><span class="p">();</span><span class="w"> </span><span class="c1">// Vec 创建,owner 是 s,s属于 make_vec 这个作用域。</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push</span><span class="p">(</span><span class="mh">0x100</span><span class="p">);</span><span class="w"> </span><span class="c1">//</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push</span><span class="p">(</span><span class="mh">0x200</span><span class="p">);</span><span class="w"> </span><span class="c1">//</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push</span><span class="p">(</span><span class="mh">0x300</span><span class="p">);</span><span class="w"> </span><span class="c1">// vec 扩容,在堆上申请新的地址, s.buf 指向新的地址。</span>
<span class="w"> </span><span class="n">s</span><span class="w"> </span><span class="c1">// 把 s 的值,传递给调用这个函数的作用域中。</span>
<span class="p">}</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">print_vec</span><span class="p">(</span><span class="n">s_parameter</span>: <span class="nb">Vec</span><span class="o"><</span><span class="kt">i32</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="c1">// 参数 s_parameter 是属于 print_str 的一部分,也存活于这个 print_str 作用域内。</span>
<span class="w"> </span><span class="c1">// s_parameter.buf 的地址与 make_vec 里面 s.buf 的地址相同,s_parameter 与 s 属于不同的栈空间。</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">s_parameter</span><span class="p">.</span><span class="n">iter</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="w"> </span><span class="c1">// 在 print_str 结束的时候,s_parameter 被回收。</span>
<span class="p">}</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">s_new</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">make_vec</span><span class="p">();</span><span class="w"> </span><span class="c1">// s_new 从 make_str 中把值拿过来,s_new 作为新的 owner 存在与 main 函数中。</span>
<span class="w"> </span><span class="n">print_vec</span><span class="p">(</span><span class="n">s_new</span><span class="p">);</span><span class="w"> </span><span class="c1">// s_new 把值传递给 print_str</span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>在 make_vec 结束之前, s 被作为返回值被返回,这样在函数结束的时候,这个值并不会被销毁掉。main 函数作为调用者接管了那个返回值。
另一方面,print_vec 函数把 s_new 作为参数拿了过来,这个 Vec<i32> 对象又被从 main 函数传递给了 print_vec 函数,而 print_vec 再也没有传递这个值给别的地方,所以在 print_vec 结束的时候,顺手把这个值回收掉。</p>
<p>一旦把某一个值的 ownership 交给别人之后,原先的 owner 就再也无法被使用。举个例子:</p>
<div class="highlight"><pre><span></span><span class="k">fn</span> <span class="nf">make_vec</span><span class="p">()</span><span class="w"> </span>-> <span class="nb">Vec</span><span class="o"><</span><span class="kt">i32</span><span class="o">></span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">s</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Vec</span>::<span class="n">new</span><span class="p">();</span><span class="w"> </span><span class="c1">// Vec 创建,owner 是 s,s属于 make_vec 这个作用域。</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push</span><span class="p">(</span><span class="mh">0x100</span><span class="p">);</span><span class="w"> </span><span class="c1">//</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push</span><span class="p">(</span><span class="mh">0x200</span><span class="p">);</span><span class="w"> </span><span class="c1">//</span>
<span class="w"> </span><span class="n">s</span><span class="p">.</span><span class="n">push</span><span class="p">(</span><span class="mh">0x300</span><span class="p">);</span><span class="w"> </span><span class="c1">// vec 扩容,在堆上申请新的地址, s.buf 指向新的地址。</span>
<span class="w"> </span><span class="n">s</span><span class="w"> </span><span class="c1">// 把 s 的值,传递给调用这个函数的作用域中。</span>
<span class="p">}</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">print_vec</span><span class="p">(</span><span class="n">s_parameter</span>: <span class="nb">Vec</span><span class="o"><</span><span class="kt">i32</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="c1">// 参数 s_parameter 是属于 print_str 的一部分,也存活于这个 print_str 作用域内。</span>
<span class="w"> </span><span class="c1">// s_parameter.buf 的地址与 make_vec 里面 s.buf 的地址相同,s_parameter 与 s 属于不同的栈空间。</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">s_parameter</span><span class="p">.</span><span class="n">iter</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="w"> </span><span class="c1">// 在 print_str 结束的时候,s_parameter 被回收。</span>
<span class="p">}</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">s_new</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">make_vec</span><span class="p">();</span><span class="w"> </span><span class="c1">// s_new 从 make_str 中把值拿过来,s_new 作为新的 owner 存在与 main 函数中。</span>
<span class="w"> </span><span class="n">print_vec</span><span class="p">(</span><span class="n">s_new</span><span class="p">);</span><span class="w"> </span><span class="c1">// s_new 把值传递给 print_str</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">s_second</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">s_new</span><span class="p">;</span><span class="w"> </span><span class="c1">// 在编译的时候会报错, value used here after move,</span>
<span class="w"> </span><span class="n">print_vec</span><span class="p">(</span><span class="n">s_second</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>上面这段代码在编译的时候会直接报错,</p>
<div class="highlight"><pre><span></span>error<span class="o">[</span>E0382<span class="o">]</span>: use of moved value: <span class="sb">`</span>s_new<span class="sb">`</span>
</pre></div>
<p>编译器说 s_new 已经被移走了,这个值的 owner 已经不在是 s_new。在这个例子中, 我们创建的 vector 已经被回收了。</p>
</div>
<div class="section" id="borrowing">
<h2>Borrowing</h2>
<p>前面的 ownership 里面,每个值都在同一时间都只有一个 owner。Rust 还允许开发者在同时对某一个值持有多个引用。举个例子:</p>
<div class="highlight"><pre><span></span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="cp">#[derive(Debug)]</span><span class="w"></span>
<span class="w"> </span><span class="k">struct</span> <span class="nc">Point</span><span class="p">{</span><span class="n">x</span>:<span class="kt">i32</span><span class="p">,</span><span class="w"> </span><span class="n">y</span>:<span class="kt">i32</span><span class="p">};</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">pt</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Point</span><span class="p">{</span><span class="n">x</span>:<span class="mi">6</span><span class="p">,</span><span class="w"> </span><span class="n">y</span>:<span class="mi">9</span><span class="p">};</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="n">pt</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="n">pt</span><span class="p">;</span><span class="w"> </span><span class="c1">// 这里不会报错。</span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello, world!{:?}"</span><span class="p">,</span><span class="w"> </span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello, world!{:?}"</span><span class="p">,</span><span class="w"> </span><span class="n">y</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>上面的例子中,我们不再是传递值给 x 或者 y,而是创建一个 pt 的引用,然后分别借给 x,y。也就是我们常说的指针。就像我们在注释中说的那样,在第二次传递 pt 的引用的时候,编译器不会报错。因为这里的指针是允许被共享的。但是与正常的指针不太一样的时候,我们没有办法通过这些指针来修改这个对象。一旦我们尝试修改,编译的时候就会报错。</p>
<p>如果想要通过某一个指针来修改对象,可以用可修改的指针引用。这种引用有点像之前的 ownership,同一时间只有一个引用。例子如下:</p>
<div class="highlight"><pre><span></span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="cp">#[derive(Debug)]</span><span class="w"></span>
<span class="w"> </span><span class="k">struct</span> <span class="nc">Point</span><span class="p">(</span><span class="kt">i32</span><span class="p">,</span><span class="w"> </span><span class="kt">i32</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">pt</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Point</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span><span class="w"> </span><span class="mi">9</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">pt</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="n">pt</span><span class="p">;</span><span class="w"> </span><span class="c1">// cannot borrow `pt` as immutable because it is also borrowed as mutable</span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">z</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">pt</span><span class="p">;</span><span class="w"> </span><span class="c1">// cannot borrow `pt` as mutable more than once at a time</span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello, world!{:?}"</span><span class="p">,</span><span class="w"> </span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"Hello, world!{:?}"</span><span class="p">,</span><span class="w"> </span><span class="n">y</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>上面的例子中,我们创建了一个独一无二的引用,而不是共享指针。在我们的代码里面,后面的 y,z 又像重复引用这个对象,这种时候会在编译时报错。这种报错跟我们在最开始的看 ownership 的时候有点类似。</p>
<p>简单来说,borrowing 有两种不同的形式,</p>
<ol class="arabic simple">
<li>Immutable references,这种形式可以被共享,大家都可以来读,但是不可以写。</li>
<li>Mutable references,这种形式可以被更新,但是不可以被共享。同一时间只有一个引用。</li>
</ol>
</div>
可扩展服务设计原则 checklist2019-02-11T00:00:00+08:002019-02-11T00:00:00+08:00SunisDowntag:sunisdown.me,2019-02-11:/ke-kuo-zhan-fu-wu-she-ji-yuan-ze-checklist.html<p>这篇十多年之前写的论文基本包含设计一个可扩展服务的要点,我们现在玩的
cloud-native 早在很多年前就已经被大佬们整理的很透彻了。 这个是基于 James
Hamilton 的 paper 来总结的
<a class="reference external" href="http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf">http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf</a></p>
<div class="section" id="id1">
<h2>基本原则</h2>
<ul class="simple">
<li>[ ] Expect failures
硬盘可能会坏,网络会不稳定,系统设计的时候是不是能够优雅的处理各种异常?</li>
<li>[ ] Keep things simple
复杂会导致更多的问题,简单的系统更容易正确的运行。去掉不必要的依赖等</li>
<li>[ ] Automate everything 使人都会犯错,把所有能够自动化的都自动化。</li>
</ul>
</div>
<div class="section" id="id2">
<h2>整体设计</h2>
<ul class="simple">
<li>[ ] 发生故障的时候,系统能否在没有人工干预的情况下自动恢复</li>
<li>[ ] 故障恢复的路径需要经常被测试</li>
<li>[ ] 把各个不同的组件都文档化,而不是每次了解某一个部分都需要看代码</li>
<li>[ ] 是否只提供一个版本给用户(单一版本迭代成本更低</li>
<li>[ ] 多租户,是否需要在没有物理隔离的情况下提供多租户功能</li>
<li>[ ] health check 是否实现了自动且快速的故障检测</li>
<li>[ ] 当你依赖的系统有问题的时候,能否服务降级</li>
<li>[ ]
相同那个的功能,只需要在一个应用里面实现 …</li></ul></div><p>这篇十多年之前写的论文基本包含设计一个可扩展服务的要点,我们现在玩的
cloud-native 早在很多年前就已经被大佬们整理的很透彻了。 这个是基于 James
Hamilton 的 paper 来总结的
<a class="reference external" href="http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf">http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf</a></p>
<div class="section" id="id1">
<h2>基本原则</h2>
<ul class="simple">
<li>[ ] Expect failures
硬盘可能会坏,网络会不稳定,系统设计的时候是不是能够优雅的处理各种异常?</li>
<li>[ ] Keep things simple
复杂会导致更多的问题,简单的系统更容易正确的运行。去掉不必要的依赖等</li>
<li>[ ] Automate everything 使人都会犯错,把所有能够自动化的都自动化。</li>
</ul>
</div>
<div class="section" id="id2">
<h2>整体设计</h2>
<ul class="simple">
<li>[ ] 发生故障的时候,系统能否在没有人工干预的情况下自动恢复</li>
<li>[ ] 故障恢复的路径需要经常被测试</li>
<li>[ ] 把各个不同的组件都文档化,而不是每次了解某一个部分都需要看代码</li>
<li>[ ] 是否只提供一个版本给用户(单一版本迭代成本更低</li>
<li>[ ] 多租户,是否需要在没有物理隔离的情况下提供多租户功能</li>
<li>[ ] health check 是否实现了自动且快速的故障检测</li>
<li>[ ] 当你依赖的系统有问题的时候,能否服务降级</li>
<li>[ ]
相同那个的功能,只需要在一个应用里面实现,没有必要实现多次,会增加维护成本。</li>
<li>[ ]
服务需要能够单独运行,而不是必须要依赖于某个其他的服务才可以正常运行。</li>
<li>[ ] 针对少数需要人工干预的情况,需要准备好文档,脚本,测试方案等。</li>
<li>[ ]
保持系统简单的架构,如果是为了优化性能而增加复杂度,则需要这个性能上的改进超过一个数量级,只有几个百分点的改进不值得增加系统复杂成都。</li>
<li>[ ] 所有层级的入口都应该有准入机制,比如 rate
limit,防止量过大导致服务不可用</li>
<li>[ ] 能否拆分服务,拆分的是否合理</li>
<li>[ ] 分布式环境下,我们是否了解里面的网络拓扑,这个是否找网络方面专家
review 过</li>
<li>[ ] 是否分析过吞吐量与延迟,是否有对应的扩容方案</li>
<li>[ ]
对于这个服务给数据库/数据服务带来的流量是否有一个明确的理解,是否验证过。</li>
<li>[ ] 是否所有的系统,都是在同一套工具链下完成的,比如相同的 code
review,测试环境等</li>
<li>[ ] 版本化,保留之前的版本,测试用例,万一需要回滚呢</li>
<li>[ ] 单点故障是不可接受的</li>
</ul>
</div>
<div class="section" id="id3">
<h2>自动化管理</h2>
<ul class="simple">
<li>[ ] 所有的服务都需要支持重启</li>
<li>[ ] 所有持久化的数据都需要备份</li>
<li>[ ]
设计上需要支持跨数据中心部署(如果设计是不做,后面要实现就会比较麻烦)</li>
<li>[ ] 部署/配置自动化</li>
<li>[ ] 配置与代码需要一起交付,不要 version A的代码用了 version B
的配置。</li>
<li>[ ] 线上的更改,需要有记录,what,when,whom,which servers
,定时扫描线上版本,以免出现不一致的情况。</li>
<li>[ ] 按照角色来来管理服务,而不是面向服务来管理。</li>
<li>[ ] 系统出现多种故障的时候,服务是否能够正常工作</li>
<li>[ ] 重要的数据不要依赖于本地存储,很容易丢数据。</li>
<li>[ ] 部署过程是否简单?</li>
<li>[ ] chaos monkey 是个好东西</li>
</ul>
</div>
<div class="section" id="id4">
<h2>依赖管理</h2>
<ul class="simple">
<li>[ ] 能否容忍 latency 比较高的情况</li>
<li>[ ] 服务调用是否有超时机制</li>
<li>[ ] 超时重试是否有限制次数</li>
<li>[ ] 是否有CB 机制</li>
<li>[ ] 是否有快速失败机制</li>
<li>[ ] 依赖的组件是否可靠,验证过?</li>
<li>[ ] 跨服务的监控告警有吗</li>
<li>[ ] 依赖双方要有一直的设计目标</li>
<li>[ ] 模块解耦,依赖的组件挂了,也要能够服务(服务降级)</li>
</ul>
</div>
<div class="section" id="id5">
<h2>发布周期与测试</h2>
<ul class="simple">
<li>[ ]
是否频繁发布,发布频繁可以减少出错的机会,发布周期太长是很危险的(3个月)</li>
<li>[ ] 是否对用户体验定义了标准,是否有测试这些标准</li>
<li>[ ] 能否回滚到某一个指定版本</li>
<li>[ ] 可以在单节点上部署测试嘛?</li>
<li>[ ] 有压测嘛</li>
<li>[ ] 新版本发布之前的测试(性能,吞吐量,latency)</li>
<li>[ ] 可以使用 production 来测试嘛</li>
<li>[ ] 是否有跟生成环境完全一直的环境,并用相同的数据进行大规模测试</li>
<li>[ ] 有监控系统吗</li>
<li>[ ] 监控系统能明显的看出系统的各种重要指标嘛</li>
<li>[ ] 能否减少误报</li>
</ul>
</div>
<div class="section" id="id6">
<h2>硬件选择与标准化</h2>
<p>这个老哥在论文里面教了怎么购买硬件,怎么搞机柜。Google
有一本更专业的书来说这个事儿,这里不总结了。</p>
</div>
<div class="section" id="id7">
<h2>运维与容量规划。</h2>
<ul class="simple">
<li>[ ] devops, 谁开发,谁治理。</li>
<li>[ ] 只做软删除,要能够恢复被误删的数据</li>
<li>[ ] 跟踪资源分配</li>
<li>[ ]
一次只做一项更改(排查问题是,一次只对应用做一次更改,方便溯源问题)</li>
<li>[ ]
配置一切,如果可以通过更新配置来完成,而不是更改代码,这样会方便很多</li>
</ul>
</div>
<div class="section" id="id8">
<h2>审计,监控与告警</h2>
<ul class="simple">
<li>[ ] 监控一切</li>
<li>[ ] 统计有问题但是没有告警的情况,把这个比例降低到0</li>
<li>[ ] 分析数据,理解那些是正常的行为,避免误报。</li>
<li>[ ] 数据是最有价值的资源,帮助我们追溯问题。</li>
<li>[ ] 日志 Level是否可以配置,而不是重启,可配置的日志 Level
可以在需要的时候,输出更详细的日志帮助排问题。</li>
<li>[ ]
所有发现的错误都要及时处理,如果有错误但是没有处理手段,那这个错误就可能会被长期忽略,最终导致灾难发生</li>
<li>[ ] 快速定位线上问题</li>
<li>[ ] 能否镜像一个线上的系统,在镜像系统调试问题</li>
</ul>
</div>
<div class="section" id="id9">
<h2>优雅的降级与准入机制</h2>
<ul class="simple">
<li>[ ] 是否有 红按钮 机制,支持拒绝不重要的请求</li>
<li>[ ] 准入控制,拒绝部分请求</li>
<li>[ ] 渐入式准入控制,慢慢放开流量,以便系统能够优化恢复</li>
</ul>
</div>
<div class="section" id="id10">
<h2>客户沟通计划</h2>
<ul class="simple">
<li>[ ]
针对大规模系统不可用,数据丢失或损坏,安全漏洞等,是否制定了沟通计划,想之前腾讯云那种情况,就是缺乏沟通导致的</li>
</ul>
</div>
<div class="section" id="id11">
<h2>客户自助</h2>
<ul class="simple">
<li>[ ] 客户自行配置可以降低成本,并提高满意度,支持客户自助也相对重要。</li>
</ul>
</div>
Linux IO2019-01-26T00:00:00+08:002019-01-26T00:00:00+08:00SunisDowntag:sunisdown.me,2019-01-26:/linux-io.html<p>这篇主要是从 scylla 的 <a class="reference external" href="https://www.scylladb.com/2017/10/05/io-access-methods-scylla/">Blog</a> 翻译过来的,同时也参考了数据库架构的这篇 <a class="reference external" href="http://db.cs.berkeley.edu/papers/fntdb07-architecture.pdf">Paper</a></p>
<div class="section" id="i-o">
<h2>数据库存储管理之 I/O</h2>
<p>绝大多数的服务器开发在讨论 I/O 的时候,基本都是在讨论网络
I/O,但是在开发数据库的时候,更多的是考虑系统
I/O。现在的数据库的存储管理大致可以分成两种,一种是直接与磁盘的底层模块(设备驱动)进行交互读写,还有一种是使用
OS
提供的文件系统。前一种常见于商业数据库,后面一种则是更常见更通用的方案。虽然理论上讲绕过文件系统,在
row-mode
下直接对磁盘进行读写,可以得到更好的性能,但是这样系统会更复杂,我们不做过多讨论,只关注第二种方式。</p>
<div class="section" id="linux-file-i-o">
<h3>Linux File I/O</h3>
<p>Linux 有 4 种常见的读写文件的方式,我们在后面会一一分析。</p>
<ul>
<li><p class="first">read/write 这个是操作系统提供的 …</p></li></ul></div></div><p>这篇主要是从 scylla 的 <a class="reference external" href="https://www.scylladb.com/2017/10/05/io-access-methods-scylla/">Blog</a> 翻译过来的,同时也参考了数据库架构的这篇 <a class="reference external" href="http://db.cs.berkeley.edu/papers/fntdb07-architecture.pdf">Paper</a></p>
<div class="section" id="i-o">
<h2>数据库存储管理之 I/O</h2>
<p>绝大多数的服务器开发在讨论 I/O 的时候,基本都是在讨论网络
I/O,但是在开发数据库的时候,更多的是考虑系统
I/O。现在的数据库的存储管理大致可以分成两种,一种是直接与磁盘的底层模块(设备驱动)进行交互读写,还有一种是使用
OS
提供的文件系统。前一种常见于商业数据库,后面一种则是更常见更通用的方案。虽然理论上讲绕过文件系统,在
row-mode
下直接对磁盘进行读写,可以得到更好的性能,但是这样系统会更复杂,我们不做过多讨论,只关注第二种方式。</p>
<div class="section" id="linux-file-i-o">
<h3>Linux File I/O</h3>
<p>Linux 有 4 种常见的读写文件的方式,我们在后面会一一分析。</p>
<ul>
<li><p class="first">read/write 这个是操作系统提供的 read 和 write 两个系统调用。
调用过程是当前进程调用 read 这个系统调用,向 kernel
申请读一个文件的多少长度的内容。这里需要注意系统的
cache,如果你要读的内容刚好在系统 page cache 里面,那 kernel
就会把这部分内容直接从 内核态 复制到 用户态,这个 read
的系统调用就完成了。反之,kernel 就需要到硬盘来读取这部分数据到自己的
page cache,这个过程会block 原来的那个
Thread,等到数据都被读取到内核态的时候,Thread
被重新唤醒,然后将数据从内核态拷贝到用户态。 write 操作与 read
有点类似,当调用 write 的时候,操作系统会把数据 copy 到 page cache
里面,然后内核会在之后将数据刷入磁盘。调用过程如图</p>
<p><img alt="image0" src="images/Untitled-ec4afb82-9185-4382-8a52-3f7161dd6627.png" /></p>
<p><img alt="image1" src="images/Untitled-7c72faa5-f079-489b-882e-13f0a428b7af.png" /></p>
</li>
</ul>
<p>注意点:</p>
<ul class="simple">
<li>可以说系统调用与磁盘的操作并不是同步的,我们在调用 write
成功之后,这个数据也有可能还在内存里面,没有被写入到磁盘上。</li>
<li>OS 的 page cache
对性能做了一些优化,但是对数据来说不够准确。buffering 实现了
read-ahead 预读技术跟 write-behind
技术。这个对数据来说不一定合适。比如
WAL需要及时落盘,比如我们查询的时候,数据的逻辑更知道应该预先读取哪些数据,而不是默认顺着文件预读。</li>
<li>还有就是 double
buffering问题以及数据拷贝问题。上面的调用过程中我们可以看到,用户态跟
kernel
都会在内存里面维护这份数据,浪费内存。同时来回拷贝数据也浪费计算资源(CPU),先从硬盘到
OS 的 page cache,然后再从 OS 的page cache 到用户空间的内存里面。</li>
<li>在这个过程中,系统调用,如果用户态频繁的调用系统调用,成本会很高。</li>
</ul>
</div>
<div class="section" id="mmap">
<h3>Mmap</h3>
<p>还有一种比较高级的方法是调用 mmap
这个系统调用,直接把一个文件映射到用户空间的内存里面。这样就可以同在相应的内存区域操作字节来访问文件内容。映射的分页会在需要的时候自动从文件中加载进来。在读数据的时候,如果这块数据已经被加载到内存中,那就会直接在用户空间的内存里面读取内容,绕过
kernel 层。如果没有加载到用户内存中,就会引发一个 page-fault,然后kernel
从硬盘给这个 page
读取数据。当数据准备完成之后,这个线程会被唤醒,然后继续从内存里面读取数据。与
read
系统调用的区别是,一旦数据加载到内存中,线程可以直接读取内存中的数据,而不需要再多一次系统调用,也避免了从
kernel 到用户空间的数据拷贝。如果数据没有被加载到内存里面,则于 read
区别不大。过程如下:</p>
<p><img alt="image2" src="images/Untitled-ca6c3240-9083-47c8-9cf2-c54107a464ea.png" /></p>
</div>
</div>
<div class="section" id="driect-i-o-read-write">
<h2>Driect I/O read/write</h2>
<p>操作系统允许应用程序在执行磁盘 I/O 的时候绕过缓冲区的 page
cache,从用户空间直接把数据传递到磁盘。也就是直接 I/O,也叫做 裸
I/O。对于普通的服务端 app 来说,用 direct I/O 可能不会比用 read/write
要好,因为操作系统本身为I/O 做了很多优化,如果用了 direct I/O
,那就没有办法受益于这些优化了。但是数据库不是这样。数据有特定的
I/O需求。而且无论是 read/write 还是 mmap,都把 I/O
的调度交给了内核,数据库为了追求性能,自己来做 I/O 调度是最好的。Direct
I/O 读写在使用上跟普通的 read/write
区别不大,只是打开文件的时候,传递一个 O_DIRECT 的 flag
进去。在读的过程中,现场对用 O_DIRECT 打开的文件调用 read
系统调用,然后这个线程直接休眠,磁盘寻址,读取数据,然后直接把数据传递给用户线程。中间不再需要
kernel 中转一下。</p>
<p><img alt="image3" src="images/Untitled-00751a29-6df2-4234-80e6-5dd9e18df05c.png" /></p>
</div>
<div class="section" id="asynchronous-direct-i-o">
<h2>Asynchronous direct I/O</h2>
<p>AIO 是 DIO 的改进版本,这两个很相似,但是区别是 DIO 在调用 write
的时候,会block;而 AIO 则是调用 io_submit 这个系统调用,并且不会
block。这个线程完全控制由用户控制。同时还可以用 io_getevents 来收集 I/O
操作的结果。这些数据都不会被拷贝到 kernel page cache
里面去,而是直接被拷贝到用户空间中</p>
<p><img alt="image4" src="images/Untitled-97696516-69cf-4ab2-8f67-b007472ea819.png" /></p>
</div>
Learning Rust with LLDB2019-01-14T00:00:00+08:002019-01-14T00:00:00+08:00SunisDowntag:sunisdown.me,2019-01-14:/learning-rust-with-lldb.html<p>为什么要学习 <cite>Rust</cite> 呢?因为 Go 自带的 GC 还不够强,同时 goroutine 的调度机制又导致 go 对 CPU 也没办法用的很充分。当需要写对 CPU 性能要求比较高的系统,就需要换一门新语言。同时因为不想写 Cpp,Rust 有媲美 Cpp 的性能,相对 Cpp 来讲更好维护的代码。学习一下没有毛病。</p>
<div class="section" id="hello-hex">
<h2>Hello Hex</h2>
<p>通常学习一门新语言都是用 <cite>Hello World</cite> 来开始,但是因为字符串类型相比与int或者是 hex 要复杂的多,我们这里用 <cite>Hex</cite> 来替代 <cite>Hello world</cite> 。在之前我已经装好了 rust 的各种环境,包含 cargo 之类的包管理,在这里我用 …</p></div><p>为什么要学习 <cite>Rust</cite> 呢?因为 Go 自带的 GC 还不够强,同时 goroutine 的调度机制又导致 go 对 CPU 也没办法用的很充分。当需要写对 CPU 性能要求比较高的系统,就需要换一门新语言。同时因为不想写 Cpp,Rust 有媲美 Cpp 的性能,相对 Cpp 来讲更好维护的代码。学习一下没有毛病。</p>
<div class="section" id="hello-hex">
<h2>Hello Hex</h2>
<p>通常学习一门新语言都是用 <cite>Hello World</cite> 来开始,但是因为字符串类型相比与int或者是 hex 要复杂的多,我们这里用 <cite>Hex</cite> 来替代 <cite>Hello world</cite> 。在之前我已经装好了 rust 的各种环境,包含 cargo 之类的包管理,在这里我用 cargo 创建一个新的项目。</p>
<div class="highlight"><pre><span></span>cargo new helloHex
<span class="nb">cd</span> helloHex
</pre></div>
<p>然后将如下内容编辑到 <cite>src/main.rs</cite> 里面,</p>
<div class="highlight"><pre><span></span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mh">0x100</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"{}"</span><span class="p">,</span><span class="w"> </span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>编辑完这个文件之后,我们再编译他,通常情况下为了方便调试,我们都会在编译的时候禁止编译器对程序进行优化,在 C/Go 中需要加一些参数来完成这项工作, <cite>cargo</cite> 这个工具省略了这个步骤,直接会生成一个 debug 版本,我们可以直接拿来调试。编译命令如下:</p>
<div class="highlight"><pre><span></span>cargo build
lldb target/debug/helloHex
</pre></div>
<p>现在我们已经进入到了 lldb 的调试界面中,然后我们可以设置断点,查看堆栈信息等。我们先做一些简单的操作来熟悉一下 lldb。 在指令后面我加上了注释,说明了每一条命令的作用。</p>
<div class="highlight"><pre><span></span>$ lldb target/debug/helloHex
<span class="o">(</span>lldb<span class="o">)</span> target create <span class="s2">"target/debug/helloHex"</span>
Current executable <span class="nb">set</span> to <span class="s1">'target/debug/helloHex'</span> <span class="o">(</span>x86_64<span class="o">)</span>.
<span class="o">(</span>lldb<span class="o">)</span> b helloHex::main // 这条指令是对 helloHex 好个程序的 main 函数打一个断点
Breakpoint <span class="m">1</span>: <span class="nv">where</span> <span class="o">=</span> helloHex<span class="sb">`</span>helloHex::main::hc8d39b80069dcddb + <span class="m">18</span> at main.rs:2, <span class="nv">address</span> <span class="o">=</span> 0x00000001000011d2
<span class="o">(</span>lldb<span class="o">)</span> r // run 开始执行 helloHex 程序
Process <span class="m">84414</span> launched: <span class="s1">'/Users/chaotang.sun/rust_project/rustlearing/lldb/helloHex/target/debug/helloHex'</span> <span class="o">(</span>x86_64<span class="o">)</span>
Process <span class="m">84414</span> stopped
* thread <span class="c1">#1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1</span>
frame <span class="c1">#0: 0x00000001000011d2 helloHex`helloHex::main::hc8d39b80069dcddb at main.rs:2</span>
<span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span>
-> <span class="m">2</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span>
<span class="m">3</span> println!<span class="o">(</span><span class="s2">"{}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">4</span> <span class="o">}</span>
Target <span class="m">0</span>: <span class="o">(</span>helloHex<span class="o">)</span> stopped.
<span class="o">(</span>lldb<span class="o">)</span> s // step 执行下一步机器指令
Process <span class="m">84414</span> stopped
* thread <span class="c1">#1, queue = 'com.apple.main-thread', stop reason = step in</span>
frame <span class="c1">#0: 0x00000001000011d9 helloHex`helloHex::main::hc8d39b80069dcddb at main.rs:3</span>
<span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span>
<span class="m">2</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span>
-> <span class="m">3</span> println!<span class="o">(</span><span class="s2">"{}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">4</span> <span class="o">}</span>
Target <span class="m">0</span>: <span class="o">(</span>helloHex<span class="o">)</span> stopped.
<span class="o">(</span>lldb<span class="o">)</span> frame v // 查看当前栈的变量
<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">256</span>
<span class="o">(</span>lldb<span class="o">)</span> p x // 打印变量 x 的值
<span class="o">(</span>int<span class="o">)</span> <span class="nv">$0</span> <span class="o">=</span> <span class="m">256</span>
<span class="o">(</span>lldb<span class="o">)</span>
<span class="o">(</span>int<span class="o">)</span> <span class="nv">$1</span> <span class="o">=</span> <span class="m">256</span>
</pre></div>
</div>
<div class="section" id="variables-const">
<h2>Variables & Const</h2>
<p>Rust 的变量相对 C 来说有一些特殊,有可变的变量跟不可变的变量。(这有点绕,不可变的变量还可以称之为变量嘛? 他们有什么区别,除了在语法上的之外,我也很好奇对于CPU来说,处理这两种不同的变量有什么区别。
在这里我创建两段不同的程序,一段是用 <cite>mutable</cite> 的变量,一段是用 <cite>immutable</cite> 的变量,然后反编译看一下这俩到底有什么区别。</p>
<div class="highlight"><pre><span></span><span class="w"> </span><span class="cp">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">new</span><span class="w"> </span><span class="n">var_immut</span><span class="w"></span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mh">0x100</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"The value of x is: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mh">0x100</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"The value of x is: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<div class="highlight"><pre><span></span><span class="w"> </span><span class="cp">$</span><span class="w"> </span><span class="n">cargo</span><span class="w"> </span><span class="n">new</span><span class="w"> </span><span class="n">var_mut</span><span class="w"></span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mh">0x100</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"The value of x is: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mh">0x100</span><span class="p">;</span><span class="w"></span>
<span class="w"> </span><span class="n">println</span><span class="o">!</span><span class="p">(</span><span class="s">"The value of x is: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>然后我们进入到 lldb 之中反编译这两段程序。</p>
<div class="highlight"><pre><span></span>lldb target/debug/var_mut
lldb target/debug/var_immut
</pre></div>
<p>这两端程序反编译之后的结果机器相似,这里就只放其中一个截取的片段来展示:</p>
<div class="highlight"><pre><span></span><span class="o">(</span>lldb<span class="o">)</span> b var_mut::main
Breakpoint <span class="m">1</span>: <span class="nv">where</span> <span class="o">=</span> var_mut<span class="sb">`</span>var_mut::main::hbe57c58c6c7f6d8b + <span class="m">18</span> at main.rs:2, <span class="nv">address</span> <span class="o">=</span> 0x00000001000012a2
<span class="o">(</span>lldb<span class="o">)</span> r
Process <span class="m">88038</span> launched: <span class="s1">'/Users/chaotang.sun/rust_project/rustlearing/lldb/var_mut/target/debug/var_mut'</span> <span class="o">(</span>x86_64<span class="o">)</span>
Process <span class="m">88038</span> stopped
* thread <span class="c1">#1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1</span>
frame <span class="c1">#0: 0x00000001000012a2 var_mut`var_mut::main::hbe57c58c6c7f6d8b at main.rs:2</span>
<span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span>
-> <span class="m">2</span> <span class="nb">let</span> mut <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span>
<span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">4</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span>
<span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">6</span> <span class="o">}</span>
Target <span class="m">0</span>: <span class="o">(</span>var_mut<span class="o">)</span> stopped.
<span class="o">(</span>lldb<span class="o">)</span> disassemble
var_mut<span class="sb">`</span>var_mut::main::hbe57c58c6c7f6d8b:
0x100001290 <+0>: pushq %rbp
0x100001291 <+1>: movq %rsp, %rbp
0x100001294 <+4>: subq <span class="nv">$0</span>xf0, %rsp
0x10000129b <+11>: leaq 0x4150e<span class="o">(</span>%rip<span class="o">)</span>, %rsi <span class="p">;</span> core::fmt::num::_<span class="nv">$LT$impl$u20$core</span>..fmt..Display<span class="nv">$u20$for$u20$i32$GT</span>$::fmt::h04ca12b7570d3d05 at num.rs:201
-> 0x1000012a2 <+18>: movl <span class="nv">$0</span>x100, -0xa4<span class="o">(</span>%rbp<span class="o">)</span> <span class="p">;</span> <span class="nv">imm</span> <span class="o">=</span> 0x100
0x1000012ac <+28>: leaq -0xa4<span class="o">(</span>%rbp<span class="o">)</span>, %rax
.....
0x100001328 <+152>: movq <span class="nv">$0</span>x1, <span class="o">(</span>%rsp<span class="o">)</span>
0x100001330 <+160>: callq 0x100001100 <span class="p">;</span> core::fmt::Arguments::new_v1_formatted::h1a26f71ed7a6be2c at mod.rs:363
0x100001335 <+165>: leaq -0xa0<span class="o">(</span>%rbp<span class="o">)</span>, %rdi
0x10000133c <+172>: callq 0x10000a010 <span class="p">;</span> std::io::stdio::_print::h85f0ba007302c9c0 at stdio.rs:708
0x100001341 <+177>: movl -0xa4<span class="o">(</span>%rbp<span class="o">)</span>, %eax
0x100001347 <+183>: addl <span class="nv">$0</span>x100, %eax <span class="p">;</span> <span class="nv">imm</span> <span class="o">=</span> 0x100
......
0x100001400 <+368>: addq <span class="nv">$0</span>xf0, %rsp
0x100001407 <+375>: popq %rbp
0x100001408 <+376>: retq
0x100001409 <+377>: leaq 0x53f40<span class="o">(</span>%rip<span class="o">)</span>, %rdi
0x100001410 <+384>: callq 0x100047bf0 <span class="p">;</span> core::panicking::panic::h3941d6082b26bb8e at panicking.rs:44
0x100001415 <+389>: nopw %cs:<span class="o">(</span>%rax,%rax<span class="o">)</span>
0x10000141f <+399>: nop
</pre></div>
<p>这里先不关注这个函数栈内的数据转换,只关注两个程序里面不同的变量有什么异同,所以直接跳到变量初始化与赋值的地方,看一下有什么异同。 下面的 gdb 调试分为左右两列,左边为 mutable, 右边 为 immutable 的变量。在不同的阶段我们查看 x 变量的地址,可以看到 mutable 的变量 x 地址是不变的,后面的赋值都是基于栈上某一个固定地址进行值的修改。而 immutable 的 x 地址是会变化的,每一次初始化,都会在栈内重新分配一个新的地址。</p>
<div class="highlight"><pre><span></span><span class="o">(</span>lldb<span class="o">)</span> r │<span class="o">(</span>lldb<span class="o">)</span> r
Process <span class="m">89402</span> launched: <span class="s1">'/Users/chaotang.sun/rust_project/rustlearing/lldb/var_mut/target/debug/var_mut'</span> <span class="o">(</span>x86_64<span class="o">)</span> │Process <span class="m">89331</span> launched: <span class="s1">'/Users/chaotang.sun/rust_project/rustlearing/lldb/var_immut/target/debug/var_immut'</span> <span class="o">(</span>x86_64<span class="o">)</span>
Process <span class="m">89402</span> stopped │Process <span class="m">89331</span> stopped
* thread <span class="c1">#1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 │* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1</span>
frame <span class="c1">#0: 0x00000001000012a2 var_mut`var_mut::main::hbe57c58c6c7f6d8b at main.rs:2 │ frame #0: 0x0000000100001242 var_immut`var_immut::main::hab4d3d25ce94e8b3 at main.rs:2</span>
<span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span> │ <span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span>
-> <span class="m">2</span> <span class="nb">let</span> mut <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span> │-> <span class="m">2</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span>
<span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │ <span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">4</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span> │ <span class="m">4</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span>
<span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │ <span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">6</span> <span class="o">}</span> │ <span class="m">6</span> <span class="o">}</span>
Target <span class="m">0</span>: <span class="o">(</span>var_mut<span class="o">)</span> stopped. │Target <span class="m">0</span>: <span class="o">(</span>var_immut<span class="o">)</span> stopped.
<span class="o">(</span>lldb<span class="o">)</span> frame v │<span class="o">(</span>lldb<span class="o">)</span> frame v
<span class="o">(</span>lldb<span class="o">)</span> s │<span class="o">(</span>lldb<span class="o">)</span> s
Process <span class="m">89402</span> stopped │Process <span class="m">89331</span> stopped
* thread <span class="c1">#1, queue = 'com.apple.main-thread', stop reason = step in │* thread #1, queue = 'com.apple.main-thread', stop reason = step in</span>
frame <span class="c1">#0: 0x00000001000012ac var_mut`var_mut::main::hbe57c58c6c7f6d8b at main.rs:3 │ frame #0: 0x000000010000124c var_immut`var_immut::main::hab4d3d25ce94e8b3 at main.rs:3</span>
<span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span> │ <span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span>
<span class="m">2</span> <span class="nb">let</span> mut <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span> │ <span class="m">2</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span>
-> <span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │-> <span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">4</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span> │ <span class="m">4</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span>
<span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │ <span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">6</span> <span class="o">}</span> │ <span class="m">6</span> <span class="o">}</span>
Target <span class="m">0</span>: <span class="o">(</span>var_mut<span class="o">)</span> stopped. │Target <span class="m">0</span>: <span class="o">(</span>var_immut<span class="o">)</span> stopped.
<span class="o">(</span>lldb<span class="o">)</span> frame v │<span class="o">(</span>lldb<span class="o">)</span> frame v
<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">256</span> │<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">256</span>
<span class="o">(</span>lldb<span class="o">)</span> │<span class="o">(</span>lldb<span class="o">)</span>
<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">256</span> │<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">256</span>
<span class="o">(</span>lldb<span class="o">)</span> p <span class="p">&</span>x │<span class="o">(</span>lldb<span class="o">)</span> p <span class="p">&</span>x
<span class="o">(</span>int *<span class="o">)</span> <span class="nv">$0</span> <span class="o">=</span> 0x00007ffeefbff17c │<span class="o">(</span>int *<span class="o">)</span> <span class="nv">$0</span> <span class="o">=</span> 0x00007ffeefbff164
<span class="o">(</span>lldb<span class="o">)</span> x/xw 0x00007ffeefbff17c │<span class="o">(</span>lldb<span class="o">)</span> x/xw 0x00007ffeefbff164
0x7ffeefbff17c: 0x00000100 │0x7ffeefbff164: 0x00000100
<span class="o">(</span>lldb<span class="o">)</span> n │<span class="o">(</span>lldb<span class="o">)</span> n
The value of x is: <span class="m">256</span> │The value of x is: <span class="m">256</span>
Process <span class="m">89443</span> stopped │Process <span class="m">89452</span> stopped
* thread <span class="c1">#1, queue = 'com.apple.main-thread', stop reason = step over │* thread #1, queue = 'com.apple.main-thread', stop reason = step over</span>
frame <span class="c1">#0: 0x0000000100001341 var_mut`var_mut::main::hbe57c58c6c7f6d8b at main.rs:4 │ frame #0: 0x00000001000012e1 var_immut`var_immut::main::hab4d3d25ce94e8b3 at main.rs:4</span>
<span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span> │ <span class="m">1</span> fn main<span class="o">()</span> <span class="o">{</span>
<span class="m">2</span> <span class="nb">let</span> mut <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span> │ <span class="m">2</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span>
<span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │ <span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
-> <span class="m">4</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span> │-> <span class="m">4</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span>
<span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │ <span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">6</span> <span class="o">}</span> │ <span class="m">6</span> <span class="o">}</span>
Target <span class="m">0</span>: <span class="o">(</span>var_mut<span class="o">)</span> stopped. │Target <span class="m">0</span>: <span class="o">(</span>var_immut<span class="o">)</span> stopped.
<span class="o">(</span>lldb<span class="o">)</span> n │<span class="o">(</span>lldb<span class="o">)</span> n
Process <span class="m">89443</span> stopped │Process <span class="m">89452</span> stopped
* thread <span class="c1">#1, queue = 'com.apple.main-thread', stop reason = step over │* thread #1, queue = 'com.apple.main-thread', stop reason = step over</span>
frame <span class="c1">#0: 0x0000000100001371 var_mut`var_mut::main::hbe57c58c6c7f6d8b at main.rs:5 │ frame #0: 0x000000010000130e var_immut`var_immut::main::hab4d3d25ce94e8b3 at main.rs:5</span>
<span class="m">2</span> <span class="nb">let</span> mut <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span> │ <span class="m">2</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> 0x100<span class="p">;</span>
<span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │ <span class="m">3</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">4</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span> │ <span class="m">4</span> <span class="nb">let</span> <span class="nv">x</span> <span class="o">=</span> x + 0x100<span class="p">;</span>
-> <span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span> │-> <span class="m">5</span> println!<span class="o">(</span><span class="s2">"The value of x is: {}"</span>, x<span class="o">)</span><span class="p">;</span>
<span class="m">6</span> <span class="o">}</span> │ <span class="m">6</span> <span class="o">}</span>
Target <span class="m">0</span>: <span class="o">(</span>var_mut<span class="o">)</span> stopped. │Target <span class="m">0</span>: <span class="o">(</span>var_immut<span class="o">)</span> stopped.
<span class="o">(</span>lldb<span class="o">)</span> frame v │<span class="o">(</span>lldb<span class="o">)</span> frame v
<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">512</span> │<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">256</span>
<span class="o">(</span>lldb<span class="o">)</span> p <span class="p">&</span>x │<span class="o">(</span>int<span class="o">)</span> <span class="nv">x</span> <span class="o">=</span> <span class="m">512</span>
<span class="o">(</span>int *<span class="o">)</span> <span class="nv">$1</span> <span class="o">=</span> 0x00007ffeefbff17c │<span class="o">(</span>lldb<span class="o">)</span> p <span class="p">&</span>x
<span class="o">(</span>lldb<span class="o">)</span> │<span class="o">(</span>int *<span class="o">)</span> <span class="nv">$1</span> <span class="o">=</span> 0x00007ffeefbff1bc
<span class="o">(</span>lldb<span class="o">)</span> x/xw 0x00007ffeefbff17c │<span class="o">(</span>lldb<span class="o">)</span> x/xw 0x00007ffeefbff1bc
0x7ffeefbff17c: 0x00000200 │0x7ffeefbff1bc: 0x00000200
</pre></div>
</div>
Closures in Go2016-09-14T00:00:00+08:002016-09-14T00:00:00+08:00SunisDowntag:sunisdown.me,2016-09-14:/closures-in-go.html<p>在 <a class="reference external" href="https://qyuhen.bearychat.com">Qyuhen</a> 老大那边潜水的时候,基本上过一段时间就会看见有人讨论闭包。而随手 <tt class="docutils literal">Google</tt> 也并没有看到哪些 Blog 说的很清楚,甚至有一些 <tt class="docutils literal">Blog</tt> 里面的内容都是错的。(当然,也有可能是我理解错了)所以就想自己写一下。</p>
<div class="section" id="id1">
<h2>什么是闭包</h2>
<p>在谈起闭包(Closure)的时候,经常也会说起匿名函数。在 <a class="reference external" href="https://gobyexample.com/closures">gobyexample</a> 里面这两个概念也是放在一起的,很多人搞不清楚闭包与匿名函数有什么关系。这两者确实经常会一起出现,但是并不是一个概念。闭包是闭包,匿名函数是匿名函数。</p>
<div class="section" id="id2">
<h3>匿名函数</h3>
<p>匿名函数与正常的函数区别不大,只是这个函数没有名字。在 <tt class="docutils literal">Go</tt> 里面,一个真正的函数里面</p>
<div class="highlight"><pre><span></span><span class="kd">func</span> <span class="nx">Foo</span><span class="p">(</span><span class="nx">message</span> <span class="kt">string</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// 声明一个正常的函数</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">Foo</span><span class="p">(</span><span class="s">"hello world!"</span><span class="p">)</span>
<span class="p">}</span>
</pre></div>
<p>里面有函数名,有函数主体。但是匿名函数的类似与下面这样子 …</p></div></div><p>在 <a class="reference external" href="https://qyuhen.bearychat.com">Qyuhen</a> 老大那边潜水的时候,基本上过一段时间就会看见有人讨论闭包。而随手 <tt class="docutils literal">Google</tt> 也并没有看到哪些 Blog 说的很清楚,甚至有一些 <tt class="docutils literal">Blog</tt> 里面的内容都是错的。(当然,也有可能是我理解错了)所以就想自己写一下。</p>
<div class="section" id="id1">
<h2>什么是闭包</h2>
<p>在谈起闭包(Closure)的时候,经常也会说起匿名函数。在 <a class="reference external" href="https://gobyexample.com/closures">gobyexample</a> 里面这两个概念也是放在一起的,很多人搞不清楚闭包与匿名函数有什么关系。这两者确实经常会一起出现,但是并不是一个概念。闭包是闭包,匿名函数是匿名函数。</p>
<div class="section" id="id2">
<h3>匿名函数</h3>
<p>匿名函数与正常的函数区别不大,只是这个函数没有名字。在 <tt class="docutils literal">Go</tt> 里面,一个真正的函数里面</p>
<div class="highlight"><pre><span></span><span class="kd">func</span> <span class="nx">Foo</span><span class="p">(</span><span class="nx">message</span> <span class="kt">string</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// 声明一个正常的函数</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">Foo</span><span class="p">(</span><span class="s">"hello world!"</span><span class="p">)</span>
<span class="p">}</span>
</pre></div>
<p>里面有函数名,有函数主体。但是匿名函数的类似与下面这样子</p>
<div class="highlight"><pre><span></span><span class="kd">func</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">func</span><span class="p">(</span><span class="nx">message</span> <span class="kt">string</span><span class="p">)</span> <span class="p">{</span> <span class="c1">//声明匿名函数并执行</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span>
<span class="p">}(</span><span class="s">"hello world!"</span><span class="p">)</span>
<span class="p">}</span>
</pre></div>
<p>并不需要函数名。</p>
</div>
<div class="section" id="id3">
<h3>闭包</h3>
<div class="highlight"><pre><span></span><span class="err">闭包(英语:</span><span class="n">Closure</span><span class="err">),又称词法闭包(</span><span class="n">Lexical</span> <span class="n">Closure</span><span class="err">)或函数闭包(</span><span class="n">function</span> <span class="n">closures</span><span class="err">),是引用了自由变量的函数。这个被引用的自由变量将和这个函数一同存在,即使已经离开了创造它的环境也不例外。</span>
<span class="o">--</span> <span class="n">wikipedia</span>
</pre></div>
<p>闭包包含两个部分,一个是函数本身,还有是这个函数所引用的环境。 <tt class="docutils literal">Go</tt> 里闭包的函数必须是匿名函数。</p>
<div class="highlight"><pre><span></span><span class="kn">package</span> <span class="nx">main</span>
<span class="kd">func</span> <span class="nx">myFunc</span><span class="p">()</span> <span class="kd">func</span><span class="p">()</span> <span class="kt">int</span><span class="p">{</span>
<span class="nx">foo</span> <span class="o">:=</span> <span class="mi">0</span>
<span class="k">return</span> <span class="kd">func</span><span class="p">()</span> <span class="kt">int</span> <span class="p">{</span>
<span class="nx">foo</span><span class="o">++</span>
<span class="k">return</span> <span class="nx">foo</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">bar</span> <span class="o">:=</span> <span class="nx">myFunc</span><span class="p">()</span>
<span class="nx">value_1</span> <span class="o">:=</span> <span class="nx">bar</span><span class="p">()</span>
<span class="nx">value_2</span> <span class="o">:=</span> <span class="nx">bar</span><span class="p">()</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">value_1</span><span class="p">)</span> <span class="c1">// 1</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">value_2</span><span class="p">)</span> <span class="c1">// 2</span>
<span class="p">}</span>
</pre></div>
<p>在上面的例子里面,myFunc 里面的匿名函数可以访问并且更新 myFunc 里面的变量,这个变量的生命周期因为匿名函数的存在而延长。</p>
<p>通过闭包可以比较优雅的实现一些功能。比如斐波那契数列</p>
<div class="highlight"><pre><span></span><span class="kn">package</span> <span class="nx">main</span>
<span class="kd">func</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">gen</span> <span class="o">:=</span> <span class="nx">makeFibGen</span><span class="p">()</span>
<span class="k">for</span> <span class="nx">i</span> <span class="o">:=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span> <span class="p"><</span> <span class="mi">10</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span> <span class="p">{</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">gen</span><span class="p">())</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nx">makeFibGen</span><span class="p">()</span> <span class="kd">func</span><span class="p">()</span> <span class="kt">int</span> <span class="p">{</span>
<span class="nx">f1</span> <span class="o">:=</span> <span class="mi">0</span>
<span class="nx">f2</span> <span class="o">:=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="kd">func</span><span class="p">()</span> <span class="p">(</span><span class="nx">fib</span> <span class="kt">int</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">fib</span> <span class="p">=</span> <span class="nx">f1</span>
<span class="nx">f2</span><span class="p">,</span> <span class="nx">f1</span> <span class="p">=</span> <span class="p">(</span><span class="nx">f1</span> <span class="o">+</span> <span class="nx">f2</span><span class="p">),</span> <span class="nx">f2</span>
<span class="k">return</span> <span class="nx">fib</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
<div class="section" id="go">
<h2>Go 中匿名函数的实现</h2>
<p>前面有提到 <tt class="docutils literal">Go</tt> 里面匿名函数与普通函数区别不大,但是这不大的区别到底在哪里?在这我们用一个简短的小例子来看一下。</p>
<div class="highlight"><pre><span></span><span class="kn">package</span> <span class="nx">main</span>
<span class="kd">func</span> <span class="nx">myFunc</span><span class="p">(</span><span class="nx">message</span> <span class="kt">int</span><span class="p">)</span> <span class="p">{</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">f</span> <span class="o">:=</span> <span class="kd">func</span><span class="p">(</span><span class="nx">message</span> <span class="kt">int</span><span class="p">)</span> <span class="p">{</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span>
<span class="p">}</span>
<span class="nx">f</span><span class="p">(</span><span class="mh">0x100</span><span class="p">)</span>
<span class="nx">myFunc</span><span class="p">(</span><span class="mh">0x100</span><span class="p">)</span>
<span class="p">}</span>
</pre></div>
<p>首先我们将上面的代码编译</p>
<div class="highlight"><pre><span></span>go build -gcflags <span class="s2">"-N -l -m"</span> -o <span class="nb">test</span>
</pre></div>
<p>生成一个 elf 格式的文件 main。</p>
<p>然后我们通过 go 提供的反汇编工具,反编译我们刚刚生成的 test 文件。</p>
<div class="highlight"><pre><span></span><span class="nv">$go</span> tool objdump -s <span class="s2">"main\.main"</span> ./test
TEXT main.main<span class="o">(</span>SB<span class="o">)</span> /root/data/example/closures/anonymous_func.go
anonymous_func.go:7 0x401040 64488b0c25f8ffffff FS MOVQ FS:0xfffffff8, CX
anonymous_func.go:7 0x401049 483b6110 CMPQ 0x10<span class="o">(</span>CX<span class="o">)</span>, SP
anonymous_func.go:7 0x40104d <span class="m">7637</span> JBE 0x401086
anonymous_func.go:7 0x40104f 4883ec10 SUBQ <span class="nv">$0</span>x10, SP
anonymous_func.go:8 0x401053 488d1d16830800 LEAQ 0x88316<span class="o">(</span>IP<span class="o">)</span>, BX
anonymous_func.go:8 0x40105a 48895c2408 MOVQ BX, 0x8<span class="o">(</span>SP<span class="o">)</span>
anonymous_func.go:11 0x40105f 48c7042400010000 MOVQ <span class="nv">$0</span>x100, <span class="m">0</span><span class="o">(</span>SP<span class="o">)</span>
anonymous_func.go:11 0x401067 488b5c2408 MOVQ 0x8<span class="o">(</span>SP<span class="o">)</span>, BX
anonymous_func.go:11 0x40106c 4889da MOVQ BX, DX
anonymous_func.go:11 0x40106f 488b1a MOVQ <span class="m">0</span><span class="o">(</span>DX<span class="o">)</span>, BX
anonymous_func.go:11 0x401072 ffd3 CALL BX
anonymous_func.go:12 0x401074 48c7042400010000 MOVQ <span class="nv">$0</span>x100, <span class="m">0</span><span class="o">(</span>SP<span class="o">)</span>
anonymous_func.go:12 0x40107c e87fffffff CALL main.myFunc<span class="o">(</span>SB<span class="o">)</span>
anonymous_func.go:13 0x401081 4883c410 ADDQ <span class="nv">$0</span>x10, SP
anonymous_func.go:13 0x401085 c3 RET
anonymous_func.go:7 0x401086 e8b59f0400 CALL runtime.morestack_noctxt<span class="o">(</span>SB<span class="o">)</span>
anonymous_func.go:7 0x40108b ebb3 JMP main.main<span class="o">(</span>SB<span class="o">)</span>
anonymous_func.go:7 0x40108d cc INT <span class="nv">$0</span>x3
anonymous_func.go:7 0x40108e cc INT <span class="nv">$0</span>x3
anonymous_func.go:7 0x40108f cc INT <span class="nv">$0</span>x3
...
</pre></div>
<p>上面的汇编输出中我们可以看到一共有三次 <tt class="docutils literal">CALL</tt>, 排除调最后那个 <tt class="docutils literal">runtime</tt> 的 <tt class="docutils literal">CALL</tt> ,剩下两次分别对应了匿名函数调用以及正常的函数调用。而两次的区别在于正常的函数是 <tt class="docutils literal">CALL main.myFunc(SB)</tt> , 匿名函数的调用是 <tt class="docutils literal">CALL BX</tt> 。这两种不同的调用方式意味着什么?我们可以通过 gdb 来动态的跟踪这段代码来具体分析一下。</p>
<div class="highlight"><pre><span></span>gdb main
Reading symbols from test...done.
<span class="o">(</span>gdb<span class="o">)</span> b main.main
Breakpoint <span class="m">1</span> at 0x401040: file /root/data/example/closures/anonymous_func.go, line <span class="m">7</span>.
<span class="o">(</span>gdb<span class="o">)</span> r
Starting program: /root/data/example/closures/test
<span class="o">[</span>New LWP <span class="m">2067</span><span class="o">]</span>
<span class="o">[</span>New LWP <span class="m">2068</span><span class="o">]</span>
<span class="o">[</span>New LWP <span class="m">2069</span><span class="o">]</span>
Breakpoint <span class="m">1</span>, main.main <span class="o">()</span> at /root/data/example/closures/anonymous_func.go:7
<span class="m">7</span> func main<span class="o">()</span> <span class="o">{</span>
<span class="o">(</span>gdb<span class="o">)</span> l
<span class="m">2</span>
<span class="m">3</span> func myFunc<span class="o">(</span>message int<span class="o">)</span> <span class="o">{</span>
<span class="m">4</span> println<span class="o">(</span>message<span class="o">)</span>
<span class="m">5</span> <span class="o">}</span>
<span class="m">6</span>
<span class="m">7</span> func main<span class="o">()</span> <span class="o">{</span>
<span class="m">8</span> f :<span class="o">=</span> func<span class="o">(</span>message int<span class="o">)</span> <span class="o">{</span>
<span class="m">9</span> println<span class="o">(</span>message<span class="o">)</span>
<span class="m">10</span> <span class="o">}</span>
<span class="m">11</span> f<span class="o">(</span>0x100<span class="o">)</span>
<span class="o">(</span>gdb<span class="o">)</span> i locals
<span class="nv">f</span> <span class="o">=</span> <span class="o">{</span>void <span class="o">(</span>int<span class="o">)}</span> 0xc820039f40
<span class="o">(</span>gdb<span class="o">)</span> x/1xg 0xc820039f40
0xc820039f40: 0x000000c820000180
</pre></div>
<p>上面在 gdb 里面把断点设置在 <tt class="docutils literal">main.main</tt> 处,然后通过输出当前的环境变量可以看到变量 f。这时候显示 f 指针指向的内存内容。</p>
<div class="highlight"><pre><span></span><span class="o">(</span>gdb<span class="o">)</span> b <span class="m">11</span>
Breakpoint <span class="m">2</span> at 0x40105f: file /root/data/example/closures/anonymous_func.go, line <span class="m">11</span>.
<span class="o">(</span>gdb<span class="o">)</span> c
Continuing.
Breakpoint <span class="m">2</span>, main.main <span class="o">()</span> at /root/data/example/closures/anonymous_func.go:11
<span class="m">11</span> f<span class="o">(</span>0x100<span class="o">)</span>
<span class="o">(</span>gdb<span class="o">)</span> i locals
<span class="nv">f</span> <span class="o">=</span> <span class="o">{</span>void <span class="o">(</span>int<span class="o">)}</span> 0xc820039f40
<span class="o">(</span>gdb<span class="o">)</span> x/1xg 0xc820039f40
0xc820039f40: 0x0000000000489370
<span class="o">(</span>gdb<span class="o">)</span> i symbol 0x0000000000489370
main.main.func1.f in section .rodata of /root/data/example/closures/test
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0x0000000000489370
0x489370 <main.main.func1.f>: 0x0000000000401090 0x0000000000441fa0
<span class="o">(</span>gdb<span class="o">)</span> i symbol 0x0000000000401090
main.main.func1 in section .text of /root/data/example/closures/test
</pre></div>
<p>然后在调用匿名函数 <tt class="docutils literal">f</tt> 的地方再设置一个断点, <tt class="docutils literal">c</tt> 让程序执行到新的断点。再输出 f 指针指向的内存,发现里面的内容已经改变了,输出符号名可以看到符号是 <tt class="docutils literal">main.main.func1.f</tt>, 这个是编译器提我们生成的符号名,然后看一下这个地址指向的内容,会发现 <tt class="docutils literal">main.main.func1</tt> ,也就是就是我们的匿名函数。接着跟</p>
<div class="highlight"><pre><span></span><span class="o">(</span>gdb<span class="o">)</span> i r
rax 0xc820000180 <span class="m">859530330496</span>
rbx 0x489370 <span class="m">4756336</span>
...
<span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.main:
0x0000000000401040 <+0>: mov %fs:0xfffffffffffffff8,%rcx
0x0000000000401049 <+9>: cmp 0x10<span class="o">(</span>%rcx<span class="o">)</span>,%rsp
0x000000000040104d <+13>: jbe 0x401086 <main.main+70>
0x000000000040104f <+15>: sub <span class="nv">$0</span>x10,%rsp
0x0000000000401053 <+19>: lea 0x88316<span class="o">(</span>%rip<span class="o">)</span>,%rbx <span class="c1"># 0x489370 <main.main.func1.f></span>
0x000000000040105a <+26>: mov %rbx,0x8<span class="o">(</span>%rsp<span class="o">)</span>
<span class="o">=</span>> 0x000000000040105f <+31>: movq <span class="nv">$0</span>x100,<span class="o">(</span>%rsp<span class="o">)</span>
0x0000000000401067 <+39>: mov 0x8<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x000000000040106c <+44>: mov %rbx,%rdx
0x000000000040106f <+47>: mov <span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x0000000000401072 <+50>: callq *%rbx
0x0000000000401074 <+52>: movq <span class="nv">$0</span>x100,<span class="o">(</span>%rsp<span class="o">)</span>
0x000000000040107c <+60>: callq 0x401000 <main.myFunc>
0x0000000000401081 <+65>: add <span class="nv">$0</span>x10,%rsp
0x0000000000401085 <+69>: retq
0x0000000000401086 <+70>: callq 0x44b040 <runtime.morestack_noctxt>
0x000000000040108b <+75>: jmp 0x401040 <main.main>
0x000000000040108d <+77>: int3
0x000000000040108e <+78>: int3
0x000000000040108f <+79>: int3
End of assembler dump.
<span class="o">(</span>gdb<span class="o">)</span> p <span class="nv">$rsp</span>
<span class="nv">$2</span> <span class="o">=</span> <span class="o">(</span>void *<span class="o">)</span> 0xc820039f38
<span class="o">(</span>gdb<span class="o">)</span> x/1xg 0xc820039f38
0xc820039f38: 0x0000000000000000
<span class="o">(</span>gdb<span class="o">)</span> ni
0x0000000000401067 <span class="m">11</span> f<span class="o">(</span>0x100<span class="o">)</span>
<span class="o">(</span>gdb<span class="o">)</span> x/1xg 0xc820039f38
0xc820039f38: 0x0000000000000100
</pre></div>
<p>输出寄存器里面的值看一下,可以注意到寄存器 <tt class="docutils literal">rbx</tt> 的内存地址是 <tt class="docutils literal">func1.f</tt> 的地址。然后反编译可以看到执行到了 +31 这一行,将常量 <tt class="docutils literal">0x100</tt> 放在 rsp 内指针指向的内存地址。输出 rsp 的内容,然后显示地址指向内存的内容,可以看到是 <tt class="docutils literal">0x0000000000000000</tt>,输入 <tt class="docutils literal">ni</tt> 执行这一行汇编之后再看,就看到内存里面的内容变成了 <tt class="docutils literal">0x0000000000000100</tt>,也就是我们输入常量。</p>
<div class="highlight"><pre><span></span><span class="o">(</span>gdb<span class="o">)</span> ni
0x000000000040106c <span class="m">11</span> f<span class="o">(</span>0x100<span class="o">)</span>
<span class="o">(</span>gdb<span class="o">)</span> ni
0x000000000040106f <span class="m">11</span> f<span class="o">(</span>0x100<span class="o">)</span>
<span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.main:
0x0000000000401040 <+0>: mov %fs:0xfffffffffffffff8,%rcx
0x0000000000401049 <+9>: cmp 0x10<span class="o">(</span>%rcx<span class="o">)</span>,%rsp
0x000000000040104d <+13>: jbe 0x401086 <main.main+70>
0x000000000040104f <+15>: sub <span class="nv">$0</span>x10,%rsp
0x0000000000401053 <+19>: lea 0x88316<span class="o">(</span>%rip<span class="o">)</span>,%rbx <span class="c1"># 0x489370 <main.main.func1.f></span>
0x000000000040105a <+26>: mov %rbx,0x8<span class="o">(</span>%rsp<span class="o">)</span>
0x000000000040105f <+31>: movq <span class="nv">$0</span>x100,<span class="o">(</span>%rsp<span class="o">)</span>
0x0000000000401067 <+39>: mov 0x8<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x000000000040106c <+44>: mov %rbx,%rdx
<span class="o">=</span>> 0x000000000040106f <+47>: mov <span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x0000000000401072 <+50>: callq *%rbx
0x0000000000401074 <+52>: movq <span class="nv">$0</span>x100,<span class="o">(</span>%rsp<span class="o">)</span>
0x000000000040107c <+60>: callq 0x401000 <main.myFunc>
0x0000000000401081 <+65>: add <span class="nv">$0</span>x10,%rsp
0x0000000000401085 <+69>: retq
0x0000000000401086 <+70>: callq 0x44b040 <runtime.morestack_noctxt>
0x000000000040108b <+75>: jmp 0x401040 <main.main>
0x000000000040108d <+77>: int3
0x000000000040108e <+78>: int3
0x000000000040108f <+79>: int3
End of assembler dump.
<span class="o">(</span>gdb<span class="o">)</span> ni
0x0000000000401072 <span class="m">11</span> f<span class="o">(</span>0x100<span class="o">)</span>
<span class="o">(</span>gdb<span class="o">)</span> p <span class="nv">$rbx</span>
<span class="nv">$5</span> <span class="o">=</span> <span class="m">4198544</span>
<span class="o">(</span>gdb<span class="o">)</span> i r
rax 0xc820000180 <span class="m">859530330496</span>
rbx 0x401090 <span class="m">4198544</span>
...
<span class="o">(</span>gdb<span class="o">)</span> x/1xg 0x401090
0x401090 <main.main.func1>: 0xfffff8250c8b4864
</pre></div>
<p>接着往下执行到 +47 这一行,可以看到 <tt class="docutils literal">rbx</tt> 里面的值在这一行会有变化,<tt class="docutils literal">ni</tt> 执行完这一行,输出寄存器的内容看一下,然后显示 <tt class="docutils literal">rbx</tt> 指向的内存可以看到我们的匿名函数 <tt class="docutils literal">func1</tt>。</p>
<p>现在基本可以理清 <tt class="docutils literal">Go</tt> 里面匿名函数与正常的函数区别,参数的传递区别不大,只是在调用方面,匿名函数需要通过一个包装对象`func1.f`` 来调用匿名函数,这个过程通过 <tt class="docutils literal">rbx</tt> 进行二次寻址来完成调用。理论上,匿名函数也会比正常函数性能要差。</p>
</div>
<div class="section" id="id4">
<h2>Go 中闭包的实现</h2>
<p>闭包函数携带着定义这个函数的的环境。</p>
<div class="highlight"><pre><span></span><span class="kn">package</span> <span class="nx">main</span>
<span class="kd">func</span> <span class="nx">myFunc</span><span class="p">()</span> <span class="kd">func</span><span class="p">()</span> <span class="kt">int</span><span class="p">{</span>
<span class="nx">foo</span> <span class="o">:=</span> <span class="mi">0</span>
<span class="k">return</span> <span class="kd">func</span><span class="p">()</span> <span class="kt">int</span> <span class="p">{</span>
<span class="nx">foo</span><span class="o">++</span>
<span class="k">return</span> <span class="nx">foo</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nx">main</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">bar</span> <span class="o">:=</span> <span class="nx">myFunc</span><span class="p">()</span>
<span class="nx">value_1</span> <span class="o">:=</span> <span class="nx">bar</span><span class="p">()</span>
<span class="nx">value_2</span> <span class="o">:=</span> <span class="nx">bar</span><span class="p">()</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">value_1</span><span class="p">)</span> <span class="c1">// 1</span>
<span class="nb">println</span><span class="p">(</span><span class="nx">value_2</span><span class="p">)</span> <span class="c1">// 2</span>
<span class="p">}</span>
</pre></div>
<p>与分析匿名函数的过程一样,编译然后通过 <tt class="docutils literal">gdb</tt> 来跟踪。</p>
<div class="highlight"><pre><span></span>$ go build -gcflags <span class="s2">"-N -l -m"</span> closure_func.go
<span class="c1"># command-line-arguments</span>
./closure_func.go:5: func literal escapes to heap
./closure_func.go:5: func literal escapes to heap
./closure_func.go:4: moved to heap: foo
./closure_func.go:6: <span class="p">&</span>foo escapes to heap
$ gdb closure_func
<span class="o">(</span>gdb<span class="o">)</span> b main.main
Breakpoint <span class="m">1</span> at 0x4010d0: file /root/data/example/closures/closure_func.go, line <span class="m">11</span>.
<span class="o">(</span>gdb<span class="o">)</span> r
Starting program: /root/data/example/closures/closure_func
<span class="o">[</span>New LWP <span class="m">5367</span><span class="o">]</span>
<span class="o">[</span>New LWP <span class="m">5368</span><span class="o">]</span>
<span class="o">[</span>New LWP <span class="m">5370</span><span class="o">]</span>
<span class="o">[</span>New LWP <span class="m">5369</span><span class="o">]</span>
Breakpoint <span class="m">1</span>, main.main <span class="o">()</span> at /root/data/example/closures/closure_func.go:11
<span class="m">11</span> func main<span class="o">()</span> <span class="o">{</span>
<span class="o">(</span>gdb<span class="o">)</span> i locals
<span class="nv">value_2</span> <span class="o">=</span> <span class="m">859530428512</span>
<span class="nv">value_1</span> <span class="o">=</span> <span class="m">0</span>
<span class="nv">bar</span> <span class="o">=</span> <span class="o">{</span>void <span class="o">(</span>int *<span class="o">)}</span> 0xc820039f40
</pre></div>
<p><tt class="docutils literal">gdb</tt> 在 <tt class="docutils literal">main.main</tt> 设置断点并输出环境变量可以看到 <tt class="docutils literal">bar</tt>,而且 <tt class="docutils literal">bar</tt> 是一个指针。</p>
<div class="highlight"><pre><span></span> <span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.main:
0x00000000004010d0 <+0>: mov %fs:0xfffffffffffffff8,%rcx
0x00000000004010d9 <+9>: cmp 0x10<span class="o">(</span>%rcx<span class="o">)</span>,%rsp
0x00000000004010dd <+13>: jbe 0x40115c <main.main+140>
0x00000000004010df <+15>: sub <span class="nv">$0</span>x20,%rsp
0x00000000004010e3 <+19>: callq 0x401000 <main.myFunc>
<span class="o">=</span>> 0x00000000004010e8 <+24>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010ec <+28>: mov %rbx,0x18<span class="o">(</span>%rsp<span class="o">)</span>
0x00000000004010f1 <+33>: mov 0x18<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010f6 <+38>: mov %rbx,%rdx
...
<span class="o">(</span>gdb<span class="o">)</span> i r
rax 0x80000 <span class="m">524288</span>
rbx 0xc82000a140 <span class="m">859530371392</span>
...
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0xc82000a140
0xc82000a140: 0x0000000000401170 0x000000c82000a0b8
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0x0000000000401170
0x401170 <main.myFunc.func1>: 0x085a8b4810ec8348 0x44c74808245c8948
</pre></div>
<p>将程序继续向下走到 +24 这一行,然后输出寄存器的信息,能够发现寄存器 <tt class="docutils literal">rbx</tt> 与之前匿名函数的作用类似,都指向了闭包返回对象。里面封装着我们需要用到的匿名函数。可以看到匿名函数作为返回结果,整个调用过程跟是否形成闭包区别不大。那这个区别在哪里呢?</p>
<div class="highlight"><pre><span></span><span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.main:
0x00000000004010d0 <+0>: mov %fs:0xfffffffffffffff8,%rcx
0x00000000004010d9 <+9>: cmp 0x10<span class="o">(</span>%rcx<span class="o">)</span>,%rsp
0x00000000004010dd <+13>: jbe 0x40115c <main.main+140>
0x00000000004010df <+15>: sub <span class="nv">$0</span>x20,%rsp
0x00000000004010e3 <+19>: callq 0x401000 <main.myFunc>
0x00000000004010e8 <+24>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010ec <+28>: mov %rbx,0x18<span class="o">(</span>%rsp<span class="o">)</span>
0x00000000004010f1 <+33>: mov 0x18<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010f6 <+38>: mov %rbx,%rdx
<span class="o">=</span>> 0x00000000004010f9 <+41>: mov <span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x00000000004010fc <+44>: callq *%rbx
0x00000000004010fe <+46>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x0000000000401102 <+50>: mov %rbx,0x10<span class="o">(</span>%rsp<span class="o">)</span>
...
End of assembler dump.
<span class="o">(</span>gdb<span class="o">)</span> ni
0x00000000004010fc <span class="m">13</span> value_1 :<span class="o">=</span> bar<span class="o">()</span>
<span class="o">(</span>gdb<span class="o">)</span> si
main.myFunc.func1 <span class="o">(</span>~r0<span class="o">=</span><span class="m">859530371392</span><span class="o">)</span> at /root/data/example/closures/closure_func.go:5
<span class="m">5</span> <span class="k">return</span> func<span class="o">()</span> int <span class="o">{</span>
<span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.myFunc.func1:
<span class="o">=</span>> 0x0000000000401170 <+0>: sub <span class="nv">$0</span>x10,%rsp
0x0000000000401174 <+4>: mov 0x8<span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x0000000000401178 <+8>: mov %rbx,0x8<span class="o">(</span>%rsp<span class="o">)</span>
0x000000000040117d <+13>: movq <span class="nv">$0</span>x0,0x18<span class="o">(</span>%rsp<span class="o">)</span>
0x0000000000401186 <+22>: mov 0x8<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x000000000040118b <+27>: mov <span class="o">(</span>%rbx<span class="o">)</span>,%rbp
...
End of assembler dump.
<span class="o">(</span>gdb<span class="o">)</span> i r
rax 0x80000 <span class="m">524288</span>
rbx 0x401170 <span class="m">4198768</span>
rcx 0xc820000180 <span class="m">859530330496</span>
rdx 0xc82000a140 <span class="m">859530371392</span>
...
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0xc82000a140
0xc82000a140: 0x0000000000401170 0x000000c82000a0b8
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0x0000000000401170
0x401170 <main.myFunc.func1>: 0x085a8b4810ec8348 0x44c74808245c8948
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0x000000c82000a0b8
0xc82000a0b8: 0x0000000000000000 0x3d534e4d554c4f43
</pre></div>
<p>让程序执行到 +44 行,<tt class="docutils literal">si</tt> 进入到匿名函数内部。在 <tt class="docutils literal">func1</tt> 内部可以看到从 <tt class="docutils literal">rdx</tt> 取数据。输出 <tt class="docutils literal">rdx</tt> 内容,可以看到前面指向匿名函数,而后面则指向另外的内容 <tt class="docutils literal">0x0000000000000000</tt>。</p>
<div class="highlight"><pre><span></span><span class="o">(</span>gdb<span class="o">)</span> b <span class="m">14</span>
Breakpoint <span class="m">2</span> at 0x401107: file /root/data/example/closures/closure_func.go, line <span class="m">14</span>.
<span class="o">(</span>gdb<span class="o">)</span> c
Continuing.
Breakpoint <span class="m">2</span>, main.main <span class="o">()</span> at /root/data/example/closures/closure_func.go:14
<span class="m">14</span> value_2 :<span class="o">=</span> bar<span class="o">()</span>
<span class="m">14</span> value_2 :<span class="o">=</span> bar<span class="o">()</span>
<span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.main:
0x00000000004010d0 <+0>: mov %fs:0xfffffffffffffff8,%rcx
0x00000000004010d9 <+9>: cmp 0x10<span class="o">(</span>%rcx<span class="o">)</span>,%rsp
0x00000000004010dd <+13>: jbe 0x40115c <main.main+140>
0x00000000004010df <+15>: sub <span class="nv">$0</span>x20,%rsp
0x00000000004010e3 <+19>: callq 0x401000 <main.myFunc>
0x00000000004010e8 <+24>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010ec <+28>: mov %rbx,0x18<span class="o">(</span>%rsp<span class="o">)</span>
0x00000000004010f1 <+33>: mov 0x18<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010f6 <+38>: mov %rbx,%rdx
0x00000000004010f9 <+41>: mov <span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x00000000004010fc <+44>: callq *%rbx
0x00000000004010fe <+46>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x0000000000401102 <+50>: mov %rbx,0x10<span class="o">(</span>%rsp<span class="o">)</span>
<span class="o">=</span>> 0x0000000000401107 <+55>: mov 0x18<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x000000000040110c <+60>: mov %rbx,%rdx
0x000000000040110f <+63>: mov <span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x0000000000401112 <+66>: callq *%rbx
0x0000000000401114 <+68>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
...
End of assembler dump.
<span class="o">(</span>gdb<span class="o">)</span> ni <span class="m">3</span>
0x0000000000401112 <span class="m">14</span> value_2 :<span class="o">=</span> bar<span class="o">()</span>
<span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.main:
0x00000000004010d0 <+0>: mov %fs:0xfffffffffffffff8,%rcx
0x00000000004010d9 <+9>: cmp 0x10<span class="o">(</span>%rcx<span class="o">)</span>,%rsp
0x00000000004010dd <+13>: jbe 0x40115c <main.main+140>
0x00000000004010df <+15>: sub <span class="nv">$0</span>x20,%rsp
0x00000000004010e3 <+19>: callq 0x401000 <main.myFunc>
0x00000000004010e8 <+24>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010ec <+28>: mov %rbx,0x18<span class="o">(</span>%rsp<span class="o">)</span>
0x00000000004010f1 <+33>: mov 0x18<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x00000000004010f6 <+38>: mov %rbx,%rdx
0x00000000004010f9 <+41>: mov <span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x00000000004010fc <+44>: callq *%rbx
0x00000000004010fe <+46>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x0000000000401102 <+50>: mov %rbx,0x10<span class="o">(</span>%rsp<span class="o">)</span>
0x0000000000401107 <+55>: mov 0x18<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x000000000040110c <+60>: mov %rbx,%rdx
0x000000000040110f <+63>: mov <span class="o">(</span>%rdx<span class="o">)</span>,%rbx
<span class="o">=</span>> 0x0000000000401112 <+66>: callq *%rbx
0x0000000000401114 <+68>: mov <span class="o">(</span>%rsp<span class="o">)</span>,%rbx
...
End of assembler dump.
<span class="o">(</span>gdb<span class="o">)</span> si
main.myFunc.func1 <span class="o">(</span>~r0<span class="o">=</span><span class="m">1</span><span class="o">)</span> at /root/data/example/closures/closure_func.go:5
<span class="m">5</span> <span class="k">return</span> func<span class="o">()</span> int <span class="o">{</span>
<span class="o">(</span>gdb<span class="o">)</span> disassemble
Dump of assembler code <span class="k">for</span> <span class="k">function</span> main.myFunc.func1:
<span class="o">=</span>> 0x0000000000401170 <+0>: sub <span class="nv">$0</span>x10,%rsp
0x0000000000401174 <+4>: mov 0x8<span class="o">(</span>%rdx<span class="o">)</span>,%rbx
0x0000000000401178 <+8>: mov %rbx,0x8<span class="o">(</span>%rsp<span class="o">)</span>
0x000000000040117d <+13>: movq <span class="nv">$0</span>x0,0x18<span class="o">(</span>%rsp<span class="o">)</span>
0x0000000000401186 <+22>: mov 0x8<span class="o">(</span>%rsp<span class="o">)</span>,%rbx
0x000000000040118b <+27>: mov <span class="o">(</span>%rbx<span class="o">)</span>,%rbp
...
End of assembler dump.
<span class="o">(</span>gdb<span class="o">)</span> i r
rax 0x80000 <span class="m">524288</span>
rbx 0x401170 <span class="m">4198768</span>
rcx 0xc820000180 <span class="m">859530330496</span>
rdx 0xc82000a140 <span class="m">859530371392</span>
...
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0xc82000a140
0xc82000a140: 0x0000000000401170 0x000000c82000a0b8
<span class="o">(</span>gdb<span class="o">)</span> x/2xg 0x000000c82000a0b8
0xc82000a0b8: 0x0000000000000001 0x3d534e4d554c4f43
<span class="o">(</span>gdb<span class="o">)</span> i locals
<span class="p">&</span><span class="nv">foo</span> <span class="o">=</span> 0xc82000a0b8
</pre></div>
<p>设置断点进入到下一次闭包内,输出相同的内容,会发现 <tt class="docutils literal">rdx</tt> 后半段指向的内容发生了变化。通过 <tt class="docutils literal">i locals</tt> 查看环境变量,可以看到 foo 的地址是 <tt class="docutils literal">0xc82000a0b8</tt> , 跟 <tt class="docutils literal">rdx</tt> 的后半段内容一样。</p>
<p>由此可以判断,闭包返回的包装对象是一个复合结构,里面包含匿名函数的地址,以及环境变量的地址。</p>
</div>
<div class="section" id="id5">
<h2>注意事项</h2>
<ol class="arabic simple">
<li>匿名函数作为返回对象性能上要比正常的函数性能要差。</li>
<li>闭包可能会导致变量逃逸到堆上来延长变量的生命周期,给 GC 带来压力。</li>
<li>破除迷信,批判性的看任何人的 Blog。</li>
</ol>
<p>PS: 有些 Blog 写的内容都是错的还自诩对 Go 底层非常了解,这种误人子弟的不要太多。</p>
</div>
GoBeansDB 架构设计2016-09-06T00:00:00+08:002016-09-06T00:00:00+08:00SunisDowntag:sunisdown.me,2016-09-06:/gobeansdb-jia-gou-she-ji.html<p>从产品转系统有一段时间了,转到核心系统之后一直在维护 GoBeansDB,最近也在慢慢的熟悉另外一个新的项目(暂时保密)。</p>
<p>BeansDB 是豆瓣内部实现的分布式存储系统,最开始的时候是 <a class="reference external" href="https://github.com/davies">Davies</a> 用 C 来实现的。2015 年的时候,内部决定开始用 Go 来重新写 BeansDB,然后 xiufeng 与 zzl 就实现了现在的 GoBeansDB。</p>
<div class="section" id="k-v">
<h2>为什么要自己实现一套 k/v 存储</h2>
<p>我在刚刚接手 GoBeansDB 的时候,想过这个问题。既然有那么多优秀的数据库系统,为什么豆瓣还需要自己重新实现一套 k/v 存储?
这个问题可以拆分成两个方面,一个是为什么要用 K/V 数据库。一个是为什么要自己造轮子。</p>
<ol class="arabic simple">
<li>首先是因为数据大,而且数据是非结构化数据,像豆瓣的日记,就是一批很长的字符串。</li>
<li>其次是非常多的随机读。</li>
<li>有的时候会有大量的写操作</li>
<li>不需要外键什么的 …</li></ol></div><p>从产品转系统有一段时间了,转到核心系统之后一直在维护 GoBeansDB,最近也在慢慢的熟悉另外一个新的项目(暂时保密)。</p>
<p>BeansDB 是豆瓣内部实现的分布式存储系统,最开始的时候是 <a class="reference external" href="https://github.com/davies">Davies</a> 用 C 来实现的。2015 年的时候,内部决定开始用 Go 来重新写 BeansDB,然后 xiufeng 与 zzl 就实现了现在的 GoBeansDB。</p>
<div class="section" id="k-v">
<h2>为什么要自己实现一套 k/v 存储</h2>
<p>我在刚刚接手 GoBeansDB 的时候,想过这个问题。既然有那么多优秀的数据库系统,为什么豆瓣还需要自己重新实现一套 k/v 存储?
这个问题可以拆分成两个方面,一个是为什么要用 K/V 数据库。一个是为什么要自己造轮子。</p>
<ol class="arabic simple">
<li>首先是因为数据大,而且数据是非结构化数据,像豆瓣的日记,就是一批很长的字符串。</li>
<li>其次是非常多的随机读。</li>
<li>有的时候会有大量的写操作</li>
<li>不需要外键什么的</li>
</ol>
<p>上面四点可以排除掉类似 MySQL 这种传统的关系型数据库。</p>
<p>排除掉传统的关系行数据库之后,就需要对比现存的 K/V 数据库。</p>
<p>现在比较流行的 K/V 数据库有 <tt class="docutils literal">LevelDB</tt> , <tt class="docutils literal">MongoDB</tt> ,还有 <tt class="docutils literal">Cassandra</tt> ,现在看来这些项目都足够成熟。但是如果追溯到 BeansDB 项目最开始的时候,也就是 2012 年的时候,那个时间点并没有太好的选择。即使现在看来,除了 <tt class="docutils literal">Cassandra</tt> 之外,像 <tt class="docutils literal">LevelDB</tt>, <tt class="docutils literal">MongoDB</tt> 也不能满足我们的目标:</p>
<ol class="arabic simple">
<li>读写都需要比较低的 latency</li>
<li>数据量非常大,所以数据要写在磁盘上,数据库需要能够容纳比内存大的多的数据</li>
<li>高可用,单点故障不影响系统正常运行</li>
<li>高吞吐,尤其是针对写操作</li>
<li>能够快速恢复有问题的节点</li>
</ol>
<p>这 5 点也可以排除调 <tt class="docutils literal">MongoDB</tt> 与 <tt class="docutils literal">LevelDB</tt> 。</p>
<p>当然上面这些都是我做的推断,但是这些应该都不是最主要的原因。最主要的原因应该是豆瓣的工程师文化比较好,鼓励工程师去寻找一个最贴合业务的解决方案,并且这个工程师的团队还足够强,两者缺一不可。如果没有很强的工程师文化,可能会直接引入开源的解决方案,虽然不一定合适,但是应该足够解决痛点。如果工程师实力不够,也就没有能力去自己实现一套类似的系统。而且与其去引入一个复杂的,自己无法完全掌控的开源项目,不如自己实现一套贴合业务的,简单的系统。这样内部可以根据业务的需要来作调整,同时自己实现一个系统也比完全掌握一个庞大的开源项目要简单。一旦出现问题也比较容易找到问题所在。</p>
</div>
<div class="section" id="go-beansdb">
<h2>为什么要用 Go 重新实现 BeansDB</h2>
<p>BeansDB 是用 C 来实现的,为什么现在改用 Go 来实现?</p>
<ul class="simple">
<li>一个很重要的原因是 Go 的代码相比与 C 更容易维护。对一个工程师而言,Go 的学习成本比 C 要低很多。<ul>
<li>用 Go 可以比较快速的写出健壮的系统,而用 C 来写的话,则需要一定的经验积累。</li>
<li>Go 提供了可用的测试框架,写测试相对于 C/C++ 要方便</li>
<li>Go 标准库里面提供了方便的性能分析工具,用 C 也有类似的工具,但是做不到开箱即用</li>
</ul>
</li>
<li>还有 Go 的标准库也足够完善,不需要用 C 来重复造轮子。</li>
<li>Go 的执行效率虽然比 C 差,但是 BeansDB 的瓶颈是 IOPS,所以 Go 的执行效率并不会成为瓶颈。</li>
</ul>
</div>
<div class="section" id="id1">
<h2>GoBeansDB 的架构设计</h2>
<p>GoBeansDB 是基于 <a class="reference external" href="http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf">Dynamo</a> 与 <a class="reference external" href="http://basho.com/wp-content/uploads/2015/05/bitcask-intro.pdf">Bitcask</a> 两篇论文来做的实现,这里优先讨论基于 Bitcask 实现的存储引擎。Bitcask 有一个缺点在于所有的 key 值都必须放在内存里面,GoBeansDB 这这个基础之上做了一些优化,绕过了 Bitcask 这个痛点。</p>
<p>GobeansDB 的存储有有两个比较重要的组成部分,一个是索引(htree),一个是数据文件(data)。索引与数据文件组成 Bucket。Bucket 的概念类似与关系行数据库里面的 table,在 GoBeansDB 的实现中就是给一个 Bucket 分配一个文件夹,这个文件夹下面放着相关的数据。每个 Bucket 下面一次只允许打开一个文件。打开的这个文件会一直保持打开的状态,一直等到追加到活跃文件超出阈值。文件超出阈值之后就关闭,然后新建一个新的继续添加。data 文件一旦关闭之后,文件就转换成为不活跃的数据文件。无法再往这个 data 文件上面追加数据。</p>
<img alt="" src="images/GoBeansDB.png" />
<p>状态为 active 的数据文件只做追加操作,这样连续的写入操作也不会明显增加磁盘的 IO 使用量。这种设计也极大的简化了数据的写入操作。</p>
<p>上面的图简单描述了 Bucket 内部文件的架构,每条数据里面包含TS(TimeStamp),Flag,Ver(Version),ValueHash,RecSize(单条记录的主要内容的大小),Value,crc(key,value,header 的 crc),ksz(Key Size),vsz(Value Size),pos(Position,这条记录在文件中的位置),Header。</p>
<p>当插入新数据的时候,直接在文件尾部添加这种结构的数据。删除操作是对原有的数据做更新操作,并将 Ver 绝对值+1,转变为负数。</p>
<p>在文件写入完成之后,需要更新内存里面的数据结构,也就是前面提到的 HTree,HTree 是一个 Hash Tree,结构如下</p>
<img alt="" src="images/htree.png" />
<p><tt class="docutils literal">levels</tt> 表示真实的树状结构, <tt class="docutils literal">leafs</tt> 是树的叶子,保存着真实的数据。</p>
<img alt="" src="images/htree_data_file.png" />
<p>这种数据结构下读取一个值也非常简单,大多数情况下最多只需要一次 seek 。我们首先在 Htree 中通过 <tt class="docutils literal">levels</tt> 找到 <tt class="docutils literal">key</tt> 对应的 <tt class="docutils literal">leafs</tt> , 然后通过 <tt class="docutils literal">leafs</tt> 里面的报错的 <tt class="docutils literal">Pos</tt> 来拿到文件编号(chunkID)以及 offset,这样就可以通过文件编号(chunkID)和 offset 来拿到保存的数据。在很多情况下文件系统的缓存会让这个读操作比预期的要快。</p>
<p>到这里关于 GoBeansDB <tt class="docutils literal">wirte/delete/read</tt> 相关的操作都已经基本完成。但是仅仅这样还不能完备。</p>
<div class="section" id="gc">
<h3>GC 操作</h3>
<p>GoBeansDB 的模型非常简单,<tt class="docutils literal">write/delete</tt> 操作都是在文件尾部追加新的数据,这样存在一个问题就是占用的磁盘空间比真实的数据要多。所以我们引入了 GC 机制来回收垃圾,释放内存与磁盘空间。</p>
<p>在 GoBeansDB 中,通过增量 GC 来减小 GC 的开销。xiufeng 通过分析 BeansDB 的日志,统计出一条新写入的数据,修改操作基本在写入之后的 7 天之内,所以我们保留 7 天之内的新数据不做 GC。然后在每天晚上,访问量较低的时候,分批做增量 GC。</p>
<p>GC 的过程是将 datafile 里面已经过时的数据清除掉,比如旧版本的value,已经标记为删除掉的key。</p>
<img alt="" src="images/GC.png" />
<p>如 上图所示,GC 会读取状态为不活跃的数据文件,用其中存活的数据或者最新版本的数据生成一份新的数据文件,同时为这个新的数据文件创建一个 hint file。</p>
<p>这就是 GoBeansDB 大概的架构。当然,整个系统里面还会涉及到很多细节这篇文章没有覆盖到。如果你对更多细节感兴趣,可以联系我们。</p>
</div>
</div>
One Year2016-02-02T00:00:00+08:002016-02-02T00:00:00+08:00Sunisdowntag:sunisdown.me,2016-02-02:/one-year.html<p>看一下其实有一年没有更新 Blog。再不写点什么对不起自己域名ˊㅅˋ</p>
<p>再过几天应该就来豆厂一周年了,豆厂大把的业余时间我都拿来做自己感兴趣的事儿。
比如看 v6 代码,研究了下 paxos(然后并没有卵用,还是不懂)。</p>
<p>认真想一下其实很难写出什么高质量的 Blog,在豆瓣一年,貌似工作中并没有形成成体系的内容。而平时自己折腾的东西,也很难形成体系,写不出很高质量的 Blog。
不过平时自己折腾的东西,倒是可以形成一本笔记,方便自己查阅。所以16年应该会以发布笔记为主,更新 Blog 为辅助。</p>
<p>我很喜欢豆厂工程师之间沟通的方式,直接在 github 上面沟通代码,每次代码都有人 review 的真的有点幸福。
要知道帮别人 review 代码是一个比较耗费精力的工作,你需要了解这次提交的目的,看一下实现是否优雅,尽可能发现潜在的问题。
所以有一堆技术还不错的小伙伴帮 review 代码,在自己的代码风格上,代码的简洁程度上,都有一定的提升,
至少在提交代码的时候会比较谨慎,不会审核恶心的代码都往里面堆。一定程度上提高了自己对自己的要求。</p>
<p>之前来豆瓣面试的时候 …</p><p>看一下其实有一年没有更新 Blog。再不写点什么对不起自己域名ˊㅅˋ</p>
<p>再过几天应该就来豆厂一周年了,豆厂大把的业余时间我都拿来做自己感兴趣的事儿。
比如看 v6 代码,研究了下 paxos(然后并没有卵用,还是不懂)。</p>
<p>认真想一下其实很难写出什么高质量的 Blog,在豆瓣一年,貌似工作中并没有形成成体系的内容。而平时自己折腾的东西,也很难形成体系,写不出很高质量的 Blog。
不过平时自己折腾的东西,倒是可以形成一本笔记,方便自己查阅。所以16年应该会以发布笔记为主,更新 Blog 为辅助。</p>
<p>我很喜欢豆厂工程师之间沟通的方式,直接在 github 上面沟通代码,每次代码都有人 review 的真的有点幸福。
要知道帮别人 review 代码是一个比较耗费精力的工作,你需要了解这次提交的目的,看一下实现是否优雅,尽可能发现潜在的问题。
所以有一堆技术还不错的小伙伴帮 review 代码,在自己的代码风格上,代码的简洁程度上,都有一定的提升,
至少在提交代码的时候会比较谨慎,不会审核恶心的代码都往里面堆。一定程度上提高了自己对自己的要求。</p>
<p>之前来豆瓣面试的时候,也曾经表示过自己想要进 DAE 或者做 anti,但是日军哥哥跟我说 anti 跟我现在组用的是同一套东西,技术上基本上是通的的。
于是我来了现在这个组 O_O,经过这一年的接触,我发现还是喜欢 anti 或者 DAE 更多一些。</p>
<p>所以还是安安静静的做自己的事儿,安安静静做自己感兴趣的事儿。</p>
理解 Python 的 LEGB2015-02-03T00:00:00+08:002015-02-03T00:00:00+08:00SunisDowntag:sunisdown.me,2015-02-03:/li-jie-python-de-legb.html<div class="section" id="id1">
<h2>名字空间</h2>
<p>Python 的名字空间是 Python 一个非常核心的内容。 其他语言中如 C
中,变量名是内存地址的别名,而在 Python
中,名字是一个字符串对象,它与他指向的对象构成一个{name:object}关联。
Python 由很多名字空间,而 LEGB 则是名字空间的一种查找规则。</p>
</div>
<div class="section" id="id2">
<h2>作用域</h2>
<p>Python
中<tt class="docutils literal"><span class="pre">name-object</span></tt>的关联存储在不同的作用域中,各个不同的作用域是相互独立的。而我们就在不同的作用域中搜索<tt class="docutils literal"><span class="pre">name-object</span></tt>。</p>
<p>举个栗子,来说明作用域是相互独立的。</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="n">i</span> <span class="o">=</span> <span class="s2">"G"</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="n">i</span> <span class="o">=</span> <span class="s2">"L"</span>
<span class="k">print</span> <span class="n">i</span><span class="p">,</span> <span class="s2">"in locals"</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="n">test</span><span class="p">()</span>
<span class="n">L …</span></pre></div></div><div class="section" id="id1">
<h2>名字空间</h2>
<p>Python 的名字空间是 Python 一个非常核心的内容。 其他语言中如 C
中,变量名是内存地址的别名,而在 Python
中,名字是一个字符串对象,它与他指向的对象构成一个{name:object}关联。
Python 由很多名字空间,而 LEGB 则是名字空间的一种查找规则。</p>
</div>
<div class="section" id="id2">
<h2>作用域</h2>
<p>Python
中<tt class="docutils literal"><span class="pre">name-object</span></tt>的关联存储在不同的作用域中,各个不同的作用域是相互独立的。而我们就在不同的作用域中搜索<tt class="docutils literal"><span class="pre">name-object</span></tt>。</p>
<p>举个栗子,来说明作用域是相互独立的。</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="n">i</span> <span class="o">=</span> <span class="s2">"G"</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="n">i</span> <span class="o">=</span> <span class="s2">"L"</span>
<span class="k">print</span> <span class="n">i</span><span class="p">,</span> <span class="s2">"in locals"</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="n">test</span><span class="p">()</span>
<span class="n">L</span> <span class="ow">in</span> <span class="nb">locals</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">14</span><span class="p">]:</span> <span class="k">print</span> <span class="n">i</span><span class="p">,</span> <span class="s2">"in globals"</span>
<span class="n">G</span> <span class="ow">in</span> <span class="nb">globals</span>
</pre></div>
<p>在上面的栗子中,我们定义了两次 i,在 test 函数中是 i-L,在外面是
i-G。为什么在 test 函数中,我们 i 指向的是对象 L,而在外面,i 指向的则是
G?这就是 LEGB 的作用。</p>
</div>
<div class="section" id="id3">
<h2>简述</h2>
<p>简而言之,LEGB 代表名字查找顺序:
<tt class="docutils literal">locals <span class="pre">-></span> enclosing function <span class="pre">-></span> globals <span class="pre">-></span> __builtins__</tt></p>
<ul class="simple">
<li>locals 是函数内的名字空间,包括局部变量和形参</li>
<li>enclosing 外部嵌套函数的名字空间(闭包中常见)</li>
<li>globals 全局变量,函数定义所在模块的名字空间</li>
<li>builtins 内置模块的名字空间</li>
</ul>
<p>所以,在 Python 中检索一个变量的时候,优先回到 locals
里面来检索,检索不到的情况下会检索 enclosing ,enclosing 没有则到
globals 全局变量里面检索,最后是到 builtins 里面来检索。</p>
<p>当然,因为 builtins 的特殊性,我们可以直接在 builtins
里面添加变量,这样就可以在任意模块中访问变量,不过这种方法太过于变态,不推荐这么做。</p>
</div>
<div class="section" id="locals-globals">
<h2>locals,globals</h2>
<p>函数的形参跟内部变量都存储在 locals 中。</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="o">...</span><span class="p">:</span> <span class="n">a</span> <span class="o">=</span> <span class="n">x</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span> <span class="n">a</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span> <span class="nb">locals</span><span class="p">()</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="n">f</span><span class="p">(</span><span class="s2">"hello"</span><span class="p">)</span>
<span class="n">hello</span>
<span class="p">{</span><span class="s1">'a'</span><span class="p">:</span> <span class="s1">'hello'</span><span class="p">,</span> <span class="s1">'x'</span><span class="p">:</span> <span class="s1">'hello'</span><span class="p">}</span>
</pre></div>
<p>不过在函数内部调用global 声明的时候,可以将变量存储在 globals 中</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">6</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="o">...</span><span class="p">:</span> <span class="k">global</span> <span class="n">a</span>
<span class="o">...</span><span class="p">:</span> <span class="n">a</span> <span class="o">=</span> <span class="n">x</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span> <span class="n">a</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span> <span class="nb">locals</span><span class="p">()</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">7</span><span class="p">]:</span> <span class="n">f</span><span class="p">(</span><span class="s2">"hello"</span><span class="p">)</span>
<span class="n">hello</span>
<span class="p">{</span><span class="s1">'x'</span><span class="p">:</span> <span class="s1">'hello'</span><span class="p">}</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="k">print</span> <span class="n">a</span>
<span class="n">hello</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="k">print</span> <span class="n">x</span>
<span class="o">---------------------------------------------------------------------------</span>
<span class="ne">NameError</span> <span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">)</span>
<span class="o"><</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">9</span><span class="o">-</span><span class="mi">2</span><span class="n">d264e11d975</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span><span class="p">()</span>
<span class="o">----></span> <span class="mi">1</span> <span class="k">print</span> <span class="n">x</span>
<span class="ne">NameError</span><span class="p">:</span> <span class="n">name</span> <span class="s1">'x'</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">defined</span>
</pre></div>
<p>如上面栗子中那样,在函数中声明 a 为全局变量,则函数 f 的 locals只有参数
x,而没有变量,而在外部可以使用变量 a,而使用 x
的时候则是<tt class="docutils literal">NameError</tt></p>
</div>
<div class="section" id="enclosed">
<h2>Enclosed</h2>
<p>Enclosing 是外部嵌套函数的名字空间。我们经常在闭包中用到。在
Python3中提供了一个
nonlocal关键字来修改外部嵌套函数的名字空间,但是要使用
Python3才有,我等使用 Python2的只能眼馋一下。</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="o">....</span><span class="p">:</span> <span class="n">a_var</span> <span class="o">=</span> <span class="s1">'enclosed value'</span>
<span class="o">....</span><span class="p">:</span> <span class="k">print</span> <span class="n">a_var</span>
<span class="o">....</span><span class="p">:</span> <span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="o">....</span><span class="p">:</span> <span class="n">a_var</span> <span class="o">=</span> <span class="s1">'local value'</span>
<span class="o">....</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="n">a_var</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span> <span class="n">inner</span><span class="p">()</span>
<span class="o">....</span><span class="p">:</span> <span class="k">print</span> <span class="n">a_var</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">outer</span><span class="p">()</span>
<span class="n">enclosed</span> <span class="n">value</span>
<span class="n">local</span> <span class="n">value</span>
<span class="n">enclosed</span> <span class="n">value</span>
</pre></div>
<p>下面的栗子简单示范一下 nonlocal 的用法,实在
Python3下面才可以正常运行的:</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="n">a_var</span> <span class="o">=</span> <span class="s1">'global value'</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="o">...</span><span class="p">:</span> <span class="n">a_var</span> <span class="o">=</span> <span class="s2">"local value"</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="s2">"outer befor"</span><span class="p">,</span> <span class="n">a_var</span><span class="p">)</span>
<span class="o">...</span><span class="p">:</span> <span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="o">...</span><span class="p">:</span> <span class="n">nonlocal</span> <span class="n">a_var</span>
<span class="o">...</span><span class="p">:</span> <span class="n">a_var</span> <span class="o">=</span> <span class="s2">"inner value"</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="s2">"in inner():"</span><span class="p">,</span> <span class="n">a_var</span><span class="p">)</span>
<span class="o">...</span><span class="p">:</span> <span class="n">inner</span><span class="p">()</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="s2">"outer inner:"</span><span class="p">,</span> <span class="n">a_var</span><span class="p">)</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="n">outer</span><span class="p">()</span>
<span class="n">outer</span> <span class="n">befor</span> <span class="n">local</span> <span class="n">value</span>
<span class="ow">in</span> <span class="n">inner</span><span class="p">():</span> <span class="n">inner</span> <span class="n">value</span>
<span class="n">outer</span> <span class="n">inner</span><span class="p">:</span> <span class="n">inner</span> <span class="n">value</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="k">print</span><span class="p">(</span><span class="n">a_var</span><span class="p">)</span>
<span class="k">global</span> <span class="n">value</span>
</pre></div>
</div>
<div class="section" id="builtins">
<h2>builtins</h2>
<p>builtins 则是内置模块,轻易不要修改</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">19</span><span class="p">]:</span> <span class="n">b</span>
<span class="o">---------------------------------------------------------------------------</span>
<span class="ne">NameError</span> <span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">)</span>
<span class="o"><</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">3</span><span class="n">b5d5c371295</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span><span class="p">()</span>
<span class="o">----></span> <span class="mi">1</span> <span class="n">b</span>
<span class="ne">NameError</span><span class="p">:</span> <span class="n">name</span> <span class="s1">'b'</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">defined</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">20</span><span class="p">]:</span> <span class="n">__builtins__</span><span class="o">.</span><span class="n">b</span> <span class="o">=</span> <span class="s2">"builtins"</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">21</span><span class="p">]:</span> <span class="n">b</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">21</span><span class="p">]:</span> <span class="s1">'builtins'</span>
</pre></div>
<p>上面栗子中在第一次调用<tt class="docutils literal">b</tt>的时候报错<tt class="docutils literal">NameError</tt>,之后我们修改
builtins
的名字空间,将名字<tt class="docutils literal">b</tt>与值<tt class="docutils literal">"builtins"</tt>进行关联,就可以正常调用了。这种非常规用法不建议使用。</p>
</div>
Python 之 GIL2015-02-03T00:00:00+08:002015-02-03T00:00:00+08:00Sunisdowntag:sunisdown.me,2015-02-03:/python-zhi-gil.html<p>注: 本篇 Blog 是参照
<a class="reference external" href="http://book.douban.com/subject/3117898/">Python源码剖析</a>过的 Python 源
码,内容算是读书笔记吧,中间还不小心翻错了分支,翻到了3.4版本,不过看起来3.4的 GIL
处理要比2.7的优雅一些.总之,感谢陈孺大神</p>
<div class="section" id="gil">
<h2>什么是 GIL</h2>
<p>GIL(Global Interpreter Lock)
是解释器全局锁,用来互斥线程对于Python虚拟机的使用.</p>
</div>
<div class="section" id="id1">
<h2>为什么用 GIL</h2>
<p>我们知道 Python
的线程调度是由机器来调度的,在线程执行的时候,我们不能决定线程什么时候挂起.假设线程
A与线程 B 都保存在对象
Obj_1,而这种情况下有可能会发生一些比较坑的问题:比如 A 在销毁 Obj_1
的过程中被挂起,而这时候 B 线程开始执行销毁操作,而且正常的将 …</p></div><p>注: 本篇 Blog 是参照
<a class="reference external" href="http://book.douban.com/subject/3117898/">Python源码剖析</a>过的 Python 源
码,内容算是读书笔记吧,中间还不小心翻错了分支,翻到了3.4版本,不过看起来3.4的 GIL
处理要比2.7的优雅一些.总之,感谢陈孺大神</p>
<div class="section" id="gil">
<h2>什么是 GIL</h2>
<p>GIL(Global Interpreter Lock)
是解释器全局锁,用来互斥线程对于Python虚拟机的使用.</p>
</div>
<div class="section" id="id1">
<h2>为什么用 GIL</h2>
<p>我们知道 Python
的线程调度是由机器来调度的,在线程执行的时候,我们不能决定线程什么时候挂起.假设线程
A与线程 B 都保存在对象
Obj_1,而这种情况下有可能会发生一些比较坑的问题:比如 A 在销毁 Obj_1
的过程中被挂起,而这时候 B 线程开始执行销毁操作,而且正常的将
Obj_1的销毁,内存被回收.这时候B 被挂起,A 接着被挂起之前的状态执行.然后
Obj_1已经木有了...类似与这种坑爹的问题,没有 GIL 的情况下会经常发生.</p>
<p>为了避免上述情况发生,我们就需要在解决多线程访问共享资源的互斥.</p>
</div>
<div class="section" id="id2">
<h2>创建 GIL</h2>
<p>要了解在之前的Blog 中,大概的讲过 Python
中的线程调度是由操作系统来调度的.而 GIL
作为多线程操作的产物,要深入了解也必须要从线程<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Modules/threadmodule.c#L850">Thread</a>模块入手.</p>
<div class="highlight"><pre><span></span><span class="k">static</span> <span class="n">PyMethodDef</span> <span class="n">thread_methods</span><span class="p">[]</span> <span class="o">=</span> <span class="p">{</span>
<span class="p">{</span><span class="s">"start_new_thread"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_PyThread_start_new_thread</span><span class="p">,</span>
<span class="n">METH_VARARGS</span><span class="p">,</span>
<span class="n">start_new_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"start_new"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_PyThread_start_new_thread</span><span class="p">,</span>
<span class="n">METH_VARARGS</span><span class="p">,</span>
<span class="n">start_new_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"allocate_lock"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_PyThread_allocate_lock</span><span class="p">,</span>
<span class="n">METH_NOARGS</span><span class="p">,</span> <span class="n">allocate_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"allocate"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_PyThread_allocate_lock</span><span class="p">,</span>
<span class="n">METH_NOARGS</span><span class="p">,</span> <span class="n">allocate_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"exit_thread"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_PyThread_exit_thread</span><span class="p">,</span>
<span class="n">METH_NOARGS</span><span class="p">,</span> <span class="n">exit_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"exit"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_PyThread_exit_thread</span><span class="p">,</span>
<span class="n">METH_NOARGS</span><span class="p">,</span> <span class="n">exit_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"interrupt_main"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_PyThread_interrupt_main</span><span class="p">,</span>
<span class="n">METH_NOARGS</span><span class="p">,</span> <span class="n">interrupt_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"get_ident"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_get_ident</span><span class="p">,</span>
<span class="n">METH_NOARGS</span><span class="p">,</span> <span class="n">get_ident_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"_count"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread__count</span><span class="p">,</span>
<span class="n">METH_NOARGS</span><span class="p">,</span> <span class="n">_count_doc</span><span class="p">},</span>
<span class="p">{</span><span class="s">"stack_size"</span><span class="p">,</span> <span class="p">(</span><span class="n">PyCFunction</span><span class="p">)</span><span class="n">thread_stack_size</span><span class="p">,</span>
<span class="n">METH_VARARGS</span><span class="p">,</span>
<span class="n">stack_size_doc</span><span class="p">},</span>
<span class="p">{</span><span class="nb">NULL</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">}</span> <span class="cm">/* sentinel */</span>
<span class="p">};</span>
</pre></div>
<p>从 Thread module我们可以看到Python
给我们提供的多线程机制的接口.非常简单,极其精简,类似于<tt class="docutils literal">start_new_thread</tt>跟<tt class="docutils literal">start_new</tt>都是同一个接口.</p>
<p>从上面我们找到创建进程的接口,然后跟踪到<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Modules/threadmodule.c#L648">thread_PyThread_start_new_thread</a>里面,官方的<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Modules/threadmodule.c#L687">注释</a>写的也算到位.</p>
<div class="highlight"><pre><span></span><span class="k">static</span> <span class="n">PyObject</span> <span class="o">*</span>
<span class="nf">thread_PyThread_start_new_thread</span><span class="p">(</span><span class="n">PyObject</span> <span class="o">*</span><span class="n">self</span><span class="p">,</span> <span class="n">PyObject</span> <span class="o">*</span><span class="n">fargs</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">PyObject</span> <span class="o">*</span><span class="n">func</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">*</span><span class="n">keyw</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="k">struct</span> <span class="n">bootstate</span> <span class="o">*</span><span class="n">boot</span><span class="p">;</span>
<span class="kt">long</span> <span class="n">ident</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">PyArg_UnpackTuple</span><span class="p">(</span><span class="n">fargs</span><span class="p">,</span> <span class="s">"start_new_thread"</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span>
<span class="o">&</span><span class="n">func</span><span class="p">,</span> <span class="o">&</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="n">keyw</span><span class="p">))</span>
<span class="k">return</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">PyCallable_Check</span><span class="p">(</span><span class="n">func</span><span class="p">))</span> <span class="p">{</span>
<span class="n">PyErr_SetString</span><span class="p">(</span><span class="n">PyExc_TypeError</span><span class="p">,</span>
<span class="s">"first arg must be callable"</span><span class="p">);</span>
<span class="k">return</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">PyTuple_Check</span><span class="p">(</span><span class="n">args</span><span class="p">))</span> <span class="p">{</span>
<span class="n">PyErr_SetString</span><span class="p">(</span><span class="n">PyExc_TypeError</span><span class="p">,</span>
<span class="s">"2nd arg must be a tuple"</span><span class="p">);</span>
<span class="k">return</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="n">keyw</span> <span class="o">!=</span> <span class="nb">NULL</span> <span class="o">&&</span> <span class="o">!</span><span class="n">PyDict_Check</span><span class="p">(</span><span class="n">keyw</span><span class="p">))</span> <span class="p">{</span>
<span class="n">PyErr_SetString</span><span class="p">(</span><span class="n">PyExc_TypeError</span><span class="p">,</span>
<span class="s">"optional 3rd arg must be a dictionary"</span><span class="p">);</span>
<span class="k">return</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">boot</span> <span class="o">=</span> <span class="n">PyMem_NEW</span><span class="p">(</span><span class="k">struct</span> <span class="n">bootstate</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">boot</span> <span class="o">==</span> <span class="nb">NULL</span><span class="p">)</span>
<span class="k">return</span> <span class="n">PyErr_NoMemory</span><span class="p">();</span>
<span class="n">boot</span><span class="o">-></span><span class="n">interp</span> <span class="o">=</span> <span class="n">PyThreadState_GET</span><span class="p">()</span><span class="o">-></span><span class="n">interp</span><span class="p">;</span>
<span class="n">boot</span><span class="o">-></span><span class="n">func</span> <span class="o">=</span> <span class="n">func</span><span class="p">;</span>
<span class="n">boot</span><span class="o">-></span><span class="n">args</span> <span class="o">=</span> <span class="n">args</span><span class="p">;</span>
<span class="n">boot</span><span class="o">-></span><span class="n">keyw</span> <span class="o">=</span> <span class="n">keyw</span><span class="p">;</span>
<span class="n">boot</span><span class="o">-></span><span class="n">tstate</span> <span class="o">=</span> <span class="n">_PyThreadState_Prealloc</span><span class="p">(</span><span class="n">boot</span><span class="o">-></span><span class="n">interp</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">boot</span><span class="o">-></span><span class="n">tstate</span> <span class="o">==</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">PyMem_DEL</span><span class="p">(</span><span class="n">boot</span><span class="p">);</span>
<span class="k">return</span> <span class="n">PyErr_NoMemory</span><span class="p">();</span>
<span class="p">}</span>
<span class="n">Py_INCREF</span><span class="p">(</span><span class="n">func</span><span class="p">);</span>
<span class="n">Py_INCREF</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="n">Py_XINCREF</span><span class="p">(</span><span class="n">keyw</span><span class="p">);</span>
<span class="n">PyEval_InitThreads</span><span class="p">();</span> <span class="cm">/* Start the interpreter's thread-awareness */</span>
<span class="n">ident</span> <span class="o">=</span> <span class="n">PyThread_start_new_thread</span><span class="p">(</span><span class="n">t_bootstrap</span><span class="p">,</span> <span class="p">(</span><span class="kt">void</span><span class="o">*</span><span class="p">)</span> <span class="n">boot</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">ident</span> <span class="o">==</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="p">{</span>
<span class="n">PyErr_SetString</span><span class="p">(</span><span class="n">ThreadError</span><span class="p">,</span> <span class="s">"can't start new thread"</span><span class="p">);</span>
<span class="n">Py_DECREF</span><span class="p">(</span><span class="n">func</span><span class="p">);</span>
<span class="n">Py_DECREF</span><span class="p">(</span><span class="n">args</span><span class="p">);</span>
<span class="n">Py_XDECREF</span><span class="p">(</span><span class="n">keyw</span><span class="p">);</span>
<span class="n">PyThreadState_Clear</span><span class="p">(</span><span class="n">boot</span><span class="o">-></span><span class="n">tstate</span><span class="p">);</span>
<span class="n">PyMem_DEL</span><span class="p">(</span><span class="n">boot</span><span class="p">);</span>
<span class="k">return</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">PyInt_FromLong</span><span class="p">(</span><span class="n">ident</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
<p><tt class="docutils literal">Start the interpreter's <span class="pre">thread-awareness</span></tt>,让解释器开始准备多线程环境,其实就是初始化多线程环境.这里有一些需要注意的地方,Python
在最开始执行的时候,是<strong>没有创建多线程的数据结构的</strong>,也没有创建
GIL.这样可以避免一些只需要单线程的程序做多余的线程调度.只有当我们执行<tt class="docutils literal">start_new_thread</tt>的时候,才会激活多线程机制,创建
GIL.</p>
<p>我们跟踪<tt class="docutils literal">PyEval_InitThreads()</tt>到<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Python/ceval.c#L249">ceval.c</a>,可以看到创建
GIL 的代码:</p>
<div class="highlight"><pre><span></span><span class="k">static</span> <span class="n">PyThread_type_lock</span> <span class="n">interpreter_lock</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="cm">/* This is the GIL */</span>
<span class="n">PyEval_InitThreads</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">interpreter_lock</span><span class="p">)</span>
<span class="k">return</span><span class="p">;</span>
<span class="n">interpreter_lock</span> <span class="o">=</span> <span class="n">PyThread_allocate_lock</span><span class="p">();</span>
<span class="n">PyThread_acquire_lock</span><span class="p">(</span><span class="n">interpreter_lock</span><span class="p">,</span> <span class="mi">1</span><span class="p">);</span>
<span class="n">main_thread</span> <span class="o">=</span> <span class="n">PyThread_get_thread_ident</span><span class="p">();</span>
<span class="p">}</span>
</pre></div>
<p>从上面的代码中我们可以看到,在初始化多线程环境的时候,会检测
<tt class="docutils literal">interpreter_lock</tt>
是不是已经创建,如果没有创建,则会用<tt class="docutils literal">PyThread_allocate_lock</tt>
创建<tt class="docutils literal">interpreter_lock</tt>.</p>
</div>
<div class="section" id="id3">
<h2>什么是 GIL</h2>
<p>上面我们跟踪到 GIL 的创建过程,可是 GIL 到底是个什么东西?</p>
<p>从前面的代码中,我们看到是由<tt class="docutils literal">PyThread_allocate_lock</tt>来创建GIL
的,而<tt class="docutils literal">PyThread_allocate_lock</tt>则是针对各个平台来做的具体实现,这里我们看<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Python/thread_pthread.h#L360">Posix标准</a>的实现:</p>
<div class="highlight"><pre><span></span><span class="n">PyThread_allocate_lock</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">pthread_lock</span> <span class="o">*</span><span class="n">lock</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">status</span><span class="p">,</span> <span class="n">error</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="n">dprintf</span><span class="p">((</span><span class="s">"PyThread_allocate_lock called</span><span class="se">\n</span><span class="s">"</span><span class="p">));</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">initialized</span><span class="p">)</span>
<span class="n">PyThread_init_thread</span><span class="p">();</span>
<span class="n">lock</span> <span class="o">=</span> <span class="p">(</span><span class="n">pthread_lock</span> <span class="o">*</span><span class="p">)</span> <span class="n">malloc</span><span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="n">pthread_lock</span><span class="p">));</span>
<span class="k">if</span> <span class="p">(</span><span class="n">lock</span><span class="p">)</span> <span class="p">{</span>
<span class="n">memset</span><span class="p">((</span><span class="kt">void</span> <span class="o">*</span><span class="p">)</span><span class="n">lock</span><span class="p">,</span> <span class="sc">'\0'</span><span class="p">,</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">pthread_lock</span><span class="p">));</span>
<span class="n">lock</span><span class="o">-></span><span class="n">locked</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="n">status</span> <span class="o">=</span> <span class="n">pthread_mutex_init</span><span class="p">(</span><span class="o">&</span><span class="n">lock</span><span class="o">-></span><span class="n">mut</span><span class="p">,</span>
<span class="n">pthread_mutexattr_default</span><span class="p">);</span>
<span class="n">CHECK_STATUS</span><span class="p">(</span><span class="s">"pthread_mutex_init"</span><span class="p">);</span>
<span class="n">status</span> <span class="o">=</span> <span class="n">pthread_cond_init</span><span class="p">(</span><span class="o">&</span><span class="n">lock</span><span class="o">-></span><span class="n">lock_released</span><span class="p">,</span>
<span class="n">pthread_condattr_default</span><span class="p">);</span>
<span class="n">CHECK_STATUS</span><span class="p">(</span><span class="s">"pthread_cond_init"</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">error</span><span class="p">)</span> <span class="p">{</span>
<span class="n">free</span><span class="p">((</span><span class="kt">void</span> <span class="o">*</span><span class="p">)</span><span class="n">lock</span><span class="p">);</span>
<span class="n">lock</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">dprintf</span><span class="p">((</span><span class="s">"PyThread_allocate_lock() -> %p</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">lock</span><span class="p">));</span>
<span class="k">return</span> <span class="p">(</span><span class="n">PyThread_type_lock</span><span class="p">)</span> <span class="n">lock</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<p>先检测是否已经初始化,如果没有,则进行初始化.中间加上 malloc
机制,最后返回一个<tt class="docutils literal">pthread_lock</tt>,这就是我们的 GIL
了,<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Python/thread_pthread.h#L113">线程互斥的锁</a>:</p>
<div class="highlight"><pre><span></span><span class="k">typedef</span> <span class="k">struct</span> <span class="p">{</span>
<span class="kt">char</span> <span class="n">locked</span><span class="p">;</span> <span class="cm">/* 0=unlocked, 1=locked */</span>
<span class="cm">/* a <cond, mutex> pair to handle an acquire of a locked lock */</span>
<span class="n">pthread_cond_t</span> <span class="n">lock_released</span><span class="p">;</span>
<span class="n">pthread_mutex_t</span> <span class="n">mut</span><span class="p">;</span>
<span class="p">}</span> <span class="n">pthread_lock</span><span class="p">;</span>
</pre></div>
</div>
<div class="section" id="id4">
<h2>什么时候释放 GIL</h2>
<p>总算回到最初我写这篇 Blog
的动机上面来了,我是在去豆瓣面试的时候被问了这个问题,一时语塞,瞎扯一通之后回来决定要好好看一下代码的...
现在都已经快准备入职豆瓣了,才来动手写 Blog,也算是拖延症晚期吧.</p>
<p>在通过 <tt class="docutils literal">PyThread_allocate_lock</tt> 创建 GIL
之后,多线程的开始正常的调度,包括<tt class="docutils literal">sys.getcheckinterval()</tt>
拿到的<tt class="docutils literal">interval</tt>(默认为100)的间隔被动放弃 GIL,或者线程阻塞放弃
GIL.总之,`PyEval_InitThreads
会通过<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Python/ceval.c#L254">PyThread_acquire_lock</a>
来获取 GIL.</p>
</div>
Python 元类2014-12-29T00:00:00+08:002014-12-29T00:00:00+08:00Sunisdowntag:sunisdown.me,2014-12-29:/python-yuan-lei.html<p><tt class="docutils literal">Python</tt>中有三个概念比较难懂,分别是:</p>
<ul class="simple">
<li>装饰器</li>
<li>描述符</li>
<li>元类</li>
</ul>
<p>这三种概念里面,元类又最为难懂,威力最强。</p>
<div class="section" id="id1">
<h2>类型,类,对象</h2>
<p>神说<tt class="docutils literal">万物皆对象</tt>,Python
中一切都是对象,凡是对象必先有类型。类,也是对象的一种。Python
中所有的对象都有一些相同的内容,包含在对象的<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Include/object.h#L106">头部信息</a>中。</p>
<div class="highlight"><pre><span></span><span class="k">typedef</span> <span class="k">struct</span> <span class="n">_object</span> <span class="p">{</span>
<span class="n">PyObject_HEAD</span>
<span class="p">}</span> <span class="n">PyObject</span><span class="p">;</span>
</pre></div>
<p>在
<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Include/object.h#L78">PyObject</a>
的定义中,<tt class="docutils literal">ob_refcnt</tt>
是<tt class="docutils literal">应用计数</tt>,用来做内存管理垃圾回收有关,这里暂时不细说。有兴趣的可以查看<a class="reference external" href="https://github.com/qyuhen/book">Q.yuhen的
Python 笔记</a>,</p>
<div class="highlight"><pre><span></span><span class="cm">/* PyObject_HEAD defines the initial segment of every PyObject …</span></pre></div></div><p><tt class="docutils literal">Python</tt>中有三个概念比较难懂,分别是:</p>
<ul class="simple">
<li>装饰器</li>
<li>描述符</li>
<li>元类</li>
</ul>
<p>这三种概念里面,元类又最为难懂,威力最强。</p>
<div class="section" id="id1">
<h2>类型,类,对象</h2>
<p>神说<tt class="docutils literal">万物皆对象</tt>,Python
中一切都是对象,凡是对象必先有类型。类,也是对象的一种。Python
中所有的对象都有一些相同的内容,包含在对象的<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Include/object.h#L106">头部信息</a>中。</p>
<div class="highlight"><pre><span></span><span class="k">typedef</span> <span class="k">struct</span> <span class="n">_object</span> <span class="p">{</span>
<span class="n">PyObject_HEAD</span>
<span class="p">}</span> <span class="n">PyObject</span><span class="p">;</span>
</pre></div>
<p>在
<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Include/object.h#L78">PyObject</a>
的定义中,<tt class="docutils literal">ob_refcnt</tt>
是<tt class="docutils literal">应用计数</tt>,用来做内存管理垃圾回收有关,这里暂时不细说。有兴趣的可以查看<a class="reference external" href="https://github.com/qyuhen/book">Q.yuhen的
Python 笔记</a>,</p>
<div class="highlight"><pre><span></span><span class="cm">/* PyObject_HEAD defines the initial segment of every PyObject. */</span>
<span class="cp">#define PyObject_HEAD \</span>
<span class="cp"> _PyObject_HEAD_EXTRA \</span>
<span class="cp"> Py_ssize_t ob_refcnt; \</span>
<span class="cp"> struct _typeobject *ob_type;</span>
</pre></div>
<p><tt class="docutils literal">ob_type</tt>则是指向指向具体类型的指针。里面定义各种 Python
的类型。每一个对象都有他的类型。</p>
<p>下面跟我念:Python
中万物皆对象,对象皆有类型,且类型也是一个对象。上面我们说了每一个对象都有一个<tt class="docutils literal">类型指针</tt>指向他的类型,我们可以通过类型指针来判断对象的类型。</p>
<p>那么问题来了:我们通过什么来确定一个对象其实是一个类型对象?答案就是
<tt class="docutils literal">metaclass</tt>,所有 class 的 class,所有类型的类型。</p>
<p>举个栗子:</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="k">class</span> <span class="nc">NewData</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="o">...</span><span class="p">:</span> <span class="k">pass</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="n">newdata</span> <span class="o">=</span> <span class="n">NewData</span><span class="p">()</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="n">NewData</span><span class="o">.</span><span class="vm">__class__</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="nb">type</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="nb">type</span><span class="p">(</span><span class="n">NewData</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="nb">type</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">5</span><span class="p">]:</span> <span class="n">newdata</span><span class="o">.</span><span class="vm">__class__</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">5</span><span class="p">]:</span> <span class="n">__main__</span><span class="o">.</span><span class="n">NewData</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">6</span><span class="p">]:</span> <span class="nb">type</span><span class="p">(</span><span class="n">newdata</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">6</span><span class="p">]:</span> <span class="n">__main__</span><span class="o">.</span><span class="n">NewData</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">7</span><span class="p">]:</span> <span class="nb">type</span><span class="o">.</span><span class="vm">__class__</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">7</span><span class="p">]:</span> <span class="nb">type</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="nb">type</span><span class="o">.</span><span class="n">__base__</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="nb">object</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="n">NewData</span><span class="o">.</span><span class="n">__base__</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="nb">object</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="nb">type</span><span class="p">(</span><span class="nb">object</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="nb">type</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="nb">object</span><span class="o">.</span><span class="vm">__class__</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="nb">type</span>
</pre></div>
<p>从上面的栗子中我们可以观察到,<tt class="docutils literal">NewData</tt>与<tt class="docutils literal">type</tt>以及<tt class="docutils literal">object</tt>的类型都是
<em>type</em>,这 里面的类型 <em>type</em> 就是
<tt class="docutils literal">metaclass</tt>的一种,现在理一下元类,类型,类,实例之间的关 系:</p>
<ul class="simple">
<li>实例的类型是类 newdata.<strong>class</strong> == class</li>
<li>类的类型是元类 NewData.<strong>class</strong> == metaclass #type</li>
</ul>
</div>
<div class="section" id="metaclass">
<h2><strong>metaclass</strong></h2>
<p>Python 中 <tt class="docutils literal">class</tt> 关键字会
<a class="reference external" href="https://github.com/python/cpython/blob/2.7/Python/ceval.c#L4621">创建类的对象</a>,
我们观察一下类的创建过程:</p>
<div class="highlight"><pre><span></span><span class="k">if</span> <span class="p">(</span><span class="n">PyDict_Check</span><span class="p">(</span><span class="n">methods</span><span class="p">))</span>
<span class="n">metaclass</span> <span class="o">=</span> <span class="n">PyDict_GetItemString</span><span class="p">(</span><span class="n">methods</span><span class="p">,</span> <span class="s">"__metaclass__"</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">metaclass</span> <span class="o">!=</span> <span class="nb">NULL</span><span class="p">)</span>
<span class="n">Py_INCREF</span><span class="p">(</span><span class="n">metaclass</span><span class="p">);</span>
<span class="k">else</span> <span class="nf">if</span> <span class="p">(</span><span class="n">PyTuple_Check</span><span class="p">(</span><span class="n">bases</span><span class="p">)</span> <span class="o">&&</span> <span class="n">PyTuple_GET_SIZE</span><span class="p">(</span><span class="n">bases</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="n">base</span> <span class="o">=</span> <span class="n">PyTuple_GET_ITEM</span><span class="p">(</span><span class="n">bases</span><span class="p">,</span> <span class="mi">0</span><span class="p">);</span>
<span class="n">metaclass</span> <span class="o">=</span> <span class="n">PyObject_GetAttrString</span><span class="p">(</span><span class="n">base</span><span class="p">,</span> <span class="s">"__class__"</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">metaclass</span> <span class="o">==</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span>
<span class="n">PyErr_Clear</span><span class="p">();</span>
<span class="n">metaclass</span> <span class="o">=</span> <span class="p">(</span><span class="n">PyObject</span> <span class="o">*</span><span class="p">)</span><span class="n">base</span><span class="o">-></span><span class="n">ob_type</span><span class="p">;</span>
<span class="n">Py_INCREF</span><span class="p">(</span><span class="n">metaclass</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>先检测在类的定义中是否指定了<tt class="docutils literal">__metaclass__</tt>,如果没有自己定义,则使用<tt class="docutils literal">object</tt>的
<tt class="docutils literal">__class__</tt>来作为元类,上面演示过<tt class="docutils literal">object.__class__</tt>
是<tt class="docutils literal">type</tt>。</p>
</div>
<div class="section" id="id2">
<h2>如何使用元类</h2>
<p>上面一节讲到,如果自己没有定义<tt class="docutils literal">__metaclass__</tt>的时候,则会使用默认的元类<tt class="docutils literal">type</tt>。
而这一节则会实验如何自己创建一个自定义元类。假设我是一个非常自恋的码农,别人把我
的名字从 Auth
里面抹去这种事儿不能忍,这种情况下也可以蛋疼的用元类(这真的是一个
蛋疼的栗子):</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">AuthMeta</span><span class="p">(</span><span class="nb">type</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
<span class="n">t</span> <span class="o">=</span> <span class="nb">type</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">bases</span><span class="p">,</span> <span class="n">attrs</span><span class="p">)</span>
<span class="n">t</span><span class="o">.</span><span class="n">auth</span> <span class="o">=</span> <span class="s2">"SunisDown"</span>
<span class="k">return</span> <span class="n">t</span>
<span class="k">class</span> <span class="nc">Blog</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="vm">__metaclass__</span> <span class="o">=</span> <span class="n">AuthMeta</span>
<span class="k">pass</span>
<span class="n">aaa</span> <span class="o">=</span> <span class="n">Blog</span><span class="p">()</span>
<span class="n">aaa</span><span class="o">.</span><span class="n">auth</span> <span class="o">==</span> <span class="s1">'SunisDown'</span>
</pre></div>
<p>如上所示,在后续的代码中,只要将__metaclass__ 指向
<tt class="docutils literal">AuthMeta</tt>,后面的类就有了属
性<tt class="docutils literal">auth</tt>,嗯,这个蛋疼的作者名字跟代码永远的绑到一起了。</p>
<p><tt class="docutils literal">import this</tt>的作者 Tim Peters
说过描述元类的话,能够非常到位的描述出元类在 Python 中的超然位置:</p>
<pre class="literal-block">
99%的用户不需要为元类这种黑魔法过渡操心.如果你想搞清楚究竟是否需要用到元类,那么你就不需要它。那些实际用到元类的人都非常 清楚地知道他们需要做什么,而且根本不需要解释为什么要用元类。
</pre>
<p>在 Django 的
<a class="reference external" href="https://github.com/django/django/blob/master/django/db/models/base.py#L60">models</a>
中,对于元类的使用可以算是一次教科书式的使用。</p>
<p>通过继承<tt class="docutils literal">models.Models</tt>里面的元类,我们就可以直接写类似:</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Blog</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">title</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">50</span><span class="p">)</span>
<span class="n">auth</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">IntegerField</span><span class="p">()</span>
</pre></div>
<p>这种简单方便的
API,用户可以直接使用,而后面负责的逻辑,就隐藏在元类之中。</p>
</div>
如何利用决策树来决定去不去年会2014-11-06T00:00:00+08:002014-11-06T00:00:00+08:00SunisDowntag:sunisdown.me,2014-11-06:/ru-he-li-yong-jue-ce-shu-lai-jue-ding-qu-bu-qu-nian-hui.html<p>又到一年年会时,吸引大家去年会的,要么是冲奖品去,要么是为了看表演的妹子,作为一个好青年我排除掉看妹子的选项,仅用奖品来做价值来决定<strong>到底年会值不值得去!!!</strong></p>
<p>如果我们去年会的话,有可能得到一个
iMac,如果我们不去,那我们可能什么都得不到.但是
如果去年会,我们需要花路费,假设我们需要坐一次公交¥1,然后再坐一次地铁¥3,现在帝都公交地
铁都涨价了,出门的成本也增加了T
T.有的小伙伴可能会开车,这样路费比公共交通更贵.反正我没有车,一天的来回路费按¥8计算.</p>
<p>通过上面的简单计算,如果我们去年会,则肯定需要支出¥8的成本,但是有可能会收获一台价
值¥8000的 iMac,看起来不错,即使没有抽到
iMac,其他的奖品也还是不错的~但是前提是你 人品要好.这是一个概率事件.</p>
<table border="1" class="docutils">
<colgroup>
<col width="25%" />
<col width="26%" />
<col width="26%" />
<col width="23%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">抽奖顺序</th>
<th class="head">奖品级别</th>
<th class="head">价值¥</th>
<th class="head">人数</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>1</td>
<td>4等</td>
<td>200 …</td></tr></tbody></table><p>又到一年年会时,吸引大家去年会的,要么是冲奖品去,要么是为了看表演的妹子,作为一个好青年我排除掉看妹子的选项,仅用奖品来做价值来决定<strong>到底年会值不值得去!!!</strong></p>
<p>如果我们去年会的话,有可能得到一个
iMac,如果我们不去,那我们可能什么都得不到.但是
如果去年会,我们需要花路费,假设我们需要坐一次公交¥1,然后再坐一次地铁¥3,现在帝都公交地
铁都涨价了,出门的成本也增加了T
T.有的小伙伴可能会开车,这样路费比公共交通更贵.反正我没有车,一天的来回路费按¥8计算.</p>
<p>通过上面的简单计算,如果我们去年会,则肯定需要支出¥8的成本,但是有可能会收获一台价
值¥8000的 iMac,看起来不错,即使没有抽到
iMac,其他的奖品也还是不错的~但是前提是你 人品要好.这是一个概率事件.</p>
<table border="1" class="docutils">
<colgroup>
<col width="25%" />
<col width="26%" />
<col width="26%" />
<col width="23%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">抽奖顺序</th>
<th class="head">奖品级别</th>
<th class="head">价值¥</th>
<th class="head">人数</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>1</td>
<td>4等</td>
<td>200</td>
<td>40</td>
</tr>
<tr><td>2</td>
<td>3等</td>
<td>300</td>
<td>30</td>
</tr>
<tr><td>3</td>
<td>2等</td>
<td>400</td>
<td>20</td>
</tr>
<tr><td>4</td>
<td>1等</td>
<td>800</td>
<td>10</td>
</tr>
<tr><td>5</td>
<td>特等</td>
<td>8000</td>
<td>1</td>
</tr>
<tr><td>5</td>
<td>特别幸运</td>
<td>8000</td>
<td>1</td>
</tr>
</tbody>
</table>
<p>当你对于一件事犹豫不决不知道该不该做的时候,决策树是帮你的好工具,下面我们利用决策树来帮助我们做一下决策.</p>
<p>首先我们开始画一个决策树: 去或者不去,</p>
<div class="figure">
<img alt="图片1" src="images/dt1.png" />
<p class="caption">图片1</p>
</div>
<p>如果我们去了,开始了抽奖环节,估计大家都跟我一样,在开始的时候祈祷:不要不要不要抽
我,因为四等奖太小了.四等奖价值¥200,有40个四等奖,全公司有1200人,这样我们可以得出
抽中4等奖的概率为40/1200,然后¥200减去我们的成本¥8,预计收益为¥192.</p>
<pre class="literal-block">
200-8
=192
</pre>
<p>决策树如下:</p>
<div class="figure">
<img alt="图2" src="images/dt2.png" />
<p class="caption">图2</p>
</div>
<p>如果没有抽中4等奖,后面还有机会竞争3等奖,由于前面有小伙伴得到了4等奖,所以我们获得3等奖的概率会增加,因为抽奖的时候会排除他们,重新计算,</p>
<pre class="literal-block">
300 - 8
= 292
</pre>
<p>得到如下决策树:</p>
<div class="figure">
<img alt="图3" src="images/dt3.png" />
<p class="caption">图3</p>
</div>
<p>根据上面的计算方法,我们可以得到完整的决策树:</p>
<div class="figure">
<img alt="图4" src="images/dt4.png" />
<p class="caption">图4</p>
</div>
<p>当然,通常情况下为了尽兴,或者老板心情好,决定给一个特别将,嗯,再加一个特别奖:</p>
<div class="figure">
<img alt="图5" src="images/dt5.png" />
<p class="caption">图5</p>
</div>
<p>上面的图中,我已经在决策树中分析了各种可能以及概率,下面我们要从末端反推决策树,确认每个分支的价值,然后做出判断:
特别奖的分支我们可以做出如下判断:</p>
<pre class="literal-block">
1/1099*7992 + (1098/1099)*(-8)
=-0.72
</pre>
<p>然后根据上面的计算简化决策树:</p>
<div class="figure">
<img alt="图6" src="images/dt6.png" />
<p class="caption">图6</p>
</div>
<p>根据上面的简化方法一直计算,最终得到最简化的树:</p>
<div class="figure">
<img alt="图7" src="images/dt7.png" />
<p class="caption">图7</p>
</div>
<p>嗯,通过最终的简化图,我们可以看到,如果去年会,我们会有¥32.85
的收益,对我们是有好处的.So,还是去吧~</p>
<p>当然,如果是周末开年会的话,计算的时候就需要再加上时间成本,小伙伴们自己来算吧:)</p>
一致性哈希2014-11-06T00:00:00+08:002014-11-06T00:00:00+08:00Sunisdowntag:sunisdown.me,2014-11-06:/yi-zhi-xing-ha-xi.html<p><em>一致性哈希</em>,顾名思义,是一种哈希算法。比较常用的哈希算法应该是取模,通常在数据库需要分库的时候,大家都会用取模哈希。但是取模有一个缺点,就是,就是增加或者删除一个槽位的时候,几乎需要对所有的关键字进行重新映射,而分库的时候用这种方法,则所有的数据都需要重新录入。</p>
<p>在我们做分布式缓存的时候,我们需要把对 A 的请求,映射到某一个节点
node_a,如果是取模来做哈希,假设我们是资源
6,集群中有4个几点,取4的模为,则资源缓存到
node_2。如果我们向集群中添加一台设备,则需要取5的模,资源6会缓存到
node_1。实际上如果用取模来做哈希,无论是增加还是删除节点,都是灾难性的。大部分数据都需要重新映射,集群内的内容也需要重新缓存。这种情况下请求则会击穿缓存,请求到数据库,给数据库带来很大的压力。这种情况下,我们会需要<strong>一致性哈希</strong>。</p>
<p><a class="reference external" href="http://people.csail.mit.edu/karger/">David
Karger</a>及其合作者列出了使得一致哈希在互联网分布式缓存中非常有用的几个特性:</p>
<ul class="simple">
<li>冗余少</li>
<li>负载均衡</li>
<li>过渡平滑</li>
<li>存储均衡</li>
<li>关键词单调</li>
</ul>
<p>一致性哈希最基础的想法是对所有的对象都使用相同的函数来映射缓存。这样可以尽可能的将同一个资源映射到同一个节点上面 …</p><p><em>一致性哈希</em>,顾名思义,是一种哈希算法。比较常用的哈希算法应该是取模,通常在数据库需要分库的时候,大家都会用取模哈希。但是取模有一个缺点,就是,就是增加或者删除一个槽位的时候,几乎需要对所有的关键字进行重新映射,而分库的时候用这种方法,则所有的数据都需要重新录入。</p>
<p>在我们做分布式缓存的时候,我们需要把对 A 的请求,映射到某一个节点
node_a,如果是取模来做哈希,假设我们是资源
6,集群中有4个几点,取4的模为,则资源缓存到
node_2。如果我们向集群中添加一台设备,则需要取5的模,资源6会缓存到
node_1。实际上如果用取模来做哈希,无论是增加还是删除节点,都是灾难性的。大部分数据都需要重新映射,集群内的内容也需要重新缓存。这种情况下请求则会击穿缓存,请求到数据库,给数据库带来很大的压力。这种情况下,我们会需要<strong>一致性哈希</strong>。</p>
<p><a class="reference external" href="http://people.csail.mit.edu/karger/">David
Karger</a>及其合作者列出了使得一致哈希在互联网分布式缓存中非常有用的几个特性:</p>
<ul class="simple">
<li>冗余少</li>
<li>负载均衡</li>
<li>过渡平滑</li>
<li>存储均衡</li>
<li>关键词单调</li>
</ul>
<p>一致性哈希最基础的想法是对所有的对象都使用相同的函数来映射缓存。这样可以尽可能的将同一个资源映射到同一个节点上面。这种情况下,无论是增加还是移出一个节点,都只会影响其相邻的节点,而其他的节点不受影响,这样最大程度的保证了缓存的有效性,将副作用降到了最低。</p>
<div class="section" id="id2">
<h2>详解</h2>
<div class="section" id="id3">
<h3>原理</h3>
<p>一致性哈希梓潼来说非常简单,你可以认为这是一个从0到一个更大的数字构成的环。给你一个任意节点
A,通过一个 hash 函数,你可以将节点 A 放在环的某一个位置。如下图所示:</p>
<img alt="" src="images/hash_ring_1.png" />
<p>然后再给你一个对象<tt class="docutils literal">15</tt>,通过相同的 hash 函数对 <tt class="docutils literal">15</tt>
进行计算,则可以得到在环上得到相应的位置 key 15。</p>
<img alt="" src="images/hash_ring_2.png" />
<p>如上图所示,key 15被缓存到顺时针遇见的第一个节点,即
node20节点上。其他对象
key6,key23,key40则分别缓存到各自遇见的第一个节点,node10,node30,node40。一个节点缓存了自己与上一个节点之间的所有数据。
这种缓存方式下,如果节点20被删除,则 key15呗映射到 node30,如果
node30与与 node40之间增加了一个节点 node35,则
key31,key33都被重新映射到 node35,其他的对象映射不需要改变。</p>
</div>
<div class="section" id="id4">
<h3>虚拟节点</h3>
<p>利用上面的方法可以说基本上已经足够了,但是有一个问题:我们对节点和对象进行哈希运算的时候,如果节点数过少,有可能会出现节点直接不均匀的情况出现。这样可能的大多数对象都映射到同一个节点上,如下图所示:</p>
<img alt="" src="images/hash_ring_4.png" />
<p>这样大部分数据都映射到
node40上,并不能够做到均衡,从而导致数据倾斜。为了解决这个问题,引入了<tt class="docutils literal">虚拟节点</tt>的机制。即新增同一个缓存设备的时候,会对这个设备进行多次哈希计算,从而产生多个节点。用上面的栗子,我们可以对每个节点进行三次哈希计算,环上则有9个节点<tt class="docutils literal">20#1 #2 #3, 10#1 #2 #3, 40#1 #2 #3</tt>:</p>
<img alt="" src="images/hash_ring_5.png" />
<p>在实际的生产环境中,我们可以对同一设备进行更多次哈希,这样数据分布会接近于平均分布。</p>
</div>
</div>
<div class="section" id="python">
<h2>Python 实现</h2>
<p>代码来做与 Python 的 hash_ring库:</p>
<pre class="code python literal-block">
<span class="kn">import</span> <span class="nn">md5</span>
<span class="k">class</span> <span class="nc">HashRing</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">nodes</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">replicas</span><span class="o">=</span><span class="mi">3</span><span class="p">):</span>
<span class="sd">"""Manages a hash ring.
`nodes` is a list of objects that have a proper __str__ representation.
`replicas` indicates how many virtual points should be used pr. node,
replicas are required to improve the distribution.
"""</span>
<span class="bp">self</span><span class="o">.</span><span class="n">replicas</span> <span class="o">=</span> <span class="n">replicas</span>
<span class="bp">self</span><span class="o">.</span><span class="n">ring</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sorted_keys</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">if</span> <span class="n">nodes</span><span class="p">:</span>
<span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">add_node</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">add_node</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">node</span><span class="p">):</span>
<span class="sd">"""Adds a `node` to the hash ring (including a number of replicas).
"""</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">replicas</span><span class="p">):</span>
<span class="n">key</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">gen_key</span><span class="p">(</span><span class="s1">'</span><span class="si">%s</span><span class="s1">:</span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">i</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sorted_keys</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sorted_keys</span><span class="o">.</span><span class="n">sort</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">remove_node</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">node</span><span class="p">):</span>
<span class="sd">"""Removes `node` from the hash ring and its replicas.
"""</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">replicas</span><span class="p">):</span>
<span class="n">key</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">gen_key</span><span class="p">(</span><span class="s1">'</span><span class="si">%s</span><span class="s1">:</span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">i</span><span class="p">))</span>
<span class="k">del</span> <span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sorted_keys</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">get_node</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">string_key</span><span class="p">):</span>
<span class="sd">"""Given a string key a corresponding node in the hash ring is returned.
If the hash ring is empty, `None` is returned.
"""</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_node_pos</span><span class="p">(</span><span class="n">string_key</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">get_node_pos</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">string_key</span><span class="p">):</span>
<span class="sd">"""Given a string key a corresponding node in the hash ring is returned
along with it's position in the ring.
If the hash ring is empty, (`None`, `None`) is returned.
"""</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span><span class="p">,</span> <span class="bp">None</span>
<span class="n">key</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">gen_key</span><span class="p">(</span><span class="n">string_key</span><span class="p">)</span>
<span class="n">nodes</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sorted_keys</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">nodes</span><span class="p">)):</span>
<span class="n">node</span> <span class="o">=</span> <span class="n">nodes</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="k">if</span> <span class="n">key</span> <span class="o"><=</span> <span class="n">node</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">[</span><span class="n">node</span><span class="p">],</span> <span class="n">i</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">[</span><span class="n">nodes</span><span class="p">[</span><span class="mi">0</span><span class="p">]],</span> <span class="mi">0</span>
<span class="k">def</span> <span class="nf">get_nodes</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">string_key</span><span class="p">):</span>
<span class="sd">"""Given a string key it returns the nodes as a generator that can hold the key.
The generator is never ending and iterates through the ring
starting at the correct position.
"""</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">:</span>
<span class="k">yield</span> <span class="bp">None</span><span class="p">,</span> <span class="bp">None</span>
<span class="n">node</span><span class="p">,</span> <span class="n">pos</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_node_pos</span><span class="p">(</span><span class="n">string_key</span><span class="p">)</span>
<span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sorted_keys</span><span class="p">[</span><span class="n">pos</span><span class="p">:]:</span>
<span class="k">yield</span> <span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
<span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sorted_keys</span><span class="p">:</span>
<span class="k">yield</span> <span class="bp">self</span><span class="o">.</span><span class="n">ring</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">gen_key</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="sd">"""Given a string key it returns a long value,
this long value represents a place on the hash ring.
md5 is currently used because it mixes well.
"""</span>
<span class="n">m</span> <span class="o">=</span> <span class="n">md5</span><span class="o">.</span><span class="n">new</span><span class="p">()</span>
<span class="n">m</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">long</span><span class="p">(</span><span class="n">m</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">(),</span> <span class="mi">16</span><span class="p">)</span>
</pre>
</div>
Emacs 速查手册2014-10-24T00:00:00+08:002014-10-24T00:00:00+08:00SunisDowntag:sunisdown.me,2014-10-24:/emacs-su-cha-shou-ce.html<p><em>注:</em></p>
<pre class="literal-block">
本文是从 Emacs 官方文档 的 help-with-tutorial中文版摘抄而来的速查手册,译者名单见文章末尾
</pre>
<p>重要提示:要退出 Emacs,请用 <tt class="docutils literal"><span class="pre">C-x</span> <span class="pre">C-c</span></tt>(两个连续的组合键)。
要退出一个正在运行中的命令,请用 <tt class="docutils literal"><span class="pre">C-g</span></tt>。</p>
<p>以下命令在翻页浏览时相当有用:</p>
<pre class="literal-block">
C-v 向前移动一屏
M-v 向后移动一屏
C-l 重绘屏幕,并将光标所在行置于屏幕的中央
(注意是 CONTROL-L,不是 CONTROL-1)
</pre>
<div class="section" id="basic-cursor-control">
<h2>基本的光标控制(BASIC CURSOR CONTROL)</h2>
<pre class="literal-block">
上一行 C-p
:
:
向左移 C-b .... 目前光标位置 .... 向右移 C-f
:
:
下一行 C-n
C-f 向右移动一个字符
C-b 向左移动一个字符
M-f 向右移动一个词【对中文是移动到下一个标点符号】
M-b …</pre></div><p><em>注:</em></p>
<pre class="literal-block">
本文是从 Emacs 官方文档 的 help-with-tutorial中文版摘抄而来的速查手册,译者名单见文章末尾
</pre>
<p>重要提示:要退出 Emacs,请用 <tt class="docutils literal"><span class="pre">C-x</span> <span class="pre">C-c</span></tt>(两个连续的组合键)。
要退出一个正在运行中的命令,请用 <tt class="docutils literal"><span class="pre">C-g</span></tt>。</p>
<p>以下命令在翻页浏览时相当有用:</p>
<pre class="literal-block">
C-v 向前移动一屏
M-v 向后移动一屏
C-l 重绘屏幕,并将光标所在行置于屏幕的中央
(注意是 CONTROL-L,不是 CONTROL-1)
</pre>
<div class="section" id="basic-cursor-control">
<h2>基本的光标控制(BASIC CURSOR CONTROL)</h2>
<pre class="literal-block">
上一行 C-p
:
:
向左移 C-b .... 目前光标位置 .... 向右移 C-f
:
:
下一行 C-n
C-f 向右移动一个字符
C-b 向左移动一个字符
M-f 向右移动一个词【对中文是移动到下一个标点符号】
M-b 向左移动一个词【对中文是移动到上一个标点符号】
C-n 移动到下一行
C-p 移动到上一行
C-a 移动到行首
C-e 移动到行尾
M-a 移动到句首
M-e 移动到句尾
M-< 移动到文件开始
M-> 移动到文件结束
</pre>
<p><em>注意:</em>在大部分键盘上,小于号(<)需要用上档键(Shift)来输入,所以在这
些键盘上你应该用 Shift 键来输入 M-<,如果不按 Shift 键,你输入的会是
M-comma(META 逗号)。</p>
<p>大部分的 Emacs 命令接受数字参数,并且对于多数命令而言,这些数字参数的
作用是指定命令的重复次数。为一个命令指定数字参数(也就是重复次数)的方
法是:先输入
<tt class="docutils literal"><span class="pre">C-u</span></tt>,然后输入数字作为参数,最后再输入命令。如果你有META (或 EDIT
或 ALT)键,那么还有另一种办法:按住 META 键不放,然后输入数字。 不
过我们还是建议你用
<tt class="docutils literal"><span class="pre">C-u</span></tt>,因为它在任何终端机上都能用。这种数字参数也称为
“前缀参数”,意思是说这个参数是先于使用它的命令而输入的。</p>
<p>举例来说, <tt class="docutils literal"><span class="pre">C-u</span> 8 <span class="pre">C-f</span></tt> 会向前移动 8 个字符。</p>
<p>虽然大部分命令把数字参数解释为其重复次数,但是也有些命令例外,它们将数
字参数另做它用。比如有些命令(我们目前还没学到)仅仅将前缀参数作为一个
标志――只要给出有一个前缀参数,不管其值为何,它都会改变命令的功能。</p>
<p>而<tt class="docutils literal"><span class="pre">C-v</span></tt> 和 <tt class="docutils literal"><span class="pre">M-v</span></tt>
则属于另一种类型的例外。当给定一个参数时,它们将滚动你指
定的“行数”,而不是“屏数”。举例来说,<tt class="docutils literal"><span class="pre">C-u</span> 8 <span class="pre">C-v</span></tt>将文本向下滚动 8
行。</p>
<p>这个命令应该已经将文字向上滚动了 8 行。如果你想将它再次地向下滚动,你
可以给定一个参数然后执行 <tt class="docutils literal"><span class="pre">M-v</span></tt>。</p>
<p>如果你正在使用图形界面,比如 X 或者微软的 Windows,那么在 Emacs窗
口的一边应该有一个长方形的区域叫“滚动条”。你可以用鼠标操纵滚动条来滚动
文字。</p>
<p>如果你的鼠标有滚轮的话,你也可以使用滚轮来滚动。</p>
</div>
<div class="section" id="inserting-and-deleting">
<h2>插入与删除(INSERTING AND DELETING)</h2>
<pre class="literal-block">
<DEL> 删除光标前的一个字符
C-d 删除光标后的一个字符
M-<DEL> 移除光标前的一个词
M-d 移除光标后的一个词
C-k 移除从光标到“行尾”间的字符
M-k 移除从光标到“句尾”间的字符
C-y 粘贴上一次(有且只有一次)
M-y 粘贴(一个循环,会往前递归之前移除的内容)
</pre>
</div>
<div class="section" id="undo">
<h2>撤销(UNDO)</h2>
<pre class="literal-block">
C-/
C-_
C-x u ##本人测试没有效果
</pre>
<p>如果你修改了一段文字,又觉得改得不好,可以用 undo 命令进行撤销:C-/。</p>
<p>通常 C-/ 会消除一个命令所造成的所有改变;如果你在一行中连续多次地使用
C-/,你会把以前的命令也依次撤销。</p>
<p>但是有两个例外: 1)
没有改变文字的命令不算(包括光标移动命令和滚动命令) 2)
从键盘输入的字符以组为单位――每组最多 20 个字符――来进行处理。
(这是为了减少你在撤销“插入文字”动作时需要输入 C-/ 的次数)</p>
<p>C-_ 也是撤销命令;它的作用跟 C-/ 一样,但是它比较容易多次输入。在
某些终端上,输入 C-/ 实际上向 Emacs 发送的是 C-_ 。 另外, C-x u 和 C-/
完全一样,但是按起来有些麻烦。</p>
<p>数字参数对于 C-/ 、 C-_ 和 C-x u 的意义是执行撤销的重复次数。</p>
</div>
<div class="section" id="searching">
<h2>搜索(SEARCHING)</h2>
<pre class="literal-block">
C-s 向后搜索
C-r 向前搜索
</pre>
</div>
<div class="section" id="emacs-if-emacs-stops-responding">
<h2>如果 EMACS 失去响应(IF EMACS STOPS RESPONDING)</h2>
<p>如果 Emacs 对你的命令失去响应,你可以用 C-g 来安全地终止这条命令。C-g
也可以终止一条执行过久的命令。</p>
<p>C-g 还可以取消数字参数和只输入到一半的命令。</p>
<p>如果你不小心按了一下 ,你也可以用 C-g 来取消它。
【这个说法似乎有问题,因为按照这个按键顺序输入的应该是 C-M-g。 取消
的正确做法是再连按两次 。】</p>
</div>
<div class="section" id="disabled-commands">
<h2>被禁用的命令(DISABLED COMMANDS)</h2>
<p>有一些 Emacs 命令被“禁用”了,以避免初学者在不了解其确切功能的情况下误
用而造成麻烦。</p>
<p>如果你用到了一个被禁用的命令,Emacs 会显示一个提示消息,告诉你这个命令
到底是干什么的,询问你是否要继续,并在得到你的肯定之后再执行这命令。</p>
<p>如果你真的想用这条命令,在 Emacs 询问你的时候应该按空格。一般来说,如 果
你不想用,就按“n”。</p>
<pre class="literal-block">
>> 试试 C-x C-l (这是一个被禁用的命令)
然后用 n 来回答询问。
</pre>
</div>
<div class="section" id="windows">
<h2>窗格(WINDOWS)</h2>
<p>Emacs 可以有多个“窗格”,每个窗格显示不同的文字。后面会介绍怎么对付多个窗
格,现在我们先学会如何关掉多余的窗格。其实也很简单:</p>
<pre class="literal-block">
C-x 1 只保留一个窗格(也就是关掉其它所有窗格)。
</pre>
<p>也就是先按 CONTROL-x 然后再按 1。C-x 1 会保留光标所在的窗格,并将其扩大
到整个屏幕,同时关掉所有其它的窗格。</p>
<p>有一系列命令是以 CONTROL-x 开始的,这些命令许多都跟“窗格、文件、缓冲区
【缓冲区(buffer)会在后文详细介绍】”等等诸如此类的东西有关,其中有些
命令可能包含了 2 个、3 个或者 4 个字符。</p>
</div>
<div class="section" id="file">
<h2>文件(FILE)</h2>
<p>想保存工作成果就要记得存盘,否则一旦退出 Emacs 你编辑的文字就会丢失。
要存盘,就要在编辑前“寻找”到一个存盘文件。(这个过程通常也被称为“访问”
文件。)</p>
<p>寻找到一个文件意味着你可以在 Emacs 里查看这个文件的内容。从许多角度看,
这就等于你在直接编辑这个文件,只是你所做的修改只有在“存盘”的时候才会
被写入文件。也正因为如此,你可以丢弃一个写到一半的文件而不必把这个残缺
文件也保存到计算机上。在存盘的时候,Emacs 会把存盘前的文件重命名保存,
以防你改完之后又想反悔。</p>
<p>寻找文件的命令有一个特点,那就是你必须给出文件名。我们称这个命令“读入
了一个参数”(在这里,这个参数显然就是文件名)。在你输入这条命令之后:</p>
<pre class="literal-block">
C-x C-f 寻找一个文件
</pre>
<p>Emacs 会提示你输入文件名。你输入的文件名会出现在屏幕最底端的一行,这一
行被称为小缓冲(minibuffer),在小缓冲里你可以使用通常的 Emacs 编辑命
令来编辑文件名。</p>
<p>在小缓冲里输入文件名(其实输入其它东西也一样)时可以用 C-g 取消。</p>
<pre class="literal-block">
>> 输入 C-x C-f,然后输入 C-g
这会关掉小缓冲,同时也会取消使用小缓冲的 C-x C-f 命令。
当然了,你也没有找任何文件。
</pre>
<p>用 结束文件名的输入。之后,小缓冲会消失,C-x C-f 将会去寻找你
指定的文件。小缓冲在 C-x C-f 命令结束之后也会消失。</p>
<p>文件被显示在了屏幕上,你可以开始编辑了。存盘用这条命令:</p>
<pre class="literal-block">
C-x C-s 储存这个文件
</pre>
<p>这条命令把 Emacs 中的文字存储到文件中。第一次存盘的时候 Emacs 会将原文
件重命名以备份。重命名的规则通常是在原文件名之后添加一个“~”字符。
【对许多人来说,这是一个烦人的特性,关掉文件备份可以用如下命令: M-x
customize-variable make-backup-files 】</p>
<p>存盘结束后,Emacs 会显示写入文件的文件名。你最好养成经常存盘的习惯,这
可以减少系统崩溃和死机给你带来的损失(也可参见下面的“自动保存”一节)。</p>
<p>你不但可以寻找一个已有的文件来查看或编辑,还可以寻找一个不存在的文件。
实际上这正是 Emacs 创建新文件的方法:找到不存在的新文件。事实上,只有
在存盘的时候,Emacs 才会真正创建这个文件。而在这之后的一切就跟编辑一个
已有文件没有区别了。</p>
</div>
<div class="section" id="buffer">
<h2>缓冲区(BUFFER)</h2>
<p>你可以用 C-x C-f 找到并打开第二个文件,但第一个文件仍然在 Emacs 中。要
切回第一个文件,一种办法是再用一次 C-x C-f。这样,你就可以在 Emacs 中
同时打开多个文件。</p>
<p>Emacs 把每个编辑中的文件都放在一个称为“缓冲区(buffer)”的地方。每寻
找到一个文件,Emacs 就在其内部开辟一个缓冲区。用下面的命令可以列出当前
所有的缓冲区:</p>
<pre class="literal-block">
C-x C-b 列出缓冲区
C-x s 保存多个缓冲区
</pre>
<p>C-x s 会找出所有已被修改但尚未存盘的缓冲区,然后向你逐个询问:是否需要
存盘?</p>
</div>
<div class="section" id="extending-the-command-set">
<h2>命令集扩展(EXTENDING THE COMMAND SET)</h2>
<p>Emacs 的命令就像天上的星星,数也数不清。把它们都对应到 CONTROL 和 META
组合键上显然是不可能的。Emacs 用扩展(eXtend)命令来解决这个问题,扩展
命令有两种风格:</p>
<pre class="literal-block">
C-x 字符扩展。 C-x 之后输入另一个字符或者组合键。
M-x 命令名扩展。M-x 之后输入一个命令名。
</pre>
<p>很多扩展命令都相当有用,虽然与你已经学过的命令比起来,他们可能不那么常
用。我们早已经见过一些扩展命令了,比如用 C-x C-f 寻找文件和用 C-x C-s
保存文件;退出 Emacs 用的 C-x C-c 也是扩展命令。(不用担心退出 Emacs
会给你带来什么损失,Emacs 会在退出之前提醒你存盘的。)</p>
<p>如果你使用图形界面,你不需要任何特殊的命令来切换 Emacs 和其他应用程序。
你可以使用鼠标或者窗口管理器的命令。然而,如果你使用只能同时显示一个应
用程序的文本终端,你需要“挂起” Emacs ,以切换到其他的应用程序。</p>
<p>C-z 可以暂时离开 Emacs――当然,你还可以再回来。在允许 C-z 的系统中,C-z
会把 Emacs“挂起”,也就是说,它会回到 shell但不杀死 Emacs 的进程。在常
用的 shell 中,通常可以用“fg”或者“%emacs”命令再次回到 Emacs 中。</p>
<p>你最好在打算退出登陆的时候再用 C-x C-c。在把 Emacs 当做一个临时的编辑
器的时候(比如被一个邮件处理程序调用),也可以用 C-x C-c 退出。</p>
<p>C-x 的扩展命令有很多,下面列出的是你已经学过的:</p>
<pre class="literal-block">
C-x C-f 寻找文件。
C-x C-s 保存文件。
C-x C-b 列出缓冲区。
C-x C-c 离开 Emacs。
C-x 1 关掉其它所有窗格,只保留一个。
C-x u 撤销。
</pre>
<p>用命令名扩展的命令通常并不常用,或只用在部分模式下。比如
replace-string(字符串替换)这个命令,它会把一个字符串替换成另一个。在
输入 M-x 之后,Emacs 会在屏幕底端向你询问并等待你输入命令名。如果你想
输入“replace-string”,其实只需要敲“repl s”就行了,Emacs 会帮你自
动补齐。输入完之后按 提交。</p>
</div>
<div class="section" id="auto-save">
<h2>自动保存(AUTO SAVE)</h2>
<p>如果你已经修改了一个文件,但是还没来得及存盘你的计算机就罢工了,那么你
所做的修改就很可能会丢失。为了避免这样的不幸发生,Emacs 会定期将正在编
辑的文件写入一个“自动保存”文件中。自动保存文件的文件名的头尾各有一个
“#”字符,比如你正在编辑的文件叫“hello.c”,那么它的自动保存文件就叫
“#hello.c#”。这个文件会在正常存盘之后被 Emacs 删除。</p>
<p>所以,假如不幸真的发生了,你大可以从容地打开原来的文件(注意不是自动保
存文件)然后输入 M-x recover file 来恢复你的自动保存文件。在
提示确认的时候,输入 yes。</p>
</div>
<div class="section" id="echo-area">
<h2>回显区(ECHO AREA)</h2>
<p>如果 Emacs 发现你输入多字符命令的节奏很慢,它会在窗格的下方称为“回显区”
的地方给你提示。回显区位于屏幕的最下面一行。</p>
</div>
<div class="section" id="mode-line">
<h2>状态栏(MODE LINE)</h2>
<p>位于回显区正上方的一行被称为“状态栏”。状态栏上会显示一些信息,比如:</p>
<pre class="literal-block">
-:**- TUTORIAL.cn 63% L749 (Fundamental)
</pre>
<p>状态栏显示了 Emacs 的状态和你正在编辑的文字的一些信息。</p>
<p>你应该知道文件名的意思吧?就是你找到的那个文件嘛。-NN%-- 显示的是光标
在全文中的位置。如果位于文件的开头,那么就显示 --Top-- 而不是 --00%--;
如果位于文件的末尾,就显示
--Bot--。如果文件很小,一屏就足以显示全部内容, 那么状态栏会显示
--All--。</p>
<p>“L” 和其后的数字给出了光标所在行的行号。</p>
<p>最开头的星号(*)表示你已经对文字做过改动。刚刚打开的文件肯定没有被改
动过,所以状态栏上显示的不是星号而是短线(-)。</p>
<p>状态栏上小括号里的内容告诉你当前正在使用的编辑模式。缺省的模式是
Fundamental,就是你现在正在使用的这个。它是一种“主模式”。</p>
<p>Emacs 的主模式林林总总。有用来编辑程序代码的――比如 Lisp 模式;也有用
来编辑各种自然语言文本的――比如 Text 模式。任何情况下只能应用一个主模
式,其名称会显示在状态栏上,也就是现在显示“Fundamental”的地方。</p>
<p>主模式通常会改变一些命令的行为。比方说,不管编辑什么语言的程序代码,你
都可以用一个相同的命令来添加注释。但是在不同的语言中注释的语法往往是不
同的,这时不同的主模式就会用各自不同的语法规则来添加注释。主模式都是可
以用 M-x 启动的扩展命令,M-x fundamental-mode 就可以切换到 Fundamental
模式。</p>
<p>编辑自然语言文本――比如现在――应该用 Text 模式。</p>
<pre class="literal-block">
>> 输入 M-x text-mode <Return>。
</pre>
<p>别担心,什么都没变。不过细心一些可以发现,M-f 和 M-b 现在把单引号(')
视为词的一部分了。而在先前的 Fundamental 模式中,M-f 和 M-b 都将单引号
视为分隔单词的符号。</p>
<p>主模式通常都会搞一些类似的小动作,因为很多命令其实完成的是“相同的工
作”,只是在不同环境下会有不同的工作方式而已。【所谓“求同存异”,在 Emacs
里得到了很好的体现】</p>
<p>用 C-h m 可以查看当前主模式的文档。</p>
<pre class="literal-block">
>> 把光标移动到下一行。
>> 用 C-l C-l 将本行带到屏幕的最上方。
>> 输入 C-h m,看看 Text 模式与 Fundamental 模式有哪些不同。
>> 输入 C-x 1 关掉文档窗格。
</pre>
<p>主模式之所以称之为“主(major)”模式,是因为同时还有“辅模式”(minor
mode)存在。辅模式并不能替代主模式,而是提供一些辅助的功能。每个辅模式
都可以独立地开启和关闭,跟其它辅模式无关,跟主模式也无关。所以你可以不
使用辅模式,也可以只使用一个或同时使用多个辅模式。</p>
<p>有一个叫做自动折行(Auto Fill)的辅模式很有用,特别是在编辑自然语言文 本
的时候。启用自动折行后,Emacs 会在你打字超出一行边界时自动替你换行。</p>
<p>用 M-x auto-fill-mode 启动自动折行模式。再用一次这条命令,自
动折行模式会被关闭。也就是说,如果自动折行模式没有被开启,这个命令会开
启它;如果已经开启了,这个命令会关闭它。所以我们说,这个命令可以用来
“开关(toggle)”模式。</p>
</div>
<div class="section" id="multiple-windows">
<h2>多窗格(MULTIPLE WINDOWS)</h2>
<p>Emacs 的迷人之处很多,能够在屏幕上同时显示多个窗格就是其中之一。</p>
<pre class="literal-block">
>> 移动光标到这一行,然后输入 C-l C-l。
>> 现在输入 C-x 2,它会将屏幕划分成两个窗格。
这两个窗格里显示的都是本篇快速指南,而光标则停留在上方的窗格里。
>> 试试用 C-M-v 滚动下方的窗格。
(如果你并没有 META 键,用 ESC C-v 也可以。)
【向上滚动是 C-M-S-v,也就是同时按住 CONTROL、META 和 SHIFT 再按 v】
>> 输入 C-x o(“o”指的是“其它(other)”),
将光标转移到下方的窗格。
>> 在下方的窗格中,用 C-v 和 M-v 来滚动。
同时继续在上方的窗格里阅读这些指导。
>> 再输入 C-x o 将光标移回到上方的窗格里。
光标会回到它在上方窗格中原本所在的位置。
</pre>
<p>连续使用 C-x o 可以遍历所有窗格。“被选中的窗格”,也就是绝大多数的编辑
操作所发生的地方,是在你不打字时闪烁光标的那个窗格。其他的窗格有它们自
己的光标位置; 如果你在图形界面下运行 Emacs ,这些光标是镂空的长方形。</p>
<p>当你在一个窗格中编辑,但用另一个窗格作为参考的时候,C-M-v 是很有用的命
令。无需离开被选中的窗格,你就可以用 C-M-v 命令滚动另外一个窗格中的文
字。【比如翻译和校对就很适合用这种方式进行。】</p>
<p>C-M-v 是一个 CONTROL-META 组合键。如果你有 META (或 Alt)键的话,可以
同时按住CONTROL 和 META 键并输入 v。CONTROL 和 META 键先按哪个都可以,
因为它们只是用来“修饰(modify)”你输入的字符的。</p>
<p>如果你并没有 META 键,你也可以用 ESC 来代替,不过这样的话就要注意按键 顺
序了:你必须先输入 ESC ,然后再输入 CONTROL-v。CONTROL-ESC v 是没用的,
因为 ESC 本身是一个字符键,而不是一个修饰键(modifier key)。</p>
<pre class="literal-block">
>> (在上方窗格里)输入 C-x 1 关掉下方窗格。
</pre>
<p>(如果你在下方的窗格里输入 C-x 1,那么就会关掉上方的窗格。你可以把这个
命令看成是“只保留一个窗格”――就是我们正在编辑的这个。)</p>
<p>不同的窗格可以显示不同的缓冲区。如果你在一个窗格里用 C-x C-f 打开了一 个
文件,另一个窗格并不会发生什么变化。任何一个窗格里都可以用来打开文件。</p>
<p>用下面的方法可以在一个新开窗格里打开文件:</p>
<pre class="literal-block">
>> 输入 C-x 4 C-f,紧跟着输入一个文件名,再用 <Return> 结束。
可以看到你指定的文件出现在下方的窗格中,同时光标也跳到了那里。
>> 输入 C-x o 回到上方的窗格,然后再用 C-x 1 关掉下方窗格。
</pre>
</div>
<div class="section" id="multiple-frames">
<h2>多窗口(MULTIPLE FRAMES)</h2>
<p>Emacs 可以创建多个窗口。窗口由许多窗格以及菜单、滚动条、回显区等组成。
在图形界面下,多个窗口可以同时显示出来。在文本终端中,只能同时显示一个
窗口。</p>
<pre class="literal-block">
>> 输入 M-x make-frame <Return>。
可以看到一个新的窗口出现在了你的屏幕上。
</pre>
<p>你可以在新的窗口里做最初的窗口里可以做的任何事情。第一个窗口没有什么特
别的。</p>
<pre class="literal-block">
>> 输入 M-x delete-frame <Return>.
这个命令将会关闭选中的窗口。
</pre>
<p>你也可以通过图形系统来关闭某个窗口(通常是在窗口上面的某个角落里的一个
“X”按钮)。如果你关闭的是 Emacs 进程的最后一个窗口, Emacs 将会退出。</p>
</div>
<div class="section" id="recursive-editing-levels">
<h2>递归编辑(RECURSIVE EDITING LEVELS)</h2>
<p>有时候你会进入所谓的“递归编辑”。递归编辑状态由位于状态栏的方括号所指
示,其中包含了用小括号来指明的模式名称。比如说,你有时可能会看到
[(Fundamental)],而不是 (Fundamental)。【比如在用 M-% 进行交互式替换的
时候你又用了 C-s 进行搜索,这时替换模式并没有结束,但你又进入了搜索模
式, 这就是所谓的递归编辑。】</p>
<p>离开递归编辑可以用 ESC ESC ESC。这是一个最通用的“离开”命令,你甚至可
以使用它来关掉多余的窗格,或者离开小缓冲。</p>
<pre class="literal-block">
>> 输入 M-x 进入小缓冲;然后输入 ESC ESC ESC 离开。
</pre>
<p>你不能用 C-g 退出递归编辑,因为 C-g 的作用是取消“本层递归编辑之内”的
命令和其参数(arguments)。</p>
</div>
<div class="section" id="getting-more-help">
<h2>获得更多帮助(GETTING MORE HELP)</h2>
<p>本快速指南的目的仅仅是帮助你在 Emacs 的海洋里下水,不至于束手无策望洋 兴
叹。有关 Emacs 的话题可谓汗牛充栋,这里自然是难尽万一。不过 Emacs 很理
解你求知若渴的心情,因为它提供的强大功能实在是太多了。为此,Emacs 提供
了一些命令来查看 Emacs 的命令文档,这些命令都以 CONTROL-h 开头,这个字
符也因此被称为“帮助(Help)字符”。</p>
<p>要使用帮助(Help)功能,请先输入 C-h,然后再输入一个字符以说明你需要什
么帮助。如果你连自己到底需要什么帮助都不知道,那么就输入 C-h ?,Emacs
会告诉你它能提供了哪些帮助。如果你按了 C-h 又想反悔,可以用 C-g 取消。</p>
<p>(如果你按 C-h 之后没有任何帮助信息显示出来,那么试试 F1 键或者 M-x help
。)</p>
<p>最基本的帮助功能是 C-h c。输入 C-h c 之后再输入一个组合键,Emacs 会给 出
这个命令的简要说明。</p>
<pre class="literal-block">
>> 输入 C-h c C-p。
</pre>
<p>显示的消息应该会是这样:</p>
<pre class="literal-block">
C-p runs the command previous-line
</pre>
<p>这条消息显示了 C-p 命令对应的函数名。命令的功能由函数完成,所以函数名
本身也可以被看成是最简单的文档――至少对于你已经学过的命令来说,它们的函
数名足以解释它们的功能了。</p>
<p>多字符命令一样可以用 C-h c 来查看。</p>
<p>想得到更多的信息,请把 C-h c 换成 C-h k 试试看。</p>
<pre class="literal-block">
>> 输入 C-h k C-p。
</pre>
<p>上面的命令会新打开一个 Emacs 窗格以显示函数的名称及其文档。你读完之后 可
以用 C-x 1 关掉这个帮助窗格。当然你并不需要立即这样做,你完全可以先在 编
辑窗格里做点别的事情,然后再关掉帮助窗格。</p>
<p>还有一些其它有用的 C-h 命令:</p>
<pre class="literal-block">
C-h f 解释一个函数。需要输入函数名。
</pre>
<p>举个栗子:</p>
<pre class="literal-block">
>> 试试看,输入 C-h f previous-line <Return>。
Emacs 会给出它所知道的所有有关“实现 C-p 命令功能的函数”的信息。
</pre>
<pre class="literal-block">
C-h v 用来显示 Emacs 变量的文档。Emacs 变量可以被用来“定制 Emacs 的行
为”。同样,你需要输入变量的名称。
C-h a 相关命令搜索(Command Apropos)。
输入一个关键词然后 Emacs 会列出所有命令名中包含此关键
词
的命令。这些命令全都可以用 M-x 来启动。对于某些命令来
说,
相关命令搜索还会列出一两个组合键。
</pre>
<p>Emacs 会在另一个窗格里显示一个 M-x 命令列表,这个列表包含了所有名称中 含
有“file”的命令。你可以看到像“C-x C-f”这样的组合键显示在“find-file”
这样的命令名的旁边。</p>
<pre class="literal-block">
>> 用 C-M-v 来回滚动 help 窗格,多试几次。
>> 输入 C-x 1 来删除 help 窗格。
C-h i 阅读手册(也就是通常讲的 Info)。
这个命令会打开一个称为“*info*”的特殊缓冲区,在那里,
你可以阅读安装在系统里的软件包使用手册。要读 Emacs 的
使
用手册,按 m emacs <Return> 就可以了。如果你之前从没用
过 Info 系统,那么请按“?”,Emacs 会带你进入 Info 的使
用指南。在看完本快速指南之后,Emacs Info 会成为你的主
要
参考文档。
</pre>
</div>
<div class="section" id="translation">
<h2>翻译(TRANSLATION)</h2>
<p>翻译:孙一江 <a class="reference external" href="mailto:sunyijiang@gmail.com">sunyijiang@gmail.com</a> 维护:薛富侨 <a class="reference external" href="mailto:xfq.free@gmail.com">xfq.free@gmail.com</a>
校对:水木社区(www.newsmth.net)Emacs 板众多网友及众多 Emacs 中文用户</p>
<p>下面列出主要术语的译词对照,并给出注释说明:</p>
<pre class="literal-block">
command 命令
cursor 光标
scrolling 滚动
numeric argument 数字参数
window 窗格 [1]
insert 插入
delete 删除 [2]
kill 移除 [2]
yank 召回 [2]
undo 撤销
file 文件
buffer 缓冲区
minibuffer 小缓冲
echo area 回显区
mode line 状态栏
search 搜索
incremental search 渐进式搜索 [3]
</pre>
<p>对于其他没有提到的术语,读者可以参考 Emacs 使用手册里的术语表。</p>
<pre class="literal-block">
[1] “window”一词在计算机相关的领域一般都被译为“窗口”。但是在 Emacs
中,还有一个“frame”的概念。在被广泛使用的 X 窗口系统和微软的视窗
(Windows)系列操作系统中,Emacs 的一个“frame”就是一个“窗口”,因
此把 Emacs 中的“frame”译成“窗口”更加符合通常的习惯。这样,Emacs
中的“window”就只能译成“窗格”了。我们认为 Emacs 中 window 和
frame 的关系用窗格和窗口来类比是十分形象的。
《学习GNU Emacs》(第二版)一书对“window”和“frame”的翻译与本教程
刚好相反(分别译作“窗口”和“窗格”)。在此特别注明,以消除可能产生
的疑惑。(感谢李旭章 <lixuzhang@gmail.com> 指出)
[2] 对于“delete”和“kill”的区别,正文已经给出了详细的说明。“删除”和
“移除”相比较起来,前者更多地隐含着“破坏”和“不可恢复”的意思,而
后者更多地隐含着“被转移”和“可恢复”的意思。因此分别选择它们作为上
述两词的译词,希望能够体现出区别。“yank”在中文文档中鲜有对应译词出
现,翻译的困难较大。究其本意是:“a strong sudden pull”(参见韦氏词
典),即“猛然拉回”。在原文档中 yank 被引申为“将先前移除的东西再移
回来”这个意思,所以我们选择了“召回”一词与其对应。
[3] “incremental”一词在计算机著作中广泛出现,被广泛接受的中文译词有两
个:“增量的”和“渐进的”。“incremental search”翻译成“增量式搜索
”或者“渐进式搜索”都讲得通,且都有各自的形象之处。还是参考原文对其
的解释:“... means that the search happens while you type in the
string to search for”。意思是之所以称其为“incremental search”,是
因为“在你输入搜索字符串的过程中,搜索就已经在进行了”。我们认为“增
量的”更加强调在现有基础上的变化(比如“增量备份”,“增量编译”);
而“渐进的”更加强调过程的逐渐发展,也更加符合原文的意思。因此我们选
择将“incremental search”译作“渐进式搜索”。
</pre>
</div>
Unix/Linux终端下代理快速设置2014-10-22T00:00:00+08:002014-10-22T00:00:00+08:00SunisDowntag:sunisdown.me,2014-10-22:/unixlinuxzhong-duan-xia-dai-li-kuai-su-she-zhi.html<p>最近开始从 Vim 往 Emacs,这是一个痛苦的过程,不过好在有
evil-mode,让这个迁移的过程不至于夭折。别问我为什么要背叛 Vim 。</p>
<p>像我这种 Emacs 新手难免会需要下载插件,而下载插件就需要FUCK GFW。笨笨用</p>
<div class="highlight"><pre><span></span><span class="nv">http_proxy</span><span class="o">=</span>http://host:port emacs -nw
</pre></div>
<p>这种方法来启动 emacs,简直要哭晕在地上。</p>
<div class="section" id="id1">
<h2>设置方法</h2>
<p>所以在配置文件中加了下面的代码,方便在终端下控制当前环境变量下FUCK GFW:</p>
<div class="highlight"><pre><span></span><span class="k">function</span> proxy_on<span class="o">()</span> <span class="o">{</span>
<span class="nb">export</span> <span class="nv">no_proxy</span><span class="o">=</span><span class="s2">"localhost,127.0.0.1,localaddress,.localdomain.com"</span>
<span class="nb">export</span> <span class="nv">http_proxy</span><span class="o">=</span><span class="s2">"http://host:port"</span>
<span class="nb">export</span> <span class="nv">https_proxy</span><span class="o">=</span><span class="nv">$http_proxy</span>
<span class="nb">echo …</span></pre></div></div><p>最近开始从 Vim 往 Emacs,这是一个痛苦的过程,不过好在有
evil-mode,让这个迁移的过程不至于夭折。别问我为什么要背叛 Vim 。</p>
<p>像我这种 Emacs 新手难免会需要下载插件,而下载插件就需要FUCK GFW。笨笨用</p>
<div class="highlight"><pre><span></span><span class="nv">http_proxy</span><span class="o">=</span>http://host:port emacs -nw
</pre></div>
<p>这种方法来启动 emacs,简直要哭晕在地上。</p>
<div class="section" id="id1">
<h2>设置方法</h2>
<p>所以在配置文件中加了下面的代码,方便在终端下控制当前环境变量下FUCK GFW:</p>
<div class="highlight"><pre><span></span><span class="k">function</span> proxy_on<span class="o">()</span> <span class="o">{</span>
<span class="nb">export</span> <span class="nv">no_proxy</span><span class="o">=</span><span class="s2">"localhost,127.0.0.1,localaddress,.localdomain.com"</span>
<span class="nb">export</span> <span class="nv">http_proxy</span><span class="o">=</span><span class="s2">"http://host:port"</span>
<span class="nb">export</span> <span class="nv">https_proxy</span><span class="o">=</span><span class="nv">$http_proxy</span>
<span class="nb">echo</span> <span class="s2">"Proxy environment variable set."</span>
<span class="o">}</span>
<span class="k">function</span> proxy_off<span class="o">(){</span>
<span class="nb">unset</span> http_proxy
<span class="nb">unset</span> https_proxy
<span class="nb">echo</span> -e <span class="s2">"Proxy environment variable removed."</span>
<span class="o">}</span>
</pre></div>
<p>将上面代码加到你的<tt class="docutils literal">.bashrc</tt>中,然后执行</p>
<div class="highlight"><pre><span></span><span class="nb">source</span> ~/.bashrc
</pre></div>
<p>zsh 用户请将上面命令中的<tt class="docutils literal">.bashrc</tt>替换成<tt class="docutils literal">.zshrc</tt>,然后在终端下就可以快
速设置代理的启动与关闭了:</p>
<div class="highlight"><pre><span></span><span class="c1">#启动代理</span>
$ proxy_on
Proxy environment variable set.
<span class="c1">#关闭代理</span>
$ proxy_off
Proxy environment variable removed.
</pre></div>
</div>
<div class="section" id="id2">
<h2>友情提示</h2>
<div class="highlight"><pre><span></span>友情提示,请将 http://host:port 替换成自己的代理地址
</pre></div>
</div>
白话Python 进程,线程,协程2014-10-19T00:00:00+08:002014-10-19T00:00:00+08:00SunisDowntag:sunisdown.me,2014-10-19:/bai-hua-python-jin-cheng-xian-cheng-xie-cheng.html<p>Python被人诟病最多的大概就是性能差,在这里讲一下 Python
的多进程,多线程与协程。首先声明这不是教程,看完这篇文章,大概能够对
Python 的多进程与多线程有一定的了解。</p>
<img alt="" src="images/process_thread.png" />
<div class="section" id="id1">
<h2>进程</h2>
<p>进程是正则执行的程序实例。执行程序的过程中,内核会讲程序代码载入虚拟内存,喂程序变量分配空间,建立bookkeeping 数据结构,来记录与进程有关的信息,比如进程 ID,用户 ID 等</p>
<p>创建进程的时候,内核会为进程分配一定的资源,并在进程存活的时候不断进行调整,比如内存,进程创建的时候会占有一部分内存。进程结束的时候资源会释放出来,来让其他资源使用。</p>
<p>我们可以把进程理解为一种容器,容器内的资源可多可少,但是在容器内的程序只能使用容器内的东西。</p>
</div>
<div class="section" id="id2">
<h2>线程</h2>
<p>UNIX
中,一个进程中可以执行多个线程。多个线程共享进程内的资源。所以可以将线程可以看成是共享同一虚拟内存以及其他属性的进程。</p>
<p>线程相对于进程的优势在于同一进程下的不同线程之间的数据共享更加容易。</p>
<p>Python被诟病最多的大概就是解释器全局锁(GIL)了,GIL 的存在是为了实现Python 中对于共享资源访问的互斥。而且是非常霸道的解释器级别的互斥。在GIL的机制下,一个线程访问解释器之后,其他的线程就需要等待这个线程释放之后才可以访问 …</p></div><p>Python被人诟病最多的大概就是性能差,在这里讲一下 Python
的多进程,多线程与协程。首先声明这不是教程,看完这篇文章,大概能够对
Python 的多进程与多线程有一定的了解。</p>
<img alt="" src="images/process_thread.png" />
<div class="section" id="id1">
<h2>进程</h2>
<p>进程是正则执行的程序实例。执行程序的过程中,内核会讲程序代码载入虚拟内存,喂程序变量分配空间,建立bookkeeping 数据结构,来记录与进程有关的信息,比如进程 ID,用户 ID 等</p>
<p>创建进程的时候,内核会为进程分配一定的资源,并在进程存活的时候不断进行调整,比如内存,进程创建的时候会占有一部分内存。进程结束的时候资源会释放出来,来让其他资源使用。</p>
<p>我们可以把进程理解为一种容器,容器内的资源可多可少,但是在容器内的程序只能使用容器内的东西。</p>
</div>
<div class="section" id="id2">
<h2>线程</h2>
<p>UNIX
中,一个进程中可以执行多个线程。多个线程共享进程内的资源。所以可以将线程可以看成是共享同一虚拟内存以及其他属性的进程。</p>
<p>线程相对于进程的优势在于同一进程下的不同线程之间的数据共享更加容易。</p>
<p>Python被诟病最多的大概就是解释器全局锁(GIL)了,GIL 的存在是为了实现Python 中对于共享资源访问的互斥。而且是非常霸道的解释器级别的互斥。在GIL的机制下,一个线程访问解释器之后,其他的线程就需要等待这个线程释放之后才可以访问。这种处理方法在单处理器下面并没有什么问题,单处理器的本质是无法并行的。但是再多处理器下面,这种方法会导致无法利用多核的优势。</p>
<p>Python
的线程调度跟操作系统的进程调度类似,都属于抢占式的调度。一个进程执行了一定时间之后,发出一个信号,操作系统响应这个时钟中断(信号),开始进程调度。而在
Python 中,则通过软件模拟这种中断,来实现线程调度。</p>
</div>
<div class="section" id="id3">
<h2>协程</h2>
<p>协程我们可以看成是一种用户空间的线程,协程是主动的协作是工作,现有的操作系统线程调度是抢占式。两种的区别在于协作是调度,需要之前的任务主动放弃时间片。</p>
</div>
<div class="section" id="id4">
<h2>总结</h2>
<p>在 Python中,使用协程来做并发是对于系统资源消耗最小,可以最大化的做高并发,但是,需要我们自己来做调度,操作系统不会主动调度任务。</p>
</div>
Mininet Walkthrough2014-10-16T00:00:00+08:002014-10-16T00:00:00+08:00SunisDowntag:sunisdown.me,2014-10-16:/mininet-walkthrough.html<p>译者注: 这篇 Blog 是在学习 SDN 过程中翻译的Mininet
官方的文档。文档主要是介绍了 Mininet
的简单用法。会分成几个部分放出来,<a class="reference external" href="http://mininet.org/walkthrough/#test-connectivity-between-hosts">原文</a>。下面是正文</p>
<div class="section" id="part-1-everyday-mininet-usage">
<h2>Part 1: Everyday Mininet Usage</h2>
<p>首先是是命令语法</p>
<ul class="simple">
<li><tt class="docutils literal">$</tt> 这个符号代表现在处于 Linux 的shell 交互下,需要使用的是 Linux
命令</li>
<li><tt class="docutils literal">mininet></tt> 这个符号表示现在处于 Mininet 交互下,需要使用的是
Mininet 的命令</li>
<li><tt class="docutils literal">#</tt> 这个符号表示的是现在处于 Linux 的 root 权限下。</li>
</ul>
<p>以上相应的状态下下属于对应的命令,就能够得到正常的输出。需要注意的是<tt class="docutils literal">mininet></tt>的情况比较特殊,需要使用
minient 的命令来进行交互。</p>
<div class="section" id="display-startup-options">
<h3>Display Startup Options</h3>
<p>我们首先来启动 …</p></div></div><p>译者注: 这篇 Blog 是在学习 SDN 过程中翻译的Mininet
官方的文档。文档主要是介绍了 Mininet
的简单用法。会分成几个部分放出来,<a class="reference external" href="http://mininet.org/walkthrough/#test-connectivity-between-hosts">原文</a>。下面是正文</p>
<div class="section" id="part-1-everyday-mininet-usage">
<h2>Part 1: Everyday Mininet Usage</h2>
<p>首先是是命令语法</p>
<ul class="simple">
<li><tt class="docutils literal">$</tt> 这个符号代表现在处于 Linux 的shell 交互下,需要使用的是 Linux
命令</li>
<li><tt class="docutils literal">mininet></tt> 这个符号表示现在处于 Mininet 交互下,需要使用的是
Mininet 的命令</li>
<li><tt class="docutils literal">#</tt> 这个符号表示的是现在处于 Linux 的 root 权限下。</li>
</ul>
<p>以上相应的状态下下属于对应的命令,就能够得到正常的输出。需要注意的是<tt class="docutils literal">mininet></tt>的情况比较特殊,需要使用
minient 的命令来进行交互。</p>
<div class="section" id="display-startup-options">
<h3>Display Startup Options</h3>
<p>我们首先来启动 Mininet。</p>
<p>键入以下命令来显示Mininet的帮助信息:</p>
<div class="highlight"><pre><span></span>$ sudo mn -h
Usage: mn <span class="o">[</span>options<span class="o">]</span>
<span class="o">(</span><span class="nb">type</span> mn -h <span class="k">for</span> details<span class="o">)</span>
The mn utility creates Mininet network from the <span class="nb">command</span> line. It can create
parametrized topologies, invoke the Mininet CLI, and run tests.
Options:
-h, --help show this <span class="nb">help</span> message and <span class="nb">exit</span>
--switch<span class="o">=</span>SWITCH ivs<span class="p">|</span>ovsk<span class="p">|</span>ovsl<span class="p">|</span>user<span class="o">[</span>,param<span class="o">=</span>value...<span class="o">]</span>
--host<span class="o">=</span>HOST cfs<span class="p">|</span>proc<span class="p">|</span>rt<span class="o">[</span>,param<span class="o">=</span>value...<span class="o">]</span>
--controller<span class="o">=</span>CONTROLLER
none<span class="p">|</span>nox<span class="p">|</span>ovsc<span class="p">|</span>ref<span class="p">|</span>remote<span class="o">[</span>,param<span class="o">=</span>value...<span class="o">]</span>
--link<span class="o">=</span>LINK default<span class="p">|</span>tc<span class="o">[</span>,param<span class="o">=</span>value...<span class="o">]</span>
--topo<span class="o">=</span>TOPO linear<span class="p">|</span>minimal<span class="p">|</span>reversed<span class="p">|</span>single<span class="p">|</span>tree<span class="o">[</span>,param<span class="o">=</span>value...<span class="o">]</span>
-c, --clean clean and <span class="nb">exit</span>
--custom<span class="o">=</span>CUSTOM <span class="nb">read</span> custom topo and node params from .pyfile
--test<span class="o">=</span>TEST cli<span class="p">|</span>build<span class="p">|</span>pingall<span class="p">|</span>pingpair<span class="p">|</span>iperf<span class="p">|</span>all<span class="p">|</span>iperfudp<span class="p">|</span>none
-x, --xterms spawn xterms <span class="k">for</span> each node
-i IPBASE, --ipbase<span class="o">=</span>IPBASE
base IP address <span class="k">for</span> hosts
--mac automatically <span class="nb">set</span> host MACs
--arp <span class="nb">set</span> all-pairs ARP entries
-v VERBOSITY, --verbosity<span class="o">=</span>VERBOSITY
info<span class="p">|</span>warning<span class="p">|</span>critical<span class="p">|</span>error<span class="p">|</span>debug<span class="p">|</span>output
--innamespace sw and ctrl in namespace?
--listenport<span class="o">=</span>LISTENPORT
base port <span class="k">for</span> passive switch listening
--nolistenport don<span class="err">'</span>t use passive listening port
--pre<span class="o">=</span>PRE CLI script to run before tests
--post<span class="o">=</span>POST CLI script to run after tests
--pin pin hosts to CPU cores <span class="o">(</span>requires --host cfs or --host
rt<span class="o">)</span>
--version
</pre></div>
<p>如上所示,输出了 mn 的帮助信息。</p>
</div>
<div class="section" id="start-wireshark">
<h3>Start Wireshark</h3>
<p>为了使用 Wireshark 来查看 OpenFlow 的控制信息,我们先打开 Wireshark
并让他在后台运行。</p>
<div class="highlight"><pre><span></span>$ sudo wireshark <span class="p">&</span>
</pre></div>
<p>在 Wireshark 的过滤选项中,输入<tt class="docutils literal">of</tt>,然后选择 Apply。</p>
<p>In Wireshark, click Capture, then Interfaces, then select Start on the
loopback interface (<tt class="docutils literal">lo</tt>).</p>
<p>现在窗口上暂时应该没有任何 OpenFlow 的数据包。</p>
<pre class="literal-block">
注:在Mininet VM镜像中Wireshark是默认已经安装的。如果你的系统中没有Wireshark的和OpenFlow,您可以使用Mininet的install.sh脚本,按以下步骤安装:
</pre>
<div class="highlight"><pre><span></span>$ <span class="nb">cd</span> ~
$ git clone https://github.com/mininet/mininet#如果它尚不存在
$ mininet/util/install.sh -w
</pre></div>
<p>如果已经安装了 Wireshark,但是运行不了(e.g.
你得到一个类似<tt class="docutils literal">$DISPLAY not set</tt>之类的错误信息,可以参考
FAQ,:<a class="reference external" href="https://github.com/mininet/mininet/wiki/FAQ#wiki-X11-forwarding">https://github.com/mininet/mininet/wiki/FAQ#wiki-X11-forwarding</a>)</p>
<p>设置好 X11就可以正常运行 GUI 程序,并且使用 xterm
之类的终端仿真器了,后面的演示中可以用到。</p>
</div>
<div class="section" id="interact-with-hosts-and-switches">
<h3>Interact with Hosts and Switches</h3>
<p>Start a minimal topology and enter the CLI:</p>
<div class="highlight"><pre><span></span>$ sudo mn
</pre></div>
<p>默认的最小拓扑结构包含有两台主机(h1,h2),还有一个 OpenFlow
的交换机,一个 OpenFlow
的控制器四台设备。这种拓扑接口也可以使用<tt class="docutils literal"><span class="pre">--topo=minimal</span></tt>来指定。当然我们也可以使用其他的拓扑结构,具体信息可以看
<tt class="docutils literal"><span class="pre">--topo</span></tt>的信息。</p>
<p>现在四个实体(h1,h2,c0,s1)都在运行着。c0作为控制器,是可以放在虚拟机外部的。</p>
<p>如果没有具体的测试作为参数传递时,我们可以使用 Mininet 交互。</p>
<p>在Wireshark的窗口中,你会看到内核交换机连接到控制器。</p>
<p>显示Mininet CLI命令:</p>
<div class="highlight"><pre><span></span>mininet> <span class="nb">help</span>
Documented commands <span class="o">(</span><span class="nb">type</span> <span class="nb">help</span> <topic><span class="o">)</span>:
<span class="o">========================================</span>
EOF <span class="nb">exit</span> intfs link noecho pingpair py <span class="nb">source</span> xterm
dpctl gterm iperf net pingall pingpairfull quit <span class="nb">time</span>
dump <span class="nb">help</span> iperfudp nodes pingallfull px sh x
You may also send a <span class="nb">command</span> to a node using:
<node> <span class="nb">command</span> <span class="o">{</span>args<span class="o">}</span>
For example:
mininet> h1 ifconfig
The interpreter automatically substitutes IP addresses
<span class="k">for</span> node names when a node is the first arg, so commands
like
mininet> h2 ping h3
should work.
Some character-oriented interactive commands require
noecho:
mininet> noecho h2 vi foo.py
However, starting up an xterm/gterm is generally better:
mininet> xterm h2
</pre></div>
<p>显示节点:</p>
<pre class="literal-block">
mininet> nodes
available nodes are:
c0 h1 h2 s1
</pre>
<p>显示网络链接:</p>
<pre class="literal-block">
mininet> net
h1 h1-eth0:s1-eth1
h2 h2-eth0:s1-eth2
s1 lo: s1-eth1:h1-eth0 s1-eth2:h2-eth0
c0
</pre>
<p>输出所有节点的信息:</p>
<pre class="literal-block">
mininet> dump
<Host h1: h1-eth0:10.0.0.1 pid=3278>
<Host h2: h2-eth0:10.0.0.2 pid=3279>
<OVSSwitch s1: lo:127.0.0.1,s1-eth1:None,s1-eth2:None pid=3282>
<OVSController c0: 127.0.0.1:6633 pid=3268>
</pre>
<p>从上面的输出中,你可以看到有一台交换机和两台主机。</p>
<p>在 Mininet 的CLI
中第一个字符串是设备名,那后面的命令就在该设备上执行。例如我们想在h1设备上执行<tt class="docutils literal">ifconfig</tt>则输入如下命令:</p>
<div class="highlight"><pre><span></span>mininet> h1 ifconfig -a
h1-eth0 Link encap:Ethernet HWaddr 3e:94:43:b1:ad:48
inet addr:10.0.0.1 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::3c94:43ff:feb1:ad48/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:22 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1764 <span class="o">(</span><span class="m">1</span>.7 KB<span class="o">)</span> TX bytes:648 <span class="o">(</span><span class="m">648</span>.0 B<span class="o">)</span>
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 <span class="o">(</span><span class="m">0</span>.0 B<span class="o">)</span> TX bytes:0 <span class="o">(</span><span class="m">0</span>.0 B<span class="o">)</span>
</pre></div>
<p>上面的输出中,可以看见 <tt class="docutils literal"><span class="pre">h1-eth0</span></tt> 跟 <tt class="docutils literal">lo</tt>两个接口,需要注意的是,在
Linux 系统的 shell 中运行<tt class="docutils literal">ifconfig</tt>是看不到h1-eth0。</p>
<p>与<tt class="docutils literal"><span class="pre">h1-eth0</span></tt>相反的是,<tt class="docutils literal">switch</tt> 默认是跑在 root
的网络namespace上面,所以在<tt class="docutils literal">switch</tt>上执行命令与在 Linux 下的 shell
中是一样的。</p>
<div class="highlight"><pre><span></span>mininet> s1 ifconfig-a
eth0 Link encap:Ethernet HWaddr <span class="m">08</span>:00:27:98:dc:aa
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe98:dcaa/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:46716 errors:0 dropped:0 overruns:0 frame:0
TX packets:40265 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10804203 <span class="o">(</span><span class="m">10</span>.8 MB<span class="o">)</span> TX bytes:40122199 <span class="o">(</span><span class="m">40</span>.1 MB<span class="o">)</span>
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:43654 errors:0 dropped:0 overruns:0 frame:0
TX packets:43654 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:37264504 <span class="o">(</span><span class="m">37</span>.2 MB<span class="o">)</span> TX bytes:37264504 <span class="o">(</span><span class="m">37</span>.2 MB<span class="o">)</span>
lxcbr0 Link encap:Ethernet HWaddr fe:5e:f0:f7:a6:f3
inet addr:10.0.3.1 Bcast:10.0.3.255 Mask:255.255.255.0
inet6 addr: fe80::a8c4:b5ff:fea6:2809/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:52 errors:0 dropped:0 overruns:0 frame:0
TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4759 <span class="o">(</span><span class="m">4</span>.7 KB<span class="o">)</span> TX bytes:2952 <span class="o">(</span><span class="m">2</span>.9 KB<span class="o">)</span>
ovs-system Link encap:Ethernet HWaddr 3e:79:59:3d:d9:bb
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 <span class="o">(</span><span class="m">0</span>.0 B<span class="o">)</span> TX bytes:0 <span class="o">(</span><span class="m">0</span>.0 B<span class="o">)</span>
s1 Link encap:Ethernet HWaddr 6e:8c:5d:91:d5:44
inet6 addr: fe80::fc47:8aff:fe6a:4155/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:13 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1026 <span class="o">(</span><span class="m">1</span>.0 KB<span class="o">)</span> TX bytes:648 <span class="o">(</span><span class="m">648</span>.0 B<span class="o">)</span>
s1-eth1 Link encap:Ethernet HWaddr 5e:a2:f7:86:f3:b1
inet6 addr: fe80::5ca2:f7ff:fe86:f3b1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:648 <span class="o">(</span><span class="m">648</span>.0 B<span class="o">)</span> TX bytes:1764 <span class="o">(</span><span class="m">1</span>.7 KB<span class="o">)</span>
s1-eth2 Link encap:Ethernet HWaddr b2:c6:37:e0:d9:61
inet6 addr: fe80::b0c6:37ff:fee0:d961/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:648 <span class="o">(</span><span class="m">648</span>.0 B<span class="o">)</span> TX bytes:1674 <span class="o">(</span><span class="m">1</span>.6 KB<span class="o">)</span>
veth14524J Link encap:Ethernet HWaddr fe:ca:13:f5:dd:b4
inet6 addr: fe80::fcca:13ff:fef5:ddb4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:648 <span class="o">(</span><span class="m">648</span>.0 B<span class="o">)</span> TX bytes:4190 <span class="o">(</span><span class="m">4</span>.1 KB<span class="o">)</span>
veth2K19CE Link encap:Ethernet HWaddr fe:f1:f7:e8:49:45
inet6 addr: fe80::fcf1:f7ff:fee8:4945/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:42 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:648 <span class="o">(</span><span class="m">648</span>.0 B<span class="o">)</span> TX bytes:4370 <span class="o">(</span><span class="m">4</span>.3 KB<span class="o">)</span>
veth9WSHRK Link encap:Ethernet HWaddr fe:87:1d:33:f6:41
inet6 addr: fe80::fc87:1dff:fe33:f641/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:648 <span class="o">(</span><span class="m">648</span>.0 B<span class="o">)</span> TX bytes:4460 <span class="o">(</span><span class="m">4</span>.4 KB<span class="o">)</span>
vethH2K7R5 Link encap:Ethernet HWaddr fe:5e:f0:f7:a6:f3
inet6 addr: fe80::fc5e:f0ff:fef7:a6f3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14 errors:0 dropped:0 overruns:0 frame:0
TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1776 <span class="o">(</span><span class="m">1</span>.7 KB<span class="o">)</span> TX bytes:5030 <span class="o">(</span><span class="m">5</span>.0 KB<span class="o">)</span>
vethO99MI2 Link encap:Ethernet HWaddr fe:cf:ee:97:fb:7f
inet6 addr: fe80::fccf:eeff:fe97:fb7f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14 errors:0 dropped:0 overruns:0 frame:0
TX packets:51 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1767 <span class="o">(</span><span class="m">1</span>.7 KB<span class="o">)</span> TX bytes:5294 <span class="o">(</span><span class="m">5</span>.2 KB<span class="o">)</span>
</pre></div>
<p>上面的输出中包含交换机的虚拟网卡 s1,以及主机的 eth0。</p>
<p>为了区别显示host
主机的网络是隔离的,我们可以通过<tt class="docutils literal">arp</tt>与<tt class="docutils literal">route</tt>命令来做演示,分别在
s1与h1上面演示如下:</p>
<div class="highlight"><pre><span></span>mininet> s1 arp
Address HWtype HWaddress Flags Mask Iface
localhost ether <span class="m">00</span>:16:3e:54:9c:03 C lxcbr0
localhost ether <span class="m">52</span>:54:00:12:35:02 C eth0
localhost ether <span class="m">52</span>:54:00:12:35:03 C eth0
localhost ether <span class="m">00</span>:16:3e:51:24:a7 C lxcbr0
mininet> s1 route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default localhost <span class="m">0</span>.0.0.0 UG <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> eth0
<span class="m">10</span>.0.2.0 * <span class="m">255</span>.255.255.0 U <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> eth0
<span class="m">10</span>.0.3.0 * <span class="m">255</span>.255.255.0 U <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> lxcbr0
<span class="m">172</span>.17.0.0 * <span class="m">255</span>.255.0.0 U <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> docker0
mininet> h1 arp
mininet> h1 route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
<span class="m">10</span>.0.0.0 * <span class="m">255</span>.0.0.0 U <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> h1-eth0
</pre></div>
<p>这样可以做到将每一个主机,交换机,以及控制器都放到他自己的标准的 network
namespace
中,但是这种做法并没有什么特别的优势,除非你想复制一个非常复杂的网络。Mininet
不支持这种做法,你可以通过<tt class="docutils literal"><span class="pre">--innamespace</span></tt>参数来查看更多的信息。
<tt class="docutils literal">译者注:感觉有点像 LXC 或者说想最近比较火的 Docker</tt></p>
<p><em>注意</em>:只有网络是虚拟出来的,每一个主机里面的进程使用的都是同一套目录,可以看到相同的进程集合,我们打印不同主机下面的进程列表看看:</p>
<div class="highlight"><pre><span></span>mininet> h1 ps -a
PID TTY TIME CMD
<span class="m">3899</span> pts/3 <span class="m">00</span>:00:00 tmux
<span class="m">4000</span> pts/23 <span class="m">00</span>:00:00 sudo
<span class="m">4001</span> pts/23 <span class="m">00</span>:00:51 wireshark
<span class="m">4030</span> pts/23 <span class="m">00</span>:00:00 dbus-launch
<span class="m">4530</span> pts/23 <span class="m">00</span>:00:43 dumpcap
<span class="m">4541</span> pts/22 <span class="m">00</span>:00:00 sudo
<span class="m">4542</span> pts/22 <span class="m">00</span>:00:00 mn
mininet> h2 ps -a
PID TTY TIME CMD
<span class="m">3899</span> pts/3 <span class="m">00</span>:00:00 tmux
<span class="m">4000</span> pts/23 <span class="m">00</span>:00:00 sudo
<span class="m">4001</span> pts/23 <span class="m">00</span>:00:52 wireshark
<span class="m">4030</span> pts/23 <span class="m">00</span>:00:00 dbus-launch
<span class="m">4530</span> pts/23 <span class="m">00</span>:00:43 dumpcap
<span class="m">4541</span> pts/22 <span class="m">00</span>:00:00 sudo
<span class="m">4542</span> pts/22 <span class="m">00</span>:00:00 mn
mininet> s1 ps -a
PID TTY TIME CMD
<span class="m">3899</span> pts/3 <span class="m">00</span>:00:00 tmux
<span class="m">4000</span> pts/23 <span class="m">00</span>:00:00 sudo
<span class="m">4001</span> pts/23 <span class="m">00</span>:00:54 wireshark
<span class="m">4030</span> pts/23 <span class="m">00</span>:00:00 dbus-launch
<span class="m">4530</span> pts/23 <span class="m">00</span>:00:46 dumpcap
<span class="m">4541</span> pts/22 <span class="m">00</span>:00:00 sudo
<span class="m">4542</span> pts/22 <span class="m">00</span>:00:00 mn
</pre></div>
<p>如上所示, h1,h2,s1三个进程列表是完全相同的。</p>
<p>其实完全可以做到各个主机完全独立,就想 LXC 那样,但是目前 Mininet
并没有这么做。在 Mininet 中所有的进程都放在 root 下面,这样你可以在
Linux的 shell
中直接用<tt class="docutils literal">kill</tt>或者<tt class="docutils literal">ps</tt>这些命令查看或者杀死进程。</p>
</div>
<div class="section" id="test-connectivity-between-hosts">
<h3>Test connectivity between hosts</h3>
<p>现在,验证您可以h1 ping 通 h2:</p>
<div class="highlight"><pre><span></span>mininet> h1 ping h2 -c <span class="m">1</span>
PING <span class="m">10</span>.0.0.2 <span class="o">(</span><span class="m">10</span>.0.0.2<span class="o">)</span> <span class="m">56</span><span class="o">(</span><span class="m">84</span><span class="o">)</span> bytes of data.
<span class="m">64</span> bytes from <span class="m">10</span>.0.0.2: <span class="nv">icmp_seq</span><span class="o">=</span><span class="m">1</span> <span class="nv">ttl</span><span class="o">=</span><span class="m">64</span> <span class="nv">time</span><span class="o">=</span><span class="m">8</span>.57 ms
--- <span class="m">10</span>.0.0.2 ping statistics ---
<span class="m">1</span> packets transmitted, <span class="m">1</span> received, <span class="m">0</span>% packet loss, <span class="nb">time</span> 0ms
rtt min/avg/max/mdev <span class="o">=</span> <span class="m">8</span>.576/8.576/8.576/0.000 ms
</pre></div>
<p>mininet中的命令语法如上所示。<tt class="docutils literal">host1 command host2</tt>。</p>
<p>在 Wireshark 中可以看到 OpenFlow 的控制流量,可以看到h1 ARPs h2的
mac,并将一个 <tt class="docutils literal">packet_in</tt>发送到
<tt class="docutils literal">c0</tt>,然后<tt class="docutils literal">c0</tt>发送<tt class="docutils literal">packet_out</tt>消息流广播到交换机(在本例中,唯一的其他数据端口)。第二个主机接受到的ARP请求,并发送一个广播答复。此回复进到控制器,该控制器将其发送到<tt class="docutils literal">h1</tt>并且
pushes down a flow entry。</p>
<p>现在第一主机知道的第二个IP地址,并且可以通过ICMP ping
来回显请求。这个请求,连同其从第二主机对应的应答,both go the controller
and result in a flow entry pushed down (along with the actual packets
getting sent out).</p>
<p>重复前一条命令:</p>
<div class="highlight"><pre><span></span>mininet> h1 ping -c <span class="m">1</span> h2
</pre></div>
<p>这次 ping 的时间将比第一次低的多, A flow entry covering ICMP ping
traffic was previously installed in the switch, so no control traffic
was generated, and the packets immediately pass through the switch.</p>
<p>使用<tt class="docutils literal">pingall</tt>命令可以让每一个节点直接都产生上面的效果。</p>
<div class="highlight"><pre><span></span>mininet> pingall
</pre></div>
</div>
<div class="section" id="run-a-simple-web-server-and-client">
<h3>Run a simple web server and client</h3>
<p>我们不单可以在主机上面运行<tt class="docutils literal">ping</tt>命令,每一条
Linux下的命令或者程序都可以在 Mininet 中运行:</p>
<p>接下来,尝试开始于h1启动一个简单的HTTP服务器上,然后从h2发出请求,最后关闭Web服务器:</p>
<div class="highlight"><pre><span></span>mininet> h1 python -m SimpleHTTPServer <span class="m">80</span> <span class="p">&</span>
mininet> h2 wget h1
--2014-09-15 <span class="m">08</span>:10:11-- http://10.0.0.1/
Connecting to <span class="m">10</span>.0.0.1:80... connected.
HTTP request sent, awaiting response... <span class="m">200</span> OK
Length: <span class="m">2647</span> <span class="o">(</span><span class="m">2</span>.6K<span class="o">)</span> <span class="o">[</span>text/html<span class="o">]</span>
Saving to: ‘index.html’
0K .. <span class="m">100</span>% <span class="m">71</span>.7M<span class="o">=</span>0s
<span class="m">2014</span>-09-15 <span class="m">08</span>:10:11 <span class="o">(</span><span class="m">71</span>.7 MB/s<span class="o">)</span> - ‘index.html’ saved <span class="o">[</span><span class="m">2647</span>/2647<span class="o">]</span>
mininet> h1 <span class="nb">kill</span> %python
</pre></div>
<p>退出mininet交互命令:</p>
<div class="highlight"><pre><span></span>mininet>exit
</pre></div>
</div>
<div class="section" id="cleanup">
<h3>cleanup</h3>
<p>如果Mininet出于某种原因崩溃,可以用下面命令来清理:</p>
<div class="highlight"><pre><span></span>sudo mn -c
</pre></div>
</div>
</div>
<div class="section" id="part-2-advanced-startup-options">
<h2>Part 2: 高级选项Advanced Startup Options</h2>
<div class="section" id="run-a-regression-test">
<h3>回归测试Run a Regression Test</h3>
<p>Mininet 可以用于直接运行回归测试,不一定要切换到他的 CLI 下面。</p>
<p>运行回归测试:</p>
<div class="highlight"><pre><span></span>$ sudo mn --test pingpair
</pre></div>
<p>这条命令会创建一个小的拓扑结构,然后启动 OpenFLow 的控制器,然后跑 ping
测试,最后再把拓扑结构跟控制器关掉。</p>
<p>另一种有用的试验是iperf的(给它约10秒来完成):
还有一直常用的测试是<tt class="docutils literal">iperf</tt>(完成这个测试大概需要10s 钟):</p>
<div class="highlight"><pre><span></span>$ sudo mn --test iperf
</pre></div>
<p>此命令创建的相同Mininet,并在其中一台 host 上面跑 iperf server,
然后在另外一台 host 上面运行iperf client 然后解析取得带宽情况。
####更改拓扑结构大小和类型 Changing Topology Size and Type</p>
<hr class="docutils" />
<p>Mininet 默认的拓扑结构是由两台 host
以及一台交换机组成的,你可以用<tt class="docutils literal"><span class="pre">--topo</span></tt>参数来更改拓扑结构。
假设你要在一个交换机与三台 host 之间做 ping 探测验证(verify all-pairs
ping connectivity)。:</p>
<p>运行回归测试:</p>
<div class="highlight"><pre><span></span>$ sudo mn --test pingall --topo single,3
</pre></div>
<p>另一个例子中,使用线性拓扑(其中每个交换机配有一个主机,并且所有的交换机连接在一起):</p>
<div class="highlight"><pre><span></span>$ sudo mn --test pingall --topo linear,4
</pre></div>
<p>课哟用参数来控制拓扑结构是 Mininet 中最有用的功能之一,非常强大。</p>
</div>
<div class="section" id="link-variations">
<h3>链路变化 Link variations</h3>
<p>Mininet2.0允许你设置连接参数,甚至可以通过命令行实现自动化设置:</p>
<div class="highlight"><pre><span></span>$ sudo mn --link tc,bw<span class="o">=</span><span class="m">10</span>,delay<span class="o">=</span>10ms
mininet> iperf
...
mininet> h1 ping -c10 h2
</pre></div>
<p>上面的设置每两个节点之间的延迟是10ms,因为 ICMP
请求传过了两条链路(一次是到大交换机,一次到达主机),往返时间(RRT)就应该是40ms。
你还可以使用
<a class="reference external" href="https://github.com/mininet/mininet/wiki/Introduction-to-Mininet">PythonAPI</a>
来做更多的事儿,不过现在我们先继续往下演练。</p>
</div>
<div class="section" id="adjustable-verbosity">
<h3>调整输出信息Adjustable Verbosity</h3>
<p>Mininet默认输出信息的级别是 <tt class="docutils literal">Info</tt>,<tt class="docutils literal">Info</tt>级别会输出
Mininet的详细信息。 我们也可以通过
<tt class="docutils literal"><span class="pre">-v</span></tt>参数来设置输出<tt class="docutils literal">DEBUG</tt>信息。</p>
<div class="highlight"><pre><span></span>$ sudo mn -v debug
...
mininet> <span class="nb">exit</span>
</pre></div>
<p>这样会打印出更多额外的细节。现在尝试一下<tt class="docutils literal">output</tt>参数,这样可以在
CLI 中打印更少的信息。</p>
<div class="highlight"><pre><span></span>$ sudo mn -v output
mininet> <span class="nb">exit</span>
</pre></div>
<p>除了上面的几个级别,还有其他的级别可以使用,比如<tt class="docutils literal">warning</tt>等</p>
</div>
<div class="section" id="custom-topologies">
<h3>Custom Topologies自定义拓扑结构</h3>
<p>在<tt class="docutils literal"><span class="pre">custom/topo-2sw-2host.py</span></tt>中是一个例子可以拿来参考,我们可以看到通过
PythonAPI 我们可以很简单的来定义拓扑结构。
这个例子直接连接两台交换机,每个交换机带有一台主机。</p>
<div class="highlight"><pre><span></span><span class="sd">"""Custom topology example</span>
<span class="sd">Two directly connected switches plus a host for each switch:</span>
<span class="sd"> host --- switch --- switch --- host</span>
<span class="sd">Adding the 'topos' dict with a key/value pair to generate our newly defined</span>
<span class="sd">topology enables one to pass in '--topo=mytopo' from the command line.</span>
<span class="sd">"""</span>
<span class="kn">from</span> <span class="nn">mininet.topo</span> <span class="kn">import</span> <span class="n">Topo</span>
<span class="k">class</span> <span class="nc">MyTopo</span><span class="p">(</span> <span class="n">Topo</span> <span class="p">):</span>
<span class="s2">"Simple topology example."</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span> <span class="bp">self</span> <span class="p">):</span>
<span class="s2">"Create custom topo."</span>
<span class="c1"># Initialize topology</span>
<span class="n">Topo</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span> <span class="bp">self</span> <span class="p">)</span>
<span class="c1"># Add hosts and switches</span>
<span class="n">leftHost</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">addHost</span><span class="p">(</span> <span class="s1">'h1'</span> <span class="p">)</span>
<span class="n">rightHost</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">addHost</span><span class="p">(</span> <span class="s1">'h2'</span> <span class="p">)</span>
<span class="n">leftSwitch</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">addSwitch</span><span class="p">(</span> <span class="s1">'s3'</span> <span class="p">)</span>
<span class="n">rightSwitch</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">addSwitch</span><span class="p">(</span> <span class="s1">'s4'</span> <span class="p">)</span>
<span class="c1"># Add links</span>
<span class="bp">self</span><span class="o">.</span><span class="n">addLink</span><span class="p">(</span> <span class="n">leftHost</span><span class="p">,</span> <span class="n">leftSwitch</span> <span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">addLink</span><span class="p">(</span> <span class="n">leftSwitch</span><span class="p">,</span> <span class="n">rightSwitch</span> <span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">addLink</span><span class="p">(</span> <span class="n">rightSwitch</span><span class="p">,</span> <span class="n">rightHost</span> <span class="p">)</span>
<span class="n">topos</span> <span class="o">=</span> <span class="p">{</span> <span class="s1">'mytopo'</span><span class="p">:</span> <span class="p">(</span> <span class="k">lambda</span><span class="p">:</span> <span class="n">MyTopo</span><span class="p">()</span> <span class="p">)</span> <span class="p">}</span>
</pre></div>
<p>我们提供一个自定义的mininet 文件,就可以创建新的拓扑结构、交换机类型。
我们在命令行里面测试一下:</p>
<div class="highlight"><pre><span></span>$ sudo mn --custom ~/mininet/custom/topo-2sw-2host.py --topo mytopo --test pingall
*** Creating network
*** Adding controller
*** Adding hosts:
h1 h2
*** Adding switches:
s3 s4
*** Adding links:
<span class="o">(</span>h1, s3<span class="o">)</span> <span class="o">(</span>h2, s4<span class="o">)</span> <span class="o">(</span>s3, s4<span class="o">)</span>
*** Configuring hosts
h1 h2
*** Starting controller
*** Starting <span class="m">2</span> switches
s3 s4
*** Ping: testing ping reachability
h1 -> h2
h2 -> h1
*** Results: <span class="m">0</span>% dropped <span class="o">(</span><span class="m">2</span>/2 received<span class="o">)</span>
*** Stopping <span class="m">2</span> switches
s3 ..s4 ..
*** Stopping <span class="m">2</span> hosts
h1 h2
*** Stopping <span class="m">1</span> controllers
c0
*** Done
completed in <span class="m">1</span>.220 seconds
</pre></div>
</div>
<div class="section" id="id-mac">
<h3>ID= MAC</h3>
<p>默认情况下,host 的 mac 地址是随机分配的。这会导致每次 mininet
创建的时候,MAC地址都会改变,这会给调试带来一些困难</p>
<p><tt class="docutils literal"><span class="pre">--mac</span></tt>参数可以解决上面的问题,栗子如下:</p>
<p>之前:</p>
<div class="highlight"><pre><span></span>$ sudo mn
mininet> h1 ifconfig
h1-eth0 Link encap:Ethernet HWaddr c2:d9:4a:37:25:17
inet addr:10.0.0.1 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::c0d9:4aff:fe37:2517/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1398 <span class="o">(</span><span class="m">1</span>.3 KB<span class="o">)</span> TX bytes:578 <span class="o">(</span><span class="m">578</span>.0 B<span class="o">)</span>
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 <span class="o">(</span><span class="m">0</span>.0 B<span class="o">)</span> TX bytes:0 <span class="o">(</span><span class="m">0</span>.0 B<span class="o">)</span>
</pre></div>
<p>使用<tt class="docutils literal"><span class="pre">--mac</span></tt>参数:</p>
<pre class="literal-block">
$ sudo mn --mac
mininet> h1 ifconfig
h1-eth0 Link encap:Ethernet HWaddr 00:00:00:00:00:01
inet addr:10.0.0.1 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::200:ff:fe00:1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1414 (1.4 KB) TX bytes:676 (676.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
</pre>
<p>n contrast, the MACs for switch data ports reported by Linux will remain
random. This is because you can ‘assign’ a MAC to a data port using
OpenFlow, as noted in the FAQ. This is a somewhat subtle point which you
can probably ignore for now.</p>
</div>
<div class="section" id="xterm-display-xterm">
<h3>XTerm Display xterm屏显</h3>
<p>为了方便更复杂的调试工作,可以使用 mininet 的 xterms</p>
<p>可以通过<tt class="docutils literal">x</tt>选项来给每一个 host 与交换机启动一个<tt class="docutils literal">xterm</tt>。</p>
<div class="highlight"><pre><span></span>$ sudo mn -x
</pre></div>
<p>后一秒钟,在xterm终端会弹出,并且具有自动设置窗口的名称(<tt class="docutils literal">h1</tt>,<tt class="docutils literal">h2</tt>...)。</p>
<p>或者,您也可以用下面的方式打开更多的xterm。</p>
<p>默认情况下,仅仅 host 需要一个但大户的
namespace,而交换机的窗口则不用(与政策的终端类似) but can be a
convenient place to run and leave up switch debug commands, such as flow
counter dumps.</p>
<p>在你想看到交互命令的时候,xterm
很有用,但是如果你仅仅想看到输出信息,那你可能想停掉 xterm</p>
<p>例如: 在<tt class="docutils literal">switch: s1 (root)</tt>的 xterm下面运行:</p>
<div class="highlight"><pre><span></span><span class="c1"># dpctl dump-flows tcp:127.0.0.1:6634</span>
</pre></div>
<p>因为交换机中没有数据流量,所以不会有信息输出。 To use <tt class="docutils literal">dpctl</tt> with
other switches, start up mininet in verbose mode and look at the passive
listening ports for the switches when they’re created.</p>
<p>现在,在<tt class="docutils literal">host: h1</tt>的xterm中运行:</p>
<div class="highlight"><pre><span></span><span class="c1"># ping 10.0.0.2</span>
</pre></div>
<p>回到<tt class="docutils literal">s1</tt>的 xterm中查看:</p>
<div class="highlight"><pre><span></span><span class="c1"># dpctl dump-flows tcp:127.0.0.1:6634</span>
</pre></div>
<p>现在就可以看见数据流了。 另外我们可以直接用<tt class="docutils literal">dpctl</tt>命令直接调用
Mininet CLI 里面的命令,而不需要启动任何<tt class="docutils literal">xterm</tt>或者指定交换机的IP
跟端口。 我们看已通过<tt class="docutils literal">ifconfig</tt>命令来判断xterm
是否在<tt class="docutils literal">root</tt>的名字空间下,如果所有的网卡都显示出来(包含<tt class="docutils literal">eth0</tt>),那他就是在<tt class="docutils literal">root</tt>下。</p>
<p>从 mininet 的 CLI中退出:</p>
<div class="highlight"><pre><span></span>mininet>exit
</pre></div>
<p>这样 mininet 的 CLI就自动关闭了。</p>
</div>
<div class="section" id="other-switch-types">
<h3>Other Switch Types 其他类型的交换机</h3>
<p>我们可以使用不同的交换机类型。例如:运行 user-space 交换机:</p>
<pre class="literal-block">
$ sudo mn --switch user --test iperf
</pre>
<p>值得注意的是这种交换机下,带宽相比于前面的内核态交换机要小的多。 如果做
ping
探测,也会有更高的延迟,这是因为现在的数据包需要从内核态转换到用户空间,消耗了更多的资源。</p>
<p>另一方面,用户空间的交换机会有一些新功能,如果交换机的性能不是关键问题是的时候。
在 Mininet 虚拟机中预装了另外一个交换机类型是
<tt class="docutils literal">Open vSwitch(OVS)</tt>,在<tt class="docutils literal">iperf</tt>测试中,带宽会比内核态交换机更大。</p>
<div class="highlight"><pre><span></span>$ sudo mn --switch ovsk --test iperf
</pre></div>
</div>
<div class="section" id="mininet-benchmark">
<h3>Mininet Benchmark</h3>
<p>To record the time to set up and tear down a topology, use test ‘none’:</p>
<div class="highlight"><pre><span></span>$ sudo mn --test none
</pre></div>
</div>
<div class="section" id="everything-in-its-own-namespace-user-switch-only">
<h3>Everything in its own Namespace (user switch only)</h3>
<p>默认情况下,主机都放在自己的命名空间,
而交换机和控制器的<tt class="docutils literal">root</tt>命名空间。
我们可以通过<tt class="docutils literal"><span class="pre">--innamespace</span></tt>参数来把交换机放在自己的名字空间中。</p>
<div class="highlight"><pre><span></span>$ sudo mn --innamespace --switch user
</pre></div>
<p>Instead of using loopback, the switches will talk to the controller
through a separately bridged control connection.
就其本身而言,这个选项是没有多大用处的,但它确实提供了如何分离不同交换机的例子。</p>
<p>请注意,此选项不会(截至12年11月19日)与Open vSwitch的工作。</p>
<p>需要注意的是这个选项在<tt class="docutils literal">Open vSwitch</tt>中是没法使用的(截至12年11月19日是没法使用)</p>
<div class="highlight"><pre><span></span>mininet>exit
</pre></div>
</div>
</div>
<div class="section" id="part-3-mininet-command-line-interface-cli-commands">
<h2>Part 3: Mininet Command-Line Interface (CLI) Commands</h2>
<p>第3部分:Mininet命令行界面(CLI)命令</p>
<div class="section" id="display-options">
<h3>Display Options</h3>
<p>我们可以通过启动一个最小拓扑结构,然后让他一直运行,来来查看 mininet 的
CLI 的选项列表:</p>
<div class="highlight"><pre><span></span>$ sudo mn
</pre></div>
<p>显示选项:</p>
<div class="highlight"><pre><span></span>mininet>help
</pre></div>
</div>
<div class="section" id="python-interpreter">
<h3>Python Interpreter</h3>
<p>如果在 Mininet CLI中的命令的第一个字符串是<tt class="docutils literal">py</tt>,那这个条命令会用
Python 来执行。 这对于扩展 Mininet,探测 mininet的内部工作机智都有帮助。
每个主机,交换机和控制器都有一个与之关联的对象。</p>
<p>在Mininet命令行下运行:</p>
<div class="highlight"><pre><span></span>mininet> py <span class="s1">'hello '</span> + <span class="s1">'world'</span>
</pre></div>
<p>打印 locals:</p>
<div class="highlight"><pre><span></span>mininet> py locals<span class="o">()</span>
<span class="o">{</span><span class="s1">'h2'</span>: <Host h2: h2-eth0:10.0.0.2 <span class="nv">pid</span><span class="o">=</span><span class="m">5166</span>> , <span class="s1">'net'</span>: <mininet.net.Mininet object at 0x7f7c47668ad0>, <span class="s1">'h1'</span>: <Host h1: h1-eth0:10.0.0.1 <span class="nv">pid</span><span class="o">=</span><span class="m">5165</span>> , <span class="s1">'c0'</span>: <OVSController c0: <span class="m">127</span>.0.0.1:6633 <span class="nv">pid</span><span class="o">=</span><span class="m">5157</span>> , <span class="s1">'s1'</span>: <OVSSwitch s1: lo:127.0.0.1,s1-eth1:None,s1-eth2:None <span class="nv">pid</span><span class="o">=</span><span class="m">5169</span>> <span class="o">}</span>
</pre></div>
<p>还可以通过 dir()函数来查看节点的方法和属性:</p>
<div class="highlight"><pre><span></span>mininet> py dir<span class="o">(</span>s1<span class="o">)</span>
<span class="o">[</span><span class="s1">'IP'</span>, <span class="s1">'MAC'</span>, <span class="s1">'TCReapply'</span>, <span class="s1">'__class__'</span>, <span class="s1">'__delattr__'</span>, <span class="s1">'__dict__'</span>, <span class="s1">'__doc__'</span>, <span class="s1">'__format__'</span>, <span class="s1">'__getattribute__'</span>, <span class="s1">'__hash__'</span>, <span class="s1">'__init__'</span>, <span class="s1">'__module__'</span>, <span class="s1">'__new__'</span>, <span class="s1">'__reduce__'</span>, <span class="s1">'__reduce_ex__'</span>, <span class="s1">'__repr__'</span>, <span class="s1">'__setattr__'</span>, <span class="s1">'__sizeof__'</span>, <span class="s1">'__str__'</span>, <span class="s1">'__subclasshook__'</span>, <span class="s1">'__weakref__'</span>, <span class="s1">'addIntf'</span>, <span class="s1">'attach'</span>, <span class="s1">'checkSetup'</span>, <span class="s1">'cleanup'</span>, <span class="s1">'cmd'</span>, <span class="s1">'cmdPrint'</span>, <span class="s1">'config'</span>, <span class="s1">'configDefault'</span>, <span class="s1">'connected'</span>, <span class="s1">'connectionsTo'</span>, <span class="s1">'controlIntf'</span>, <span class="s1">'controllerUUIDs'</span>, <span class="s1">'datapath'</span>, <span class="s1">'defaultDpid'</span>, <span class="s1">'defaultIntf'</span>, <span class="s1">'deleteIntfs'</span>, <span class="s1">'detach'</span>, <span class="s1">'dpctl'</span>, <span class="s1">'dpid'</span>, <span class="s1">'dpidLen'</span>, <span class="s1">'execed'</span>, <span class="s1">'failMode'</span>, <span class="s1">'fdToNode'</span>, <span class="s1">'inNamespace'</span>, <span class="s1">'inToNode'</span>, <span class="s1">'intf'</span>, <span class="s1">'intfIsUp'</span>, <span class="s1">'intfList'</span>, <span class="s1">'intfNames'</span>, <span class="s1">'intfs'</span>, <span class="s1">'isSetup'</span>, <span class="s1">'lastCmd'</span>, <span class="s1">'lastPid'</span>, <span class="s1">'linkTo'</span>, <span class="s1">'listenPort'</span>, <span class="s1">'monitor'</span>, <span class="s1">'name'</span>, <span class="s1">'nameToIntf'</span>, <span class="s1">'newPort'</span>, <span class="s1">'opts'</span>, <span class="s1">'outToNode'</span>, <span class="s1">'params'</span>, <span class="s1">'pexec'</span>, <span class="s1">'pid'</span>, <span class="s1">'pollOut'</span>, <span class="s1">'popen'</span>, <span class="s1">'portBase'</span>, <span class="s1">'ports'</span>, <span class="s1">'read'</span>, <span class="s1">'readbuf'</span>, <span class="s1">'readline'</span>, <span class="s1">'sendCmd'</span>, <span class="s1">'sendInt'</span>, <span class="s1">'setARP'</span>, <span class="s1">'setDefaultRoute'</span>, <span class="s1">'setHostRoute'</span>, <span class="s1">'setIP'</span>, <span class="s1">'setMAC'</span>, <span class="s1">'setParam'</span>, <span class="s1">'setup'</span>, <span class="s1">'shell'</span>, <span class="s1">'start'</span>, <span class="s1">'startShell'</span>, <span class="s1">'stdin'</span>, <span class="s1">'stdout'</span>, <span class="s1">'stop'</span>, <span class="s1">'terminate'</span>, <span class="s1">'waitOutput'</span>, <span class="s1">'waitReadable'</span>, <span class="s1">'waiting'</span>, <span class="s1">'write'</span><span class="o">]</span>
</pre></div>
<p>您可以通过使用help()函数读取在线文档,查看节点上可用的方法:</p>
<div class="highlight"><pre><span></span>mininet> py help<span class="o">(</span>h1<span class="o">)</span> <span class="c1">#(按`q`退出文档)</span>
</pre></div>
<p>You can also evaluate methods of variables:</p>
<div class="highlight"><pre><span></span><span class="n">mininet</span><span class="o">></span> <span class="n">py</span> <span class="n">h1</span><span class="o">.</span><span class="n">IP</span>
<span class="o"><</span><span class="n">bound</span> <span class="n">method</span> <span class="n">Host</span><span class="o">.</span><span class="n">IP</span> <span class="n">of</span> <span class="o"><</span><span class="n">Host</span> <span class="n">h1</span><span class="p">:</span> <span class="n">h1</span><span class="o">-</span><span class="n">eth0</span><span class="p">:</span><span class="mf">10.0</span><span class="o">.</span><span class="mf">0.1</span> <span class="n">pid</span><span class="o">=</span><span class="mi">5165</span><span class="o">></span> <span class="o">></span>
<span class="n">mininet</span><span class="o">></span> <span class="n">py</span> <span class="n">h1</span><span class="o">.</span><span class="n">IP</span><span class="p">()</span>
<span class="mf">10.0</span><span class="o">.</span><span class="mf">0.1</span>
</pre></div>
</div>
<div class="section" id="link-up-down">
<h3>Link Up/Down</h3>
<p>断开/联通链路,对于提供容错能力的测试非常有用。</p>
<p>比如端口<tt class="docutils literal">h1</tt>与<tt class="docutils literal">s1</tt>之间的连接:</p>
<div class="highlight"><pre><span></span>mininet> link s1 h1 down
</pre></div>
<p>你应该可以看到一个OpenFlow产生了一个的端口状态变化通知。</p>
<p>重新连接<tt class="docutils literal">h1</tt> <tt class="docutils literal">s1</tt>:</p>
<div class="highlight"><pre><span></span>mininet>link s1 h1 up
</pre></div>
</div>
<div class="section" id="xterm-display">
<h3>XTerm Display</h3>
<p>要显示<tt class="docutils literal">h1</tt> 与 <tt class="docutils literal">h2</tt>的 xterm:</p>
<div class="highlight"><pre><span></span>mininet> xterm h1 h2
</pre></div>
</div>
</div>
<div class="section" id="part-4-python-api-examples">
<h2>Part 4: Python API Examples</h2>
<p>在<a class="reference external" href="https://github.com/mininet/mininet/tree/master/examples">Mininet源代码</a>
中的示例目录包括如何使用Mininet的Python的API,
还有一些可能有用的代码并没有放到主代码库中。</p>
<div class="section" id="ssh-daemon-per-host">
<h3>SSH daemon per host</h3>
<p>这个栗子对于要在每台设备上启用 ssh 服务可能很有帮助。</p>
<div class="highlight"><pre><span></span>$ sudo ~/mininet/examples/sshd.py
</pre></div>
<p>在另外一个终端上,就可以ssh到任何主机并运行交互式命令:</p>
<div class="highlight"><pre><span></span>$ ssh <span class="m">10</span>.0.0.1
$ ping <span class="m">10</span>.0.0.2
...
$ <span class="nb">exit</span>
</pre></div>
<p>退出mininet:</p>
<div class="highlight"><pre><span></span><span class="nb">exit</span>
</pre></div>
<p>你会想重新看看那这些栗子可以看<a class="reference external" href="https://github.com/mininet/mininet/wiki/Introduction-to-Mininet">Introduction to
Mininet</a>
,里面介绍了 Python API。</p>
</div>
</div>
<div class="section" id="part-5-walkthrough-complete">
<h2>Part 5: Walkthrough Complete!</h2>
<p>恭喜!你已经完成了Mininet演练。之后可以随意尝试新的拓扑结构和控制器或查看源代码。</p>
<div class="section" id="next-steps-to-mastering-mininet">
<h3>Next Steps to mastering Mininet</h3>
<p>阅读 <a class="reference external" href="https://github.com/mininet/openflow-tutorial/wiki">OpenFlow
的教程</a></p>
<p>虽然你可以得到合理的利用Mininet的CLI,但是如果你掌握了 Python
API,Mininet会变得更加有用和强大的。 所以去看 <a class="reference external" href="https://github.com/mininet/mininet/wiki/Introduction-to-Mininet">Mininet
的文档</a></p>
<p>后面会解释如何远程控制 mininet(e.g. one running outside Mininet’s
control)。</p>
</div>
<div class="section" id="appendix-supplementary-information">
<h3>Appendix: Supplementary Information</h3>
<p>这些都不是必需的,但你会发现它们非常有用。</p>
</div>
<div class="section" id="using-a-remote-controller">
<h3>Using a Remote Controller</h3>
<p><em>注意:这一步是不是默认演练的一部分;如果你在mininet
之外运行一个控制器,这个附录将有些帮助。 在 OpenFLow
的教程中介绍了可以使用``controller --remote``参数来启动一个交换机,然后你可以用SDN
控制器比如``POX``, ``NOX``, ``Beacon`` 或者
``Floodlight``之类的来控制这个交换机。</em></p>
<p>当您启动Mininet网络,每个交换机可以连接到控制器,无论这个控制器在哪里。</p>
<p>如果你本地装有开发工具或者控制器,又或者你想在不同的物理机上面运行控制器,这种设置会非常方便。</p>
<p>如果你想尝试一下这个,只需要加上 ip 或者port 就可以:</p>
<div class="highlight"><pre><span></span>$ sudo mn --controller<span class="o">=</span>remote,ip<span class="o">=[</span>controller IP<span class="o">]</span>,port<span class="o">=[</span>controller listening port<span class="o">]</span>
</pre></div>
<p>例如,要运行POX的交换机,你可以这样做</p>
<div class="highlight"><pre><span></span>$ <span class="nb">cd</span> ~/pox
$ ./pox.py forwarding.l2_learning
</pre></div>
<p>在另一个窗口,启动Mininet连接到“远程”控制器(这实际上是在本地运行,但Mininet的控制范围之外):</p>
<div class="highlight"><pre><span></span>$ sudo mn --controller<span class="o">=</span>remote,ip<span class="o">=</span><span class="m">127</span>.0.0.1,port<span class="o">=</span><span class="m">6633</span>
</pre></div>
<p><em>注意,这些其实都是默认的IP地址和端口值。</em></p>
<p>如果你制造一些流量(如h1 ping h2),
你应该能够观察到窗口显示该交换机连接,而且输出了一些流量数据。</p>
<p>mininet虚拟机中已经预装了一些OpenFlow的控制器,你可以很轻松的就把这些东西搞起来。</p>
</div>
<div class="section" id="nox-classic">
<h3>NOX Classic</h3>
<p>使用 mininet 的默认<tt class="docutils literal">util/install.sh <span class="pre">-a</span></tt>并不会安装 NOX。
如果你想安装它,执行<tt class="docutils literal">sudo ~/mininet/util/install.sh <span class="pre">-x</span></tt></p>
<p>需要注意的是NOX Classic已被弃用,可能不会在将来得到支持。</p>
<p>早 NOX 中运行<tt class="docutils literal">pyswitch</tt>来做一个回归测试,
首先确认<tt class="docutils literal">NOX_CORE_DIR</tt>已经在环境变量中设置好。</p>
<p>首先验证NOX正在运行:</p>
<div class="highlight"><pre><span></span>$ <span class="nb">cd</span> <span class="nv">$NOX_CORE_DIR</span>
$ ./nox_core -v -i ptcp:
</pre></div>
<p>Ctrl-C来杀死 NOX 进程,然后运行与NOX 的 <tt class="docutils literal">pyswitch</tt>测试:</p>
<pre class="literal-block">
$ cd
$ sudo -E mn --controller=nox,pyswitch --test pingpair
</pre>
<p>注意,<tt class="docutils literal"><span class="pre">--controller</span></tt>选项具有方便的语法来向控制器类型指定选项
(在这种情况下,nox 运行 pyswitch。)</p>
<p>几秒钟之后,而NOX加载完成并且交换机之间相互连接,随后<tt class="docutils literal">ping</tt>。</p>
<p>注意,此时,<tt class="docutils literal">mn</tt>应该由<tt class="docutils literal">sudo <span class="pre">-E</span></tt>来调用,以保持NOX_CORE_DIR环境变量。
如果你是通过<tt class="docutils literal"><span class="pre">--controller</span> remote</tt>来远程启用的
nox,那就不需要加<tt class="docutils literal"><span class="pre">-E</span></tt>参数了。
或者,你可以改变的<tt class="docutils literal">/etc/sudoers</tt>文件,把</p>
<pre class="literal-block">
Defaults env_reset
</pre>
<p>修改成</p>
<pre class="literal-block">
Defaults !env_reset
</pre>
<p>使运行sudo的时候的环境变量的设置不会改变。</p>
</div>
</div>
Python 装饰器之 functools.wraps2013-07-08T00:00:00+08:002013-07-08T00:00:00+08:00SunisDowntag:sunisdown.me,2013-07-08:/python-zhuang-shi-qi-zhi-functoolswraps.html<p>在看 Bottle 代码中看见 functools.wraps 这种用法。</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">make_default_app_wrapper</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="sd">""" Return a callable that relays calls to the current default app. """</span>
<span class="n">a</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">Bottle</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
<span class="nd">@functools.wraps</span><span class="p">(</span><span class="nb">getattr</span><span class="p">(</span><span class="n">Bottle</span><span class="p">,</span> <span class="n">name</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">**</span><span class="n">ka</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">app</span><span class="p">(),</span> <span class="n">name</span><span class="p">)(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">**</span><span class="n">ka</span><span class="p">)</span>
<span class="k">return</span> <span class="n">wrapper</span>
</pre></div>
<p>之前没有看过,于是查文档了解了一下他的用处 先下定义: <strong>functools.wraps
是 ``装饰器``的``装饰器 …</strong></p><p>在看 Bottle 代码中看见 functools.wraps 这种用法。</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">make_default_app_wrapper</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="sd">""" Return a callable that relays calls to the current default app. """</span>
<span class="n">a</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">Bottle</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
<span class="nd">@functools.wraps</span><span class="p">(</span><span class="nb">getattr</span><span class="p">(</span><span class="n">Bottle</span><span class="p">,</span> <span class="n">name</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">**</span><span class="n">ka</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">app</span><span class="p">(),</span> <span class="n">name</span><span class="p">)(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">**</span><span class="n">ka</span><span class="p">)</span>
<span class="k">return</span> <span class="n">wrapper</span>
</pre></div>
<p>之前没有看过,于是查文档了解了一下他的用处 先下定义: <strong>functools.wraps
是 ``装饰器``的``装饰器``</strong></p>
<p>要明白 functiools.wraps 首先要明白 Python 的 <tt class="docutils literal">Decorator</tt></p>
<div class="section" id="decorator">
<h2>Decorator</h2>
<p>在以前的 Blog 中曾经简单写过 Decorator。这次需要讲的更细一些。</p>
<p><tt class="docutils literal">Decorator 通过返回包装对象实现间接调用,以此插入额外逻辑。</tt>是从老大那边偷来的哪里摘抄来的,应该算是言简意赅了。</p>
<div class="highlight"><pre><span></span><span class="nd">@dec2</span>
<span class="nd">@dec1</span>
<span class="k">def</span> <span class="nf">func</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">,</span> <span class="o">...</span><span class="p">):</span>
<span class="k">pass</span>
</pre></div>
<p>可以还原成</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">func</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">,</span> <span class="o">...</span><span class="p">):</span>
<span class="k">pass</span>
<span class="n">func</span> <span class="o">=</span> <span class="n">dec2</span><span class="p">(</span><span class="n">dec1</span><span class="p">(</span><span class="n">func</span><span class="p">))</span>
</pre></div>
<div class="highlight"><pre><span></span><span class="nd">@decomaker</span><span class="p">(</span><span class="n">argA</span><span class="p">,</span> <span class="n">argB</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">func</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">,</span> <span class="o">...</span><span class="p">):</span>
<span class="k">pass</span>
</pre></div>
<p>可以还原成</p>
<div class="highlight"><pre><span></span><span class="n">func</span> <span class="o">=</span> <span class="n">decomaker</span><span class="p">(</span><span class="n">argA</span><span class="p">,</span> <span class="n">argB</span><span class="p">,</span> <span class="o">...</span><span class="p">)(</span><span class="n">func</span><span class="p">)</span>
</pre></div>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">outer</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="o">...</span><span class="p">:</span> <span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="o">...</span><span class="p">:</span> <span class="k">print</span> <span class="s2">"before func"</span>
<span class="o">...</span><span class="p">:</span> <span class="n">ret</span> <span class="o">=</span> <span class="n">func</span><span class="p">()</span>
<span class="o">...</span><span class="p">:</span> <span class="k">return</span> <span class="n">ret</span> <span class="o">+</span> <span class="mi">1</span>
<span class="o">...</span><span class="p">:</span> <span class="k">return</span> <span class="n">inner</span> <span class="c1">#返回 inner 函数对象</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="nd">@outer</span> <span class="c1"># 解释器执⾏行 foo = outer(foo)</span>
<span class="o">...</span><span class="p">:</span> <span class="k">def</span> <span class="nf">foo</span><span class="p">():</span>
<span class="o">...</span><span class="p">:</span> <span class="k">return</span> <span class="mi">1</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="n">foo</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="o"><</span><span class="n">function</span> <span class="n">__main__</span><span class="o">.</span><span class="n">inner</span><span class="o">></span>
<span class="n">In</span> <span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="n">foo</span><span class="p">()</span>
<span class="n">before</span> <span class="n">func</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="mi">2</span>
</pre></div>
<p>这个过程中执行了下面几步</p>
<ol class="arabic simple">
<li>函数 foo 作为 装饰器 outer 的参数被传入</li>
<li>函数 inner 对 func 进行调用,然后装饰器 outer 返回 inner</li>
<li>原来的函数名 foo 关联到
inner,如上面的<tt class="docutils literal">foo <function __main__.inner></tt> 所示,调用 foo
时间上是在调用 inner</li>
</ol>
<p>装饰器不仅可以用函数返回包装对象,也可以是个类,不过这种方法太尼玛啰嗦,这里就不介绍了,想了解的自己去翻吧。下面我们写一个有点用处的
Decorator。 假想我们有个coordinate类,而且这个类提供了
<tt class="docutils literal">x, y</tt>坐标,而我们要对两个coordinate 对象进行计算。代码如下:</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Coordinate</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">x</span> <span class="o">=</span> <span class="n">x</span>
<span class="bp">self</span><span class="o">.</span><span class="n">y</span> <span class="o">=</span> <span class="n">y</span>
<span class="k">def</span> <span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"Coord: "</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Coordinate</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="o">+</span> <span class="n">b</span><span class="o">.</span><span class="n">x</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="o">+</span> <span class="n">b</span><span class="o">.</span><span class="n">y</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sub</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Coordinate</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="o">-</span> <span class="n">b</span><span class="o">.</span><span class="n">x</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="o">-</span> <span class="n">b</span><span class="o">.</span><span class="n">y</span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="n">one</span> <span class="o">=</span> <span class="n">Coordinate</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="n">two</span> <span class="o">=</span> <span class="n">Coordinate</span><span class="p">(</span><span class="mi">300</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="n">three</span> <span class="o">=</span> <span class="n">Coordinate</span><span class="p">(</span><span class="o">-</span><span class="mi">100</span><span class="p">,</span> <span class="o">-</span><span class="mi">100</span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="n">sub</span><span class="p">(</span><span class="n">one</span><span class="p">,</span> <span class="n">three</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="n">Coord</span><span class="p">:</span> <span class="p">{</span><span class="s1">'y'</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span> <span class="s1">'x'</span><span class="p">:</span> <span class="mi">200</span><span class="p">}</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">add</span><span class="p">(</span><span class="n">one</span><span class="p">,</span> <span class="n">three</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">Coord</span><span class="p">:</span> <span class="p">{</span><span class="s1">'y'</span><span class="p">:</span> <span class="mi">100</span><span class="p">,</span> <span class="s1">'x'</span><span class="p">:</span> <span class="mi">0</span><span class="p">}</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="n">sub</span><span class="p">(</span><span class="n">one</span><span class="p">,</span> <span class="n">two</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="n">Coord</span><span class="p">:</span> <span class="p">{</span><span class="s1">'y'</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="s1">'x'</span><span class="p">:</span> <span class="o">-</span><span class="mi">200</span><span class="p">}</span>
</pre></div>
<p>上面例子中的<tt class="docutils literal">sub(one, two)</tt>与<tt class="docutils literal">three</tt>都有负数,当我们把坐标限制在第一象限时,这两个就不符合我们的要求,用
Decorator 来做一个检测再好不过了</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">14</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="o">....</span><span class="p">:</span> <span class="k">def</span> <span class="nf">checker</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="o">....</span><span class="p">:</span> <span class="k">if</span> <span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="o"><</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="o"><</span> <span class="mi">0</span><span class="p">:</span>
<span class="o">....</span><span class="p">:</span> <span class="n">a</span> <span class="o">=</span> <span class="n">Coordinate</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="k">if</span> <span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="mi">0</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="k">if</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="mi">0</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span> <span class="k">if</span> <span class="n">b</span><span class="o">.</span><span class="n">x</span> <span class="o"><</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">b</span><span class="o">.</span><span class="n">y</span> <span class="o"><</span> <span class="mi">0</span><span class="p">:</span>
<span class="o">....</span><span class="p">:</span> <span class="n">b</span> <span class="o">=</span> <span class="n">Coordinate</span><span class="p">(</span><span class="n">b</span><span class="o">.</span><span class="n">x</span> <span class="k">if</span> <span class="n">b</span><span class="o">.</span><span class="n">x</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="mi">0</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">y</span> <span class="k">if</span> <span class="n">b</span><span class="o">.</span><span class="n">y</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="mi">0</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span> <span class="n">ret</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span> <span class="k">if</span> <span class="n">ret</span><span class="o">.</span><span class="n">x</span> <span class="o"><</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">ret</span><span class="o">.</span><span class="n">y</span> <span class="o"><</span><span class="mi">0</span><span class="p">:</span>
<span class="o">....</span><span class="p">:</span> <span class="n">ret</span> <span class="o">=</span> <span class="n">Coordinate</span><span class="p">(</span><span class="n">ret</span><span class="o">.</span><span class="n">x</span> <span class="k">if</span> <span class="n">ret</span><span class="o">.</span><span class="n">x</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="mi">0</span><span class="p">,</span> <span class="n">ret</span><span class="o">.</span><span class="n">y</span> <span class="k">if</span> <span class="n">ret</span><span class="o">.</span><span class="n">y</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="mi">0</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span> <span class="k">return</span> <span class="n">ret</span>
<span class="o">....</span><span class="p">:</span> <span class="k">return</span> <span class="n">checker</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">16</span><span class="p">]:</span> <span class="nd">@wrapper</span>
<span class="o">....</span><span class="p">:</span> <span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="o">....</span><span class="p">:</span> <span class="k">return</span> <span class="n">Coordinate</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="o">+</span> <span class="n">b</span><span class="o">.</span><span class="n">x</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="o">+</span> <span class="n">b</span><span class="o">.</span><span class="n">y</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">17</span><span class="p">]:</span> <span class="nd">@wrapper</span>
<span class="o">....</span><span class="p">:</span> <span class="k">def</span> <span class="nf">sub</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="o">....</span><span class="p">:</span> <span class="k">return</span> <span class="n">Coordinate</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="o">-</span> <span class="n">b</span><span class="o">.</span><span class="n">x</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="o">+</span> <span class="n">b</span><span class="o">.</span><span class="n">y</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">18</span><span class="p">]:</span> <span class="n">add</span><span class="p">(</span><span class="n">one</span><span class="p">,</span> <span class="n">three</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">18</span><span class="p">]:</span> <span class="n">Coord</span><span class="p">:</span> <span class="p">{</span><span class="s1">'y'</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span> <span class="s1">'x'</span><span class="p">:</span> <span class="mi">100</span><span class="p">}</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">19</span><span class="p">]:</span> <span class="n">one</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">19</span><span class="p">]:</span> <span class="n">Coord</span><span class="p">:</span> <span class="p">{</span><span class="s1">'y'</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span> <span class="s1">'x'</span><span class="p">:</span> <span class="mi">100</span><span class="p">}</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">20</span><span class="p">]:</span> <span class="n">sub</span><span class="p">(</span><span class="n">one</span><span class="p">,</span> <span class="n">two</span><span class="p">)</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">20</span><span class="p">]:</span> <span class="n">Coord</span><span class="p">:</span> <span class="p">{</span><span class="s1">'y'</span><span class="p">:</span> <span class="mi">400</span><span class="p">,</span> <span class="s1">'x'</span><span class="p">:</span> <span class="mi">0</span><span class="p">}</span>
</pre></div>
<p>这样,只计算的函数<tt class="docutils literal">add</tt>与<tt class="docutils literal">sub</tt>前面加一个 Decorator
就可以完成坐标的校验。比在函数内实现要优雅一些。</p>
<p>Decorator 还可以为类增加额外的成员,</p>
<div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">21</span><span class="p">]:</span> <span class="k">def</span> <span class="nf">hello</span><span class="p">(</span><span class="bp">cls</span><span class="p">):</span>
<span class="o">....</span><span class="p">:</span> <span class="bp">cls</span><span class="o">.</span><span class="n">hello</span> <span class="o">=</span> <span class="nb">staticmethod</span><span class="p">(</span><span class="k">lambda</span><span class="p">:</span> <span class="s2">"HELLO"</span><span class="p">)</span>
<span class="o">....</span><span class="p">:</span> <span class="k">return</span> <span class="bp">cls</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">22</span><span class="p">]:</span> <span class="nd">@hello</span>
<span class="o">....</span><span class="p">:</span> <span class="k">class</span> <span class="nc">World</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span><span class="k">pass</span>
<span class="o">....</span><span class="p">:</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">23</span><span class="p">]:</span> <span class="n">World</span><span class="o">.</span><span class="n">hello</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">23</span><span class="p">]:</span> <span class="o"><</span><span class="n">function</span> <span class="n">__main__</span><span class="o">.<</span><span class="k">lambda</span><span class="o">>></span>
<span class="n">In</span> <span class="p">[</span><span class="mi">24</span><span class="p">]:</span> <span class="n">World</span><span class="o">.</span><span class="n">hello</span><span class="p">()</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">24</span><span class="p">]:</span> <span class="s1">'HELLO'</span>
</pre></div>
<div class="section" id="functools-wraps">
<h3>functools.wraps</h3>
<p>我们在使用 Decorator 的过程中,难免会损失一些原本的功能信息。直接拿
stackoverflow 里面的栗子</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">logged</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">with_logging</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">print</span> <span class="n">func</span><span class="o">.</span><span class="vm">__name__</span> <span class="o">+</span> <span class="s2">" was called"</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">with_logging</span>
<span class="nd">@logged</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="sd">"""does some math"""</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="n">x</span> <span class="o">*</span> <span class="n">x</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="sd">"""does some math"""</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="n">x</span> <span class="o">*</span> <span class="n">x</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">logged</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">24</span><span class="p">]:</span> <span class="n">f</span><span class="o">.</span><span class="vm">__name__</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">24</span><span class="p">]:</span> <span class="n">with_logging</span>
</pre></div>
<p>而functools.wraps 则可以将原函数对象的指定属性复制给包装函数对象, 默认有
<tt class="docutils literal">__module__</tt>、<tt class="docutils literal">__name__</tt>、<tt class="docutils literal">__doc__</tt>,或者通过参数选择。代码如下:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="n">wraps</span>
<span class="k">def</span> <span class="nf">logged</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="nd">@wraps</span><span class="p">(</span><span class="n">func</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">with_logging</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">print</span> <span class="n">func</span><span class="o">.</span><span class="vm">__name__</span> <span class="o">+</span> <span class="s2">" was called"</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">with_logging</span>
<span class="nd">@logged</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="sd">"""does some math"""</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="n">x</span> <span class="o">*</span> <span class="n">x</span>
<span class="k">print</span> <span class="n">f</span><span class="o">.</span><span class="vm">__name__</span> <span class="c1"># prints 'f'</span>
<span class="k">print</span> <span class="n">f</span><span class="o">.</span><span class="vm">__doc__</span> <span class="c1"># prints 'does some math'</span>
</pre></div>
</div>
</div>
python之 unicode_literals2013-06-03T00:00:00+08:002013-06-03T00:00:00+08:00SunisDowntag:sunisdown.me,2013-06-03:/pythonzhi-unicode_literals.html<p>在邮件列表中看见有人在讨论github上面的内容,然后看见一阁大牛说"看见别人的python代码没有from
__future__ import unicode_literals就想fork改掉…"</p>
<p>之前代码中确实没有加这个,嗯,受教啦</p>
<pre class="literal-block">
from __future__ import unicode_literals
</pre>
<p>是python2.6
之后新增加的新特性,可以使得所有的字符串文本成为Unicode字符串.</p>
<pre class="literal-block">
In [4]: from __future__ import unicode_literals
In [5]: a = '你好'
In [6]: a
Out[6]: u'\u4f60\u597d'
</pre>
<p>在邮件列表中看见有人在讨论github上面的内容,然后看见一阁大牛说"看见别人的python代码没有from
__future__ import unicode_literals就想fork改掉…"</p>
<p>之前代码中确实没有加这个,嗯,受教啦</p>
<pre class="literal-block">
from __future__ import unicode_literals
</pre>
<p>是python2.6
之后新增加的新特性,可以使得所有的字符串文本成为Unicode字符串.</p>
<pre class="literal-block">
In [4]: from __future__ import unicode_literals
In [5]: a = '你好'
In [6]: a
Out[6]: u'\u4f60\u597d'
</pre>
Python 之 enumerate2013-06-01T00:00:00+08:002013-06-01T00:00:00+08:00SunisDowntag:sunisdown.me,2013-06-01:/python-zhi-enumerate.html<p>今天看到enumerate 这个函数,然后Google之,发现limodou大牛写过enumerate了~</p>
<p>enumerate 主要是用再类似的场景:</p>
<div class="highlight"><pre><span></span>In <span class="o">[</span><span class="m">20</span><span class="o">]</span>:for i in xrange<span class="o">(</span>len<span class="o">(</span>a<span class="o">))</span>:
print i, a<span class="o">[</span>i<span class="o">]</span>
....:
<span class="m">0</span> <span class="m">1</span>
<span class="m">1</span> <span class="m">2</span>
<span class="m">2</span> <span class="m">3</span>
<span class="m">3</span> alkj
</pre></div>
<p>使用enumerate可以简化代码为:</p>
<div class="highlight"><pre><span></span>In <span class="o">[</span><span class="m">21</span><span class="o">]</span>: <span class="k">for</span> i, j in enumerate<span class="o">(</span>a<span class="o">)</span>:
....: print i, j
....:
<span class="m">0</span> <span class="m">1</span>
<span class="m">1</span> <span class="m">2</span>
<span class="m">2</span> <span class="m">3</span>
<span class="m">3</span> alkj
<span class="o">{</span>i:j <span class="k">for</span> i …</pre></div><p>今天看到enumerate 这个函数,然后Google之,发现limodou大牛写过enumerate了~</p>
<p>enumerate 主要是用再类似的场景:</p>
<div class="highlight"><pre><span></span>In <span class="o">[</span><span class="m">20</span><span class="o">]</span>:for i in xrange<span class="o">(</span>len<span class="o">(</span>a<span class="o">))</span>:
print i, a<span class="o">[</span>i<span class="o">]</span>
....:
<span class="m">0</span> <span class="m">1</span>
<span class="m">1</span> <span class="m">2</span>
<span class="m">2</span> <span class="m">3</span>
<span class="m">3</span> alkj
</pre></div>
<p>使用enumerate可以简化代码为:</p>
<div class="highlight"><pre><span></span>In <span class="o">[</span><span class="m">21</span><span class="o">]</span>: <span class="k">for</span> i, j in enumerate<span class="o">(</span>a<span class="o">)</span>:
....: print i, j
....:
<span class="m">0</span> <span class="m">1</span>
<span class="m">1</span> <span class="m">2</span>
<span class="m">2</span> <span class="m">3</span>
<span class="m">3</span> alkj
<span class="o">{</span>i:j <span class="k">for</span> i,j in enumerate<span class="o">(</span><span class="s1">'abcdef'</span><span class="o">)}</span>
</pre></div>