How To Create a Self-Restartable Application, from CodeProject.

This article describes the way to add restarting support to your Win32 applications. Key pointer is as followings:

Old Instance Create a Mutex and create a new process with command retrieved by GetModuleFileName().

g_RA_hMutexOtherRestarting = ::CreateMutex( NULL, TRUE, RA_MUTEX_OTHER_RESTARTING);
TCHAR szAppPath[MAX_PATH] = {0};
::GetModuleFileName(NULL, szAppPath, MAX_PATH);
CreateProcess(NULL, szAppPath, NULL, NULL, FALSE, 0, NULL, NULL, &si, &pi);

New Instance wait for this mutex for release which means old instance has been dead.

// Releasing mutex signal that process finished
DWORD dwWaitResult = WaitForSingleObject(g_RA_hMutexOtherRestarting, 0);
if (dwWaitResult == WAIT_TIMEOUT)
::ReleaseMutex(g_RA_hMutexOtherRestarting);
::CloseHandle(g_RA_hMutexOtherRestarting);

Popularity: 32% [?]

Van Jacobson’s network channels, from LWN.net.

Van Jacobson 在今年一月底的 linux.conf.au 上展示了他的 network channels 的 idea, 然後引發了一系列的討論. 雖然有不少障礙要克服, 不過設計上應該有蠻多學習的地方 ;-)

從 Van 的 slides [PDF] 上講到, 傳統的 Networking stack, 到現在的實作已經變成了 “Standard Model”. 在 Linux kernel 實作上大概如下 (From Van’s slides):

當封包被網路卡收到, kernel 會收到 interrupt, 然後呼叫 ISR, 或是有註冊 NET_RX_SOFTIRQ 的 softirq handler (一般應該是 driver; 另, 這裡也可能是 tasklet), 會根據自己的硬體運作方法把 packer 收下來組成 skb, 然後呼叫 net/core/dev.c:netif_receive_skb(). netif_receive_skb() 這裡會檢查 payload 然後解多工. 舉個例子, IP 應該會送到 net/ipv4/ip_input.c:ip_recv() 去. 當然, 後面的 Socket (更高的像是 UDP/TCP Layer) 也是會參考這個 skb, 當然就大家所知, TCP 甚至還要組成 Byte-Stream.

這樣的設計當可能有一些缺點, 為了不失原意, 我摘錄原文如下:

• Passing network packets through multiple layers of the kernel.
  When a packet arrives, the network card’s interrupt handler begins the task of feeding the packet to the kernel. The remainder of the work may well be performed at software interrupt level within the driver (in a tasklet, perhaps). The core network processing happens in another software interrupt. Copying the data (an expensive operation in itself) to the application happens in kernel context. Finally the application itself does something interesting with the data. The context changes are expensive, and if any of these changes causes the work to move from one CPU to another, a big cache penalty results. Much work has been done to improve CPU locality in the networking subsystem, but much remains to be done.
• Locking is expensive.
  Taking a lock requires a cross-system atomic operation and moves a cache line between processors. Locking costs have led to the development of lock-free techniques like seqlocks and read-copy-update (RCU), but the the networking stack (like the rest of the kernel) remains full of locks.
• The networking code makes extensive use of queues implemented with doubly-linked lists.
  These lists have poor cache behavior since they require each user to make changes (and thus move cache lines) in multiple places.

因此, 為了要增加 networking scalability, 首要就是要消除 locking 和 shared data. Van 利用 end-to-end principle 來達成這個目的. 也就是說, 盡可能的讓資料交給 application, 而不要在 kernel 任何地方等待. 於是他設計了 net channel — 一個 circular buffer (應該是 Circular FIFO queue implemented by Array) 用來取代 skb 和目前用在 networking stack 的 queue. 比方說, 原先需要用 softirq 的地方 (driver -> socket), 改用 netchannel, locking 數都明顯的下降, 進而提高 scalability.

但是, 這個方法遇到的第一個問題就是, 讓資料從 packet 一條鞭的到 application, 會把 netfilter 的原先 hook 的點變相的消除, 為了加回來 netfilter 的 support, 利用得到的優勢便蕩然無存了.

不過我有一點不懂的是, 一個還算簡單的 circular buffer, 為何是 “Cache aware, cache friendly queue” 呢? 是因為用 Array implement 這樣嗎? :p

Popularity: 20% [?]

Untwisting Python Network Programming, from O’ReillyNet.

This article introduces basic client-side networking using both core Python modules and the Twisted framework. For its example, I will show how to send, receive, and delete emails, and conduct Telnet sessions.

I have written two functionally equivalent examples, one using the core modules (mail-core.py) and another using Twisted (mail-twisted.py), with both start, stop, and interact with a server to process emails.

Here is outline:

core Python modules

• Sending Mails with smtplib
• Retrieving Emails with poplib
• Conducting Telnet with telnetlib

Twisted Way

• Sending Mails the Twisted Way
• Retrieving Mails with Twisted
• Doing Telnet with Twisted

When to Be Twisted?

The two functionally equivalent programs, one using Python core modules and the other using the Twisted framework, significantly differ from each other in terms of programming style and the amount of code. Then when should you use either of the two options?

For basic programs such as the command-line client of this example, the Python core networking modules are more desirable due to the simplicity and performance advantages. However, most real-world networking programs are very complex, and Twisted’s asynchronous programming model is more effective.

For example, BitTorrent, the popular peer-to-peer file sharing client that performs massive parallel downloading of data chunks from different sources, uses Twisted. Twisted also works well in programs with graphical user interface (GUI), because its asynchronous nature fits more seamlessly with the event-driven programming models of modern GUI frameworks. In fact, Twisted has integration with popular GUI frameworks including PyGTK, Qt, Tkinter, WxPython, and Win32.

The other area where Twisted shines is in server programming. A typical network server uses multithreading so that it can handle multiple clients concurrently. The asynchronous mechanism of Twisted alleviates the creation and handling of threads by server programs. In addition, Twisted provides several protocols on which to build new networking services, enabling rapid development of complex servers. One such project is Quotient, which adopts Twisted to build a multiprotocol messaging server that supports a variety of protocols and services including SMTP, POP3, IMAP, webmail, and SIP.

Popularity: 17% [?]

Although mordem md5sum(1) of coreutils in Linux can acknowledge the md5sum of FreeBSD format, I still want to convert FreeBSD format to Linux one via python. Here is original file:

md5sum generated by md5(1) of FreeBSD

MD5 (Thunderbird-1.5.0.4.exe) = 3b862cd35571ecec3de5e34112bf3f15
MD5 (Thunderbird-1.5.0.5.exe) = 3b93bd5529e2a81054514f57dc4b3916

And I want to convert to:

md5sum generated by md5sum(1) of Linux

3b862cd35571ecec3de5e34112bf3f15 Thunderbird-1.5.0.4.exe
3b93bd5529e2a81054514f57dc4b3916 Thunderbird-1.5.0.5.exe

The sample code is quite simple:

# Open file named md5sum.txt
hFile = open ("md5sum.txt", "rU")

# Read whole content into a List file handle
aListLines = [szLine for szLine in hFile]

# Loading completed, then close file handle
hFile.close()

# Reset output list
aListOutputLines = []

# Process each line one by one
for szLine in aListLines:

....# split 'szFile = szHash' by three charactes of ' = '
....szFile, szHash = szLine.split(' = ')

....# Strip what we don't need and add two characters of space
....aListOutputLines.append(szHash.rstrip('\n') \
........+ ' ' * 2 \
........+ szFile.lstrip('MD5 (').rstrip(')'))

# Now aListOutputLines is final result, hence we open file to write next
hFile = open("md5sum.txt.new", "w")

# Process each line one by one
for szLine in aListOutputLines:
....hFile.write("%s\n" % szLine)

# Closing a handle is a good habit, it's unnecessary though
hFile.close()

It’s all ;-)

By the way, the meaning of open (”md5sum.txt”, “rU”) is called open file with Universal Newline.

Popularity: 16% [?]

Key Porting Differences from LinuxThreads to NPTL, from OSNews.

The LinuxThreads project originally brought multithreading to Linux, but didn’t conform to POSIX threading standards.

The introduction of Native POSIX Thread Library (NPTL) however, overcame many of these disadvantages.

This article describes some of the differences between these two Linux threading models for developers who may need to port their applications or who simply want to understand where the differences lie.

Popularity: 8% [?]

在 Windows 上, 就 Diff 和 Merge 的 Tool 來說, Araxis Merge 算是一絕, 那在 Linux 上就沒有這麼好用的 Tool 了. 不過今天才發現在 sourceforce.net 上有個 Meld 的 project, 可能也是 Gnome2 上 diff and merge 的殺手級 tool.

這個是比對檔案的 screenshot:

這個是比對目錄的 screenshot:

安裝的方法如果是 Fedora, 可以參考我寫的這篇 [RedHat] How to Fetch RPM? #1 - Fetch Manually. 它在 Fedora Extras 裡.

Popularity: 23% [?]

CString Management, from CodeProject.

CStrings are a useful data type. They greatly simplify a lot of operations in MFC, making it much more convenient to do string manipulation. However, there are some special techniques to using CStrings, particularly hard for people coming from a pure-C background to learn. This essay discusses some of these techniques.

Much of what you need to do is pretty straightforward. This is not a complete tutorial on CStrings, but captures the most common basic questions

From this article, we can learn how to effectively use CStrings.

Popularity: 33% [?]