Latest web development tutorials

Python3 Multithreading

Multiple threads simultaneously executed is similar to a number of different programs, multithreaded operation has the following advantages:

  • Using threads can occupy long program in the task into the background to deal with.
  • The user interface may be more attractive, so for example the user clicks a button to trigger some event handling, you can pop a progress bar to show the progress of the process
  • Running speed could accelerate
  • On realized some tasks, such as waiting for user input, document literacy and network to send and receive data, the thread is more useful. In this case, we can free up some valuable resources, such as memory usage and so on.

Threads in the implementation process and the process is different. Each has a separate thread running entry, exit sequence and procedures for the implementation of the order. But the thread is not able to perform independently, it must exist according to the application, providing multiple execution threads controlled by the application.

Each thread has his own set of CPU registers, called the context of the thread, the thread context reflects the last run of the thread CPU register state.

Instruction pointer and stack pointer registers are the two most important thread context registers, the thread always gets run in the context of the process, these addresses are used to mark the process that owns the thread of memory address space.

  • Thread can be preempted (interrupted).
  • In other threads are running, the thread can be held in abeyance (also known as sleep) - this is the thread of concessions.

Thread can be divided into:

  • Kernel threads: the operating system kernel created and destroyed.
  • User threads: kernel support and does not require the user program implemented thread.

Python3 common thread two modules:

  • _thread
  • threading (recommended)

thread module has been abandoned. Users can use the threading module instead. So, in Python3 can no longer use the "thread" module. For compatibility, Python3 thread will rename "_thread".


Start learning Python threads

Python threads used in two ways: with a function or class to wrap the thread object.

Functional: Call _thread module start_new_thread () function to generate a new thread. The syntax is as follows:

_thread.start_new_thread ( function, args[, kwargs] )

Parameter Description:

  • function - thread function.
  • args - the thread parameters passed to the function, he must be a tuple type.
  • kwargs - Optional.

Example:

#!/usr/bin/python3

import _thread
import time

# 为线程定义一个函数
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print ("%s: %s" % ( threadName, time.ctime(time.time()) ))

# 创建两个线程
try:
   _thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   _thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print ("Error: 无法启动线程")

while 1:
   pass

The above program output results are as follows:

Thread-1: Wed Apr  6 11:36:31 2016
Thread-1: Wed Apr  6 11:36:33 2016
Thread-2: Wed Apr  6 11:36:33 2016
Thread-1: Wed Apr  6 11:36:35 2016
Thread-1: Wed Apr  6 11:36:37 2016
Thread-2: Wed Apr  6 11:36:37 2016
Thread-1: Wed Apr  6 11:36:39 2016
Thread-2: Wed Apr  6 11:36:41 2016
Thread-2: Wed Apr  6 11:36:45 2016
Thread-2: Wed Apr  6 11:36:49 2016

After executing the above process can press ctrl-c to exit.


Threading module

Python3 provide support for threading through two standard library _thread and threading.

_thread provides a low-level, the original thread and a simple lock, it is compared to the threading function module is still relatively limited.

Other methods in addition to threading module contains _thread module all the methods, but also to provide:

  • threading.currentThread (): Returns the current thread variable.
  • threading.enumerate (): Returns a running thread list contains. Refers running thread started before the end, it does not include the thread before starting and after the termination.
  • threading.activeCount (): Returns the number of threads that are running, and len (threading.enumerate ()) have the same result.

In addition to the use of methods, threading module also provides a Thread class to handle the thread, Thread class provides the following methods:

  • run (): to indicate the method of active threads.
  • start (): start thread activity.

  • join ([time]): Wait until the thread is aborted.This blocks the calling thread until the thread's join () method is called suspension - normal exit or throw an unhandled exception - or optional timeout occurs.
  • isAlive (): Returns the thread is active.
  • getName (): Returns the thread name.
  • setName (): Set the thread name.

Use threading module to create a thread

We can be created directly inherited from threading.Thread a new sub-class, after the call start and instantiate () method to start a new thread, which it calls the thread's run () method:

#!/usr/bin/python3

import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print ("开始线程:" + self.name)
        print_time(self.name, self.counter, 5)
        print ("退出线程:" + self.name)

def print_time(threadName, delay, counter):
    while counter:
        if exitFlag:
            threadName.exit()
        time.sleep(delay)
        print ("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

# 创建新线程
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# 开启新线程
thread1.start()
thread2.start()
thread1.join()
thread2.join()
print ("退出主线程")

The above program execution results are as follows;

开始线程:Thread-1
开始线程:Thread-2
Thread-1: Wed Apr  6 11:46:46 2016
Thread-1: Wed Apr  6 11:46:47 2016
Thread-2: Wed Apr  6 11:46:47 2016
Thread-1: Wed Apr  6 11:46:48 2016
Thread-1: Wed Apr  6 11:46:49 2016
Thread-2: Wed Apr  6 11:46:49 2016
Thread-1: Wed Apr  6 11:46:50 2016
退出线程:Thread-1
Thread-2: Wed Apr  6 11:46:51 2016
Thread-2: Wed Apr  6 11:46:53 2016
Thread-2: Wed Apr  6 11:46:55 2016
退出线程:Thread-2
退出主线程

Thread Synchronization

If multiple threads of a common data modification, unpredictable results might occur, in order to ensure the accuracy of the data, we need to synchronize multiple threads.

Thread object using the Lock and Rlock can achieve a simple thread synchronization, the two objects have methods acquire and release methods, for those who need each allowing only one thread operating data, it can be placed in operation acquire and release methods of the between. as follows:

Multithreading advantage is that it can simultaneously run multiple tasks (at least feels like this). But when threads need to share data, there may be no data synchronization issues.

Consider a case: a list of all elements are zero, the thread "set" from back to front all the elements into one, and the thread "print" from front to back in charge of reading the list and print.

Then, the thread may be "set" began to change when the thread "print" it to print a list, it would be half the output of half a 0, which is the data is not synchronized. To avoid this situation, we introduce the concept of the lock.

Lock has two states - locked and unlocked. Whenever a thread such as "set" to access the shared data, you must first obtain the lock; if you already have another thread, such as "print" to get locked up, then let the thread "set" pause, which is synchronous blocking; wait until the thread " print "access is completed, after the release of the lock, let the thread" set "to continue.

After this process, when you print a list of all output either 0 or 1 full output, no longer appears half embarrassment 0 1 half.

Example:

#!/usr/bin/python3

import threading
import time

class myThread (threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print ("开启线程: " + self.name)
        # 获取锁,用于线程同步
        threadLock.acquire()
        print_time(self.name, self.counter, 3)
        # 释放锁,开启下一个线程
        threadLock.release()

def print_time(threadName, delay, counter):
    while counter:
        time.sleep(delay)
        print ("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

threadLock = threading.Lock()
threads = []

# 创建新线程
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# 开启新线程
thread1.start()
thread2.start()

# 添加线程到线程列表
threads.append(thread1)
threads.append(thread2)

# 等待所有线程完成
for t in threads:
    t.join()
print ("退出主线程")

The above program, the output is:

开启线程: Thread-1
开启线程: Thread-2
Thread-1: Wed Apr  6 11:52:57 2016
Thread-1: Wed Apr  6 11:52:58 2016
Thread-1: Wed Apr  6 11:52:59 2016
Thread-2: Wed Apr  6 11:53:01 2016
Thread-2: Wed Apr  6 11:53:03 2016
Thread-2: Wed Apr  6 11:53:05 2016
退出主线程

Thread priority queue (Queue)

Python's Queue module provides synchronization, thread-safe queue classes, including FIFO (first in first out) queue Queue, LIFO (last in, first out) queue LifoQueue, and priority queue PriorityQueue.

These queues are implemented locking primitives can be used directly in a multithreaded, you can use the queue to achieve synchronization between threads.

Queue module commonly used methods:

  • Queue.qsize () returns the size of the queue
  • Queue.empty () if the queue is empty, returns True, False and vice versa
  • Queue.full () If the queue is full, return True, False and vice versa
  • Corresponding to the size and maxsize Queue.full
  • Queue.get ([block [, timeout]]) Gets the queue, timeout waiting time
  • Queue.get_nowait () rather Queue.get (False)
  • Queue.put (item) write queue, timeout waiting time
  • Queue.put_nowait (item) quite Queue.put (item, False)
  • Queue.task_done () after the completion of a work, Queue.task_done () function sends a signal to the task has been completed queue
  • Queue.join () really means until the queue is empty, then perform other operations

Example:

#!/usr/bin/python3

import queue
import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, q):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.q = q
    def run(self):
        print ("开启线程:" + self.name)
        process_data(self.name, self.q)
        print ("退出线程:" + self.name)

def process_data(threadName, q):
    while not exitFlag:
        queueLock.acquire()
        if not workQueue.empty():
            data = q.get()
            queueLock.release()
            print ("%s processing %s" % (threadName, data))
        else:
            queueLock.release()
        time.sleep(1)

threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]
queueLock = threading.Lock()
workQueue = queue.Queue(10)
threads = []
threadID = 1

# 创建新线程
for tName in threadList:
    thread = myThread(threadID, tName, workQueue)
    thread.start()
    threads.append(thread)
    threadID += 1

# 填充队列
queueLock.acquire()
for word in nameList:
    workQueue.put(word)
queueLock.release()

# 等待队列清空
while not workQueue.empty():
    pass

# 通知线程是时候退出
exitFlag = 1

# 等待所有线程完成
for t in threads:
    t.join()
print ("退出主线程")

The results of the above program:

开启线程:Thread-1
开启线程:Thread-2
开启线程:Thread-3
Thread-3 processing One
Thread-1 processing Two
Thread-2 processing Three
Thread-3 processing Four
Thread-1 processing Five
退出线程:Thread-3
退出线程:Thread-2
退出线程:Thread-1
退出主线程