multithreading - python 3多线程输出到CSV文件( 空白)

  显示原文与译文双语对照的内容
0 0

我是 python的新手,我已经从一个教程中运行了这个多线程。 不确定是否 goo 。

我想归档的内容: ping主机名列表并返回或者下移。 将结果写入csv文件

这里文件当前执行的操作为: ping主机名列表并返回或者下移。 它创建的csv文件是空的,不会将任何结果写入它。

我已经做了一些测试,并发现在串行代码上,multithreadin的超过了 16倍的速度。 我正在做大量的ping,大约 9000,希望尽快返回。

你能让我知道我的csv部分哪里出错了。


import threading
from queue import Queue
import time
import subprocess as sp
import csv

# lock to serialize console output
lock = threading.Lock()

def do_work(item):
 #time.sleep(1) # pretend to do some lengthy work.
 # Make sure the whole print completes or threads can mix up output in one line.

 status,result = sp.getstatusoutput("ping -n 3" + str(item))
 if status == 0:
 result = 'Up'
 else:
 result = 'Down'

 with lock:
 output.writerow({'hostname': item,'status': result})
 array.append({'hostname': item,'status': result})
 print(threading.current_thread().name,item,result)

# The worker thread pulls an item from the queue and processes it
def worker():
 while True:
 item = q.get()
 do_work(item)
 q.task_done()

# Create the queue and thread pool.
q = Queue()
for i in range(100):
 t = threading.Thread(target=worker)
 t.daemon = True # thread dies when main thread (only non-daemon thread) exits.
 t.start()

array = []
# stuff work items on the queue (in this case, just a number).
start = time.perf_counter()

headers = ['status','hostname']
output = csv.DictWriter(open('host-export.csv','w'), delimiter=',', lineterminator='n', fieldnames=headers)
output.writeheader()

txt = open("hosts.txt", 'r', encoding="utf8")
for line in txt:
 q.put(line,array)

q.join() # block until all tasks are done

#"Work" took. 1 seconds per task.
# 20 tasks serially would be 2 seconds.
# With 4 threads should be about. 5 seconds (contrived because non-CPU intensive"work")
print(array)

print('time:',time.perf_counter() - start)

我还添加了大量的csv语句,可以能只是无法访问函数中的csv对象,但也没有如下所示。


headers = ['status','hostname']
output = csv.DictWriter(open('host-export.csv','w'), delimiter=',', lineterminator='n', fieldnames=headers)
output.writeheader()
output.writerows(array)

时间: 原作者:

0 0

我把自己做的错误 fiugured 。 我没有关闭文件连接,所以它没有写入文件。

下面是我现在使用的代码来访问我的csv文件。


fieldnames = ['ip', 'dns', 'pings'] #headings 
test_file = open('test2-p.csv','w', newline='') #open file
csvwriter = csv.DictWriter(test_file, delimiter=',', fieldnames=fieldnames) #set csv writing settings
csvwriter.writeheader() #write csv headings 
for row in rows: #write to csv file
 csvwriter.writerow(row)
test_file.close()

原作者:
...