python 操作excel工具及xlsxwriter处理数据量大的方法

  • Z时代
  • 2024-01-10
  • 分类：综合

python

前言

- 在我们的web项目中一些图表的预览页面常常伴随着数据导出至excel的功能。在python中我们常常使用xlrd、xlwt、xlutils、openpyxl、xlsxwriter来达成我们的需求。

方法简介

一.xlsxwriter 只写 (推荐)

注：xlsxwriter 只能写，不能读，但功能强大，可进行添加图表等复杂操作，详见https://xlsxwriter.readthedocs.io/

1.本地生成(数据量大时建议保存在本地)

简单的例子

import xlsxwriter as xw
#新建excel，这样在本地会生成这个文件，下文介绍与写入内存中
workbook  = xw.Workbook(\'myexcel.xlsx\')
#新建工作薄
worksheet = workbook.add_worksheet()
#写入数据
worksheet.wirte(\'A1\',1)
#关闭保存
workbook.close()

插入图表的例子

import xlsxwriter
def get_chart(series):
    chart = workbook.add_chart({\'type\': \'line\'})
    for ses in series:
        name = ses["name"]
        values = ses["values"]
        chart.add_series({
            \'name\': name,
            \'categories\': \'A2:A10\',
            \'values\':values
        })
    chart.set_size({\'width\': 600, \'height\': 250})
    return chart
if __name__ == \'__main__\':
    workbook = xlsxwriter.Workbook(\'xxxx.xlsx\')
    worksheet = workbook.add_worksheet("工作簿1")
    headings = [\'日期\', \'平均值\']
    worksheet.write_row(\'A1\', headings)
    index=0
    for row in range(1,10):
        for com in [0,1]:
            worksheet.write(row,com,index)
            index+=1
    series = [{"name":"平均值","values":"B2:B10"}]
    chart = get_chart(series)
    chart.set_title ({\'name\': \'每日页面分享数据\'}) 
    worksheet.insert_chart(\'H7\', chart)
    workbook.close()

2.内存生成(web中常用直接返回响应数据)

python3 中操作需 借助 BytesIO， Python2 需借助 StringIO

场景 python3 + django 实现excel导出

from io import BytesIO
import xlsxwriter as xw
from django.http import FileResponse, HttpResponse
def export(request, *arg, **kwargs):
    out_put = BytesIO()
    #新建excel
    workbook  = xw.Workbook(out_put)
    #新建工作薄
    worksheet = workbook.add_worksheet()
    #写入数据
    worksheet.wirte(\'A1\',1)
    #关闭保存
    workbook.close()
    # make 响应对象
    response = HttpResponse()
    response[\'Content-Type\'] = "application/octet-stream"
    disposition = \'attachment;filename={}.xlsx\'.format(escape_uri_path(file_name))
    response[\'Content-Disposition\'] = disposition
    response.write(out_put.getvalue())
    return response

3.xlsxwriter处理大数据

注：默认情况下，XlsxWriter将所有单元数据保存在内存中。当格式化与数据分开应用时，这将允许将来使用功能。这样做的结果是XlsxWriter会消耗大量内存，并且在创建大文件时可能会耗尽内存。

解决办法：

- 我们可以通过Workbook() 的 constant_memory 属性来几乎完全减少内存使用量
workbook = xlsxwriter.Workbook(filename, {\'constant_memory\': True})

源码：

当设置为True时源码解释        
    # Write out the worksheet data as a single row with cells.
    # This method is used when constant_memory is on. A single
    # row is written and the data table is reset. That way only
    # one row of data is kept in memory at any one time. We don\'t
    # write span data in the optimized case since it is optional.
官网解释：
# Ok. With \'constant_memory\' you must write data in row by column order.
for row in range(0, row_max):
    for col in range(0, col_max):
        worksheet.write(row, col, some_data)
# Not ok. With \'constant_memory\' this will only write the first column of data.
for col in range(0, col_max):
    for row in range(0, row_max):
        worksheet.write(row, col, some_data)

缺点：

无法在写入单元数据后利用任何新功能来操纵单元数据。当前，该add_table()方法不适用于此模式，merge_range()并且set_row()仅适用于当前行。

二.openpyxl可以对excel文件进行读写操作

示例1：

from openpyxl import Workbook
from openpyxl import load_workbook
from openpyxl.writer.excel import ExcelWriter 
workbook_ = load_workbook(u"test1477881109469.xlsx")
sheetnames =workbook_.get_sheet_names() #获得表单名字
print sheetnames
sheet = workbook_.get_sheet_by_name(sheetnames[0])
print sheet.cell(row=3,column=3).value
sheet[\'A1\'] = \'47\'
workbook_.save(u"test1477881109469_new.xlsx") 
wb = Workbook()
ws = wb.active
ws[\'A1\'] = 4
wb.save("test.xlsx")

示例2

import openpyxl
# 新建文件
workbook = openpyxl.Workbook()
# 写入文件
sheet = workbook.activesheet[\'A1\']=\'A1\'
# 保存文件
workbook.save(\'test.xlsx\')

三.xlrd 主要是用来读取excel文件

import xlrd
data = xlrd.open_workbook(\'abcd.xls\') # 打开xls文件
table = data.sheets()[0] # 打开第一张表
nrows = table.nrows # 获取表的行数
for i in range(nrows): # 循环逐行打印
    if i == 0:# 跳过第一行
        continue
    print (table.row_values(i)[:13]) # 取前十三列

四.xlutils结合xlrd可以达到修改excel文件目的

import xlrd
from xlutils.copy import copy
workbook = xlrd.open_workbook(\'xxxxx.xls\')
workbooknew = copy(workbook)
ws = workbooknew.get_sheet(0)
ws.write(3, 0, \'changed!\')
workbooknew.save(\'xxxx.xls\')

五.xlwt主要是用来写excel文件

import xlwt
wbk = xlwt.Workbook()
sheet = wbk.add_sheet(\'sheet 1\')
sheet.write(0,1,\'test text\')#第0行第一列写入内容
wbk.save(\'xxxx.xls\')

以上是 python 操作excel工具及xlsxwriter处理数据量大的方法 的全部内容， 来源链接： utcz.com/z/388803.html

回到顶部