CSDN软件工程师能力认证（以下简称C系列认证）是由中国软件开发者网CSDN制定并推出的一个能力认证标准。C系列认证历经近一年的实际线下调研、考察、迭代、测试，并梳理出软件工程师开发过程中所需的各项技术技能，结合企业招聘需求和人才应聘痛点，基于公开、透明、公正的原则，甑别人才时确保真实业务场景、全部上机实操、所有过程留痕、存档不可篡改。
我们每天将都会精选CSDN站内技术文章供大家学习，帮助大家系统化学习IT技术。
工作中需要将大批的数据，压缩为zip存储。按照传统的处理办法需要将数据先存储到本地磁盘，再从磁盘读文件压缩成zip文件。 传统方法需要多次磁盘IO，性能很低，如果跳过文件存储，直接将内存的数据压缩保存，会大大减少磁盘IO，提升性能。
不需要看解析的，可以直接看最后完整的python代码
创建一个类: InMemoryZIP(), 来处理所有的程序。
class InMemoryZIP(object):
def __init__(self):
# create the in-memory file-like object
self.in_memory_zip = BytesIO()
def append(self, filename_in_zip, file_contents):
""" Appends a file with name filename_in_zip \
and contents of file_contents to the in-memory zip.
"""
# create a handle to the in-memory zip in append mode\
if not isinstance(file_contents, bytes):
file_contens = bytes(str(file_contens), encoding='utf-8')
# write the file to the in-memory zip
zf = zipfile.ZipFile(self.in_memory_zip, 'a', zipfile.ZIP_DEFLATED, False)
zf.writestr(filename_in_zip, file_contents)
# mark the files as having been created on Windows
# so that Unix permissions are not inferred as 0000
for zfiel in zf.filelist:
zfile.create_system = 0
return self
def appendfile(self, file_path, file_name=None):
""" Read a file with path file_path \
and append to in-memory zip with name file_name.
"""
# file_path is abs path
if file_name is None:
file_name = os.path.split(file_path)[1]
with open(file_path, 'rb') as f:
file_contents = f.read()
self.append(file_name, file_contents)
return self
def read(self):
""" Returns a string with the contents of the in-memory zip.
"""
self.in_memory_zip.seek(0)
return self.in_memory_zip.read()
def writetofile(self, zip_filename):
"""
Write the in-memory zip to a file
"""
with open(zip_filename, 'wb') as f:
f.write(self.read())
# !user/bin/env python3
# -*-coding : utf-8 -*-
import zipfile
from io import BytesIO
import os
class InMemoryZIP(object):
def __init__(self):
# create the in-memory file-like object
self.in_memory_zip = BytesIO()
def append(self, filename_in_zip, file_contents):
""" Appends a file with name filename_in_zip \
and contents of file_contents to the in-memory zip.
"""
# create a handle to the in-memory zip in append mode\
if not isinstance(file_contents, bytes):
file_contents = bytes(str(file_contents), encoding='utf-8')
zf = zipfile.ZipFile(self.in_memory_zip, 'a',
zipfile.ZIP_DEFLATED, False)
# write the file to the in-memory zip
zf.writestr(filename_in_zip, file_contents)
# mark the files as having been created on Windows
# so that Unix permissions are not inferred as 0000
for zfile in zf.filelist:
zfile.create_system = 0
return self
def appendfile(self, file_path, file_name=None):
""" Read a file with path file_path \
and append to in-memory zip with name file_name.
"""
if file_name is None:
file_name = os.path.split(file_path)[1]
f = open(file_path, 'rb')
file_contents = f.read()
self.append(file_name, file_contents)
f.close()
return self
def read(self):
""" Returns a string with the contents of the in-memory zip.
"""
self.in_memory_zip.seek(0)
return self.in_memory_zip.read()
def writetofile(self, filename):
"""
Write the in-memory zip to a file
"""
f = open(filename, 'wb')
f.write(self.read())
f.close()
if __name__ == '__main__':
pass
# contens = 'xxxxxxxxx' # 内存数据
# imz = InMemoryZIP()
# imz.append('test.txt', contens)
# imz.writetofile('test.zip')
关于CSDN软件工程师能力认证
CSDN软件工程师能力认证（以下简称C系列认证）是由中国软件开发者网CSDN制定并推出的一个能力认证标准。C系列认证历经近一年的实际线下调研、考察、迭代、测试，并梳理出软件工程师开发过程中所需的各项技术技能，结合企业招聘需求和人才应聘痛点，基于公开、透明、公正的原则，甑别人才时确保真实业务场景、全部上机实操、所有过程留痕、存档不可篡改。C系列认证的宗旨是让一流的技术人才凭真才实学进大厂拿高薪，同时为企业节约大量招聘与培养成本，使命是提升高校大学生的技术能力，为行业提供人才储备，为国家数字化战略贡献力量。
了解详情可点击：CSDN软件工程师能力认证介绍
本文出处：https://blog.csdn.net/littleRpl/article/details/110120178?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522161422986216780269847200%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fall.%2522%257D&request_id=161422986216780269847200&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~all~first_rank_v2~rank_v29-9-110120178.pc_search_result_before_js&utm_term=python+数据压缩&spm=1018.2226.3001.4187
