使用peewee+sqlite批量插入数据报错
peewee.OperationalError: too many SQL variables
sqlite 每次插入有数量限制,不能插入太多,资料显示999,不过我设置为500就报错了
peewee提供了一个方式,分次插入,两种方式等价
from peewee import chunked
# Insert rows 100 at a time.
with db.atomic():
for idx in range(0, len(data_source), 100):
MyModel.insert_many(data_source[idx:idx+100]).execute()
# Insert rows 100 at a time.
with db.atomic():
for batch in chunked(data_source, 100):
MyModel.insert_many(batch).execute()
参考:
- How to increase SQLITE_MAX_VARIABLE_NUMBER at compilation-time?
- http://docs.peewee-orm.com/en/latest/peewee/querying.html#bulk-inserts