Sunday, September 14, 2025

Split large file into mutiple files via Python

Below python code shows how to split large file into multiple small files. Here reads ach 1000 lines from files and write into small file. 


from unidecode import unidecode
lines_per_file = 1000
smallfile = None
with open('large-file.txt',encoding="utf8") as bigfile:
    for lineno, line in enumerate(bigfile):
        if lineno % lines_per_file == 0:
            if smallfile:
                smallfile.close()
            small_filename = 'small_file_{}.txt'.format(lineno + lines_per_file)
            smallfile = open(small_filename, "w", encoding="utf-8")
        smallfile.write(line)
    if smallfile:
        smallfile.close()

No comments: