Message 199649 - Python tracker

➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author	tebeka
Recipients	tebeka
Date	2013-10-13.01:42:57
SpamBayes Score	-1.0
Marked as misclassified	Yes
Message-id	<[email protected]>
In-reply-to

Content
Currently glob.iglob calls os.listdir internally. Which means that if there are many files in the directory - a big list of them is created in memory. iglob should try to use readdir and be a "true" iterator, not consuming a lot of memory. See one possible implementation using ctypes at http://stackoverflow.com/questions/4403598/list-files-in-a-folder-as-a-stream-to-begin-process-immediately

Currently glob.iglob calls os.listdir internally. Which means that if there are many files in the directory - a big list of them is created in memory.

iglob should try to use readdir and be a "true" iterator, not consuming a lot of memory.

See one possible implementation using ctypes at http://stackoverflow.com/questions/4403598/list-files-in-a-folder-as-a-stream-to-begin-process-immediately

History
Date	User	Action	Args
2013-10-13 01:42:58	tebeka	set	recipients: + tebeka
2013-10-13 01:42:58	tebeka	set	messageid: <[email protected]>
2013-10-13 01:42:57	tebeka	link	issue19240 messages
2013-10-13 01:42:57	tebeka	create