{"id":63,"date":"2023-01-13T04:40:35","date_gmt":"2023-01-13T04:40:35","guid":{"rendered":"https:\/\/othman.benomar.fr.to\/blog\/?p=63"},"modified":"2023-01-13T04:45:18","modified_gmt":"2023-01-13T04:45:18","slug":"generate-large-amount-of-images-without-crashes-in-python","status":"publish","type":"post","link":"https:\/\/othmanbenomar.dev\/blog\/2023\/01\/13\/generate-large-amount-of-images-without-crashes-in-python\/","title":{"rendered":"Generate large amount of images without crashes in Python"},"content":{"rendered":"\n<p>Python is an excellent programming language to quickly develop small applications. However it has substantial issues in terms of memory management and of speed. One particular inefficiency concerns the creation of a large quantities of images with for example the following code,<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\nNiter=10000\nfor i in range(Niter):\n     fig, ax = plt.subplots(1, figsize=(12, 6))\n     ax.plot(np.random.normal(1, 10, 100))\n     fig.savefig('fig_' + str(i) + '.jpg', dpi=300)\n     plt.close()<\/code><\/pre>\n\n\n\n<p>Despite a <code>plt.close()<\/code>, you may saturate the memory of your system and eventually python will crash.<\/p>\n\n\n\n<p>What is the problem here? Python incrementally index figures and still keep them in memory. If you want to completely erase from memory the figure, you need the following modification,<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><meta charset=\"utf-8\">import numpy as np\n<meta charset=\"utf-8\">import matplotlib.pyplot as plt\nNiter=10000\nfor i in range(Niter):\n     fig, ax = plt.subplots(1, figsize=(12, 6), num=1, clear=True)\n     ax.plot(np.random.normal(1, 10, 100))\n     fig.savefig('fig_' + str(i) + '.jpg', dpi=300)<\/code><\/pre>\n\n\n\n<p>We have added <code>num=1, clear=True<\/code> as arguments to the subplots function. This imposes to Python to overwrite the figure storage <code>num=1<\/code> with the new plot after having erased it (<code><meta charset=\"utf-8\">clear=True<\/code>). Now, you will not experience any more any exponential growth of the memory usage of Python due to image generation.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Python is an excellent programming language to quickly develop small applications. However it has substantial issues in terms of memory management and of speed. One particular inefficiency concerns the creation of a large quantities of images with for example the following code, Despite a plt.close(), you may saturate the memory of your system and eventually [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,3],"tags":[],"class_list":["post-63","post","type-post","status-publish","format-standard","hentry","category-python","category-technology"],"_links":{"self":[{"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/posts\/63","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/comments?post=63"}],"version-history":[{"count":2,"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/posts\/63\/revisions"}],"predecessor-version":[{"id":66,"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/posts\/63\/revisions\/66"}],"wp:attachment":[{"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/media?parent=63"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/categories?post=63"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/othmanbenomar.dev\/blog\/wp-json\/wp\/v2\/tags?post=63"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}