<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Sitek Blog &#187; blob</title>
	<atom:link href="http://blog.sitek.com.au/tag/blob/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.sitek.com.au</link>
	<description>About IT and Technology</description>
	<lastBuildDate>Fri, 03 Jul 2009 23:37:54 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Comparison between storing images/files in mySQL and on filesystem.</title>
		<link>http://blog.sitek.com.au/2008/03/comparison-between-storing-imagesfiles-in-mysql-and-on-filesystem/</link>
		<comments>http://blog.sitek.com.au/2008/03/comparison-between-storing-imagesfiles-in-mysql-and-on-filesystem/#comments</comments>
		<pubDate>Thu, 06 Mar 2008 13:16:20 +0000</pubDate>
		<dc:creator>James Moey</dc:creator>
				<category><![CDATA[Database]]></category>
		<category><![CDATA[PHP]]></category>
		<category><![CDATA[blob]]></category>
		<category><![CDATA[images]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[optimisation]]></category>

		<guid isPermaLink="false">http://bblog.sitek.com.au/?p=3</guid>
		<description><![CDATA[There was a discussion in regard to &#8220;store images in MySQL&#8221; in one of WebHostingTalk forum. Many believe that storing images/large set of data in mySQL will hurt performance, specially when you want to do query. Unfortunately I could not find any resource in regard to this or support this. So I set off to [...]]]></description>
			<content:encoded><![CDATA[<p>There was a discussion in regard to &#8220;<span class="navbar"><strong>store images in MySQL</strong>&#8221; in one of WebHostingTalk forum. Many believe that storing images/large set of data in mySQL will hurt performance, specially when you want to do query. Unfortunately I could not find any resource in regard to this or support this. So I set off to do my own performance test run. The test is written using PHP, since PHP/mySQL is such a popular combination. </span></p>
<p>Test run are ran on following specs:</p>
<p>CPU: Intel 4 3.06GHz<br />
RAM: 1 GB.<br />
Linux Distribution: Ubuntu 6.2 Server edition</p>
<p>mySQL version 5.0.45<br />
PHP-cli 5.2.3</p>
<p><span id="more-3"></span></p>
<p>The image file is 2.4MB, in total there is 1000 copies in the database. Overall, the physical size of the database file is 2.5GB.</p>
<p>The database structure contain 2 tables, image and imageblob.</p>
<ul>
<li>Table image have id, name, imageid and imagelocation as it&#8217;s field.</li>
<li> Table imageblob have id and image as it&#8217;s field.</li>
</ul>
<p>Let me explain what each field contain;</p>
<ul>
<li>image.id &#8211; primary id, integer type, unique and incremental.</li>
<li>image.name &#8211; varchar type.</li>
<li>image.imageid &#8211; forgein key to imageblob table, map to imageblob.id, indexed.</li>
<li>image.imagelocation &#8211; text type, to store path of the image location.</li>
<li>imageblob.id &#8211; primary id, integer type.</li>
<li>imageblob.image &#8211; mediumblob type, this is where the binary data is stored.</li>
</ul>
<p>Why did I have 2 table? I will let tell you later.</p>
<p>So first thing I do is insert the image file into those 2 tables.</p>
<pre class="php">
<span class="phpFunction">mysql_connect</span><span class="phpOperator">(</span><span class="phpString">"localhost"</span>, <span class="phpString">"test"</span>, <span class="phpString">"xxxxxx"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpFunction">mysql_select_db</span><span class="phpOperator">(</span><span class="phpString">"mysqltester_blob"</span><span class="phpOperator">)</span><span class="phpText">;</span>
$filecontent <span class="phpOperator">=</span> <span class="phpFunction">file_get_contents</span><span class="phpOperator">(</span><span class="phpString">"map_large_community<span class="phpOperator">.</span>png"</span>, FILE_BINARY<span class="phpOperator">)</span><span class="phpText">;</span>
$imgid <span class="phpOperator">=</span> 0;
<span class="phpKeyword">
for </span><span class="phpOperator">(</span>$i <span class="phpOperator">=</span> <span class="phpNumber">1</span><span class="phpText">;</span> $i <span class="phpOperator">&lt;</span> 1000; $i<span class="phpOperator"><span class="phpOperator">+</span><span class="phpOperator">+</span></span><span class="phpOperator">)</span> <span class="phpOperator">{</span>
$imgid <span class="phpOperator">=</span> $i<span class="phpText">;</span>
<span class="phpFunction">mysql_query</span><span class="phpOperator">(</span><span class="phpString">"INSERT INTO imageblob <span class="phpOperator">(</span>id, image<span class="phpOperator">)</span> VALUES <span class="phpOperator">(</span>"</span> <span class="phpOperator">.</span> $imgid <span class="phpOperator">.</span> <span class="phpString">",<span class="phpString">'".mysql_real_escape_string($filecontent)."'</span><span class="phpOperator">)</span>"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpFunction">mysql_query</span><span class="phpOperator">(</span><span class="phpString">"INSERT INTO image <span class="phpOperator">(</span>name, imageid<span class="phpOperator">)</span> VALUES <span class="phpOperator">(</span><span class="phpString">'testing"</span>.$i<span class="phpOperator">.</span><span class="phpString">"'</span>, "</span> <span class="phpOperator">.</span>$imgid<span class="phpOperator">.</span> <span class="phpString">"<span class="phpOperator">)</span>"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpOperator">}</span>
</pre>
<p>I ran the above script 3 time to profile the performance. 1000 records were insert into the table in 2minutes and 38 secs. That translate to 0.158sec per record. Not bad, for each record containing 2.4MB worth of data.</p>
<p>Next step is to do the selection from those table. I will need 2 PHP script,</p>
<ol>
<li>Will query the table and retrieve the image from the file system.</li>
<li>Will query the table and retrieve the image from the database.</li>
</ol>
<p style="text-align: center">&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-</p>
<pre class="php">
<span class="phpFunction">mysql_connect</span><span class="phpOperator">(</span><span class="phpString">"localhost"</span>, <span class="phpString">"test"</span>, <span class="phpString">"xxxxxx"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpFunction">mysql_select_db</span><span class="phpOperator">(</span><span class="phpString">"mysqltester_blob"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpKeyword">
for </span><span class="phpOperator">(</span>$i <span class="phpOperator">=</span> 0; $i <span class="phpOperator">&lt;</span> 10; $i<span class="phpOperator"><span class="phpOperator">+</span><span class="phpOperator">+</span></span><span class="phpOperator">)</span> <span class="phpOperator">{</span>
$result <span class="phpOperator">=</span> <span class="phpFunction">mysql_query</span><span class="phpOperator">(</span><span class="phpString">"SELECT image.id, name, imagelocation FROM image WHERE name <span class="phpOperator">=</span> <span class="phpString">'testing"</span>.<span class="phpFunction">rand</span><span class="phpOperator">(</span><span class="phpNumber">1</span>,500<span class="phpOperator">)</span>.<span class="phpString">"'</span>"</span><span class="phpOperator">)</span><span class="phpText">;</span>
$output <span class="phpOperator">=</span> <span class="phpFunction">file_get_contents</span><span class="phpOperator">(</span><span class="phpString">"map_large_team<span class="phpOperator">.</span>png"</span>, FILE_BINARY<span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpKeyword">
while </span><span class="phpOperator">(</span>$row <span class="phpOperator">=</span> <span class="phpFunction">mysql_fetch_array</span><span class="phpOperator">(</span>$result, MYSQL_NUM<span class="phpOperator">)</span><span class="phpOperator">)</span> <span class="phpOperator">{</span>
<span class="phpFunction">printf</span><span class="phpOperator">(</span><span class="phpString">"ID<span class="phpOperator">:</span> %s  Name<span class="phpOperator">:</span> %s Size<span class="phpOperator">:</span> %d\n"</span>, $row<span class="phpOperator">[</span><span class="phpNumber">0</span><span class="phpOperator">]</span>, $row<span class="phpOperator">[</span><span class="phpNumber">1</span><span class="phpOperator">]</span>, <span class="phpFunction">strlen</span><span class="phpOperator">(</span>$output<span class="phpOperator">)</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpOperator">}</span>
<span class="phpOperator">}</span>
</pre>
<p>This php script will retrieve image from the <em>file-system after database query</em>. It ran 3 times and the results are:</p>
<ul>
<li>0.144sec</li>
<li>0.144sec</li>
<li>0.144sec</li>
</ul>
<p style="text-align: center">&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-</p>
<pre class="php">
<span class="phpFunction">mysql_connect</span><span class="phpOperator">(</span><span class="phpString">"localhost"</span>, <span class="phpString">"tester"</span>, <span class="phpString">"xxxxxx"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpFunction">mysql_select_db</span><span class="phpOperator">(</span><span class="phpString">"mysqltester_blob"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpKeyword">
for </span><span class="phpOperator">(</span>$i <span class="phpOperator">=</span> 0; $i <span class="phpOperator">&lt;</span> 10; $i<span class="phpOperator"><span class="phpOperator">+</span><span class="phpOperator">+</span></span><span class="phpOperator">)</span> <span class="phpOperator">{</span>
$result <span class="phpOperator">=</span> <span class="phpFunction">mysql_query</span><span class="phpOperator">(</span><span class="phpString">"SELECT image.id, name, image FROM image inner join imageblob on image.imageid <span class="phpOperator">=</span> imageblob<span class="phpOperator">.</span>id WHERE image.name <span class="phpOperator">=</span> <span class="phpString">'testing"</span>.<span class="phpFunction">rand</span><span class="phpOperator">(</span><span class="phpNumber">1</span>,1000<span class="phpOperator">)</span>.<span class="phpString">"'</span>"</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpKeyword">
while </span><span class="phpOperator">(</span>$row <span class="phpOperator">=</span> <span class="phpFunction">mysql_fetch_array</span><span class="phpOperator">(</span>$result, MYSQL_NUM<span class="phpOperator">)</span><span class="phpOperator">)</span> <span class="phpOperator">{</span>
<span class="phpFunction">printf</span><span class="phpOperator">(</span><span class="phpString">"ID<span class="phpOperator">:</span> %s  Name<span class="phpOperator">:</span> %s Size<span class="phpOperator">:</span> %d\n"</span>, $row<span class="phpOperator">[</span><span class="phpNumber">0</span><span class="phpOperator">]</span>, $row<span class="phpOperator">[</span><span class="phpNumber">1</span><span class="phpOperator">]</span>, <span class="phpFunction">strlen</span><span class="phpOperator">(</span>$row<span class="phpOperator">[</span><span class="phpNumber">2</span><span class="phpOperator">]</span><span class="phpOperator">)</span><span class="phpOperator">)</span><span class="phpText">;</span>
<span class="phpOperator">}</span>
<span class="phpOperator">}</span>
</pre>
<p>This php script will <em>retrieve image from the database</em>. It ran 3 times and the results are:</p>
<ul>
<li>1.389sec</li>
<li>1.337sec</li>
<li>1.256sec</li>
</ul>
<p style="text-align: center">&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-</p>
<p>As you can see from the result, the difference of performance very minimal. At the beginning of the test run, I was using 1 table to store both image detail and image blog. The performance was extremely poor, a similar SELECT query on name took a few minute to complete. I think it is because mySQL is going through all the huge blob column while search for the matching name. Splitting image detail and the image content help the query speed up tremendously. That is why I have split the data into 2 tables. Another important tip is image.imageid and imageblob.id must be indexed, it also help with the query.</p>
<p>In conclusion, we as programmer should see store large document/image in database as a solution. It&#8217;s performance is comparable to storing large document/image on file system. There are many advantage for store large data on database. At the end, as long as the table is structure correctly, the performance issue will not be an issue.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.sitek.com.au/2008/03/comparison-between-storing-imagesfiles-in-mysql-and-on-filesystem/feed/</wfw:commentRss>
		<slash:comments>10</slash:comments>
		</item>
	</channel>
</rss>
