Preloading images stored in a database

[Keith Hughitt]

Hi all,

I am having trouble preloading images in a javascript application, and
was wondering if anyone had any suggestions.
Basically I have a bunch of images stored in a database as BLOBs. At
any given point in time a subset of those images is displayed on-
screen. At certains times I want to swap out those on screen with new
ones from the database, and do some as seamlessly as possible.

So what I've tried to do is first create Image objects for each of the
new images to load. I then reference all of the old image's dom-nodes,
append the new images to the document, and finally remove the old dom-
nodes.

It isn't working so well though. After doing some looking around in
Firebug, it seems that by the time I'm ready to remove the old images,
only some of the new images have been preloaded (the rest are still
being retrieved according to firebug). I'm wondering if this has to do
with the fact that I'm making a number of server-side calls, and the
lag-time it takes to do so is what is causing only some of the images
to be displayed.

Any ideas?

Any help would be greatly appreciated.


Thanks!
Keith

[Thomas 'PointedEars' Lahn]

Re: Preloading images stored in a database

Keith Hughitt wrote:

I am having trouble preloading images in a javascript application, and
was wondering if anyone had any suggestions. Basically I have a bunch of
images stored in a database as BLOBs.

That might not be the best approach. BLOBs increase the size of your
database heavily, thereby reducing its performance and that of the server.
You might want to use the filesystem for storing image files instead; it is
optimized for this kind of data. Then you can store only the path of the
file in the database, which is optimized for that kind of data, and let the
server-side application handle the rest.

At any given point in time a subset of those images is displayed on-
screen. At certains times I want to swap out those on screen with new
ones from the database, and do some as seamlessly as possible.
>
So what I've tried to do is first create Image objects for each of the
new images to load.

This might or might not cache the image client-side, depending on the cache
settings. In the Worst Case it would error out (unless you did some proper
runtime feature tests before), because `Image' objects are proprietary host
objects; in the Second Worst Case you would be downloading the image data twice.

I then reference all of the old image's dom-nodes, append the new images
to the document, and finally remove the old dom- nodes.
>
It isn't working so well though.

It is not supposed to in your case.

After doing some looking around in Firebug, it seems that by the time I'm
ready to remove the old images, only some of the new images have been
preloaded (the rest are still being retrieved according to firebug).
>
I'm wondering if this has to do with the fact that I'm making a number of
server-side calls, and the lag-time it takes to do so is what is causing
only some of the images to be displayed.

That is a distinct possibility.

Any ideas?

You should modify the `src' property of the existing (HTML)Image(Ele ment)
objects instead. Use cache-controlling headers to make sure you GET the
current images, from the server.

While your searching for "image rollover" or "image hover" before posting
would have already helped (see <http://jibbering.com/faq/>), this might give
you some ideas:

<http://PointedEars.de/hoverMe>


HTH

PointedEars
--
var bugRiddenCrashP ronePieceOfJunk = (
navigator.userA gent.indexOf('M SIE 5') != -1
&& navigator.userA gent.indexOf('M ac') != -1
) // Plone, register_functi on.js:16

[Bart Van der Donck]

Re: Preloading images stored in a database

Thomas 'PointedEars' Lahn wrote:

Keith Hughitt wrote:

>I am having trouble preloading images in a javascript application, and
>was wondering if anyone had any suggestions. Basically I have a bunch of
>images stored in a database as BLOBs.

>
That might not be the best approach.  BLOBs increase the size of your
database heavily, thereby reducing its performance and that of the server..
You might want to use the filesystem for storing image files instead; it is
optimized for this kind of data.  Then you can store only the path of the
file in the database, which is optimized for that kind of data, and let the
server-side application handle the rest.

That is a good step indeed, but still a waste of database memory. The
uploaded image needs to get a name on server anyhow - just calling it
2546.jpg for record ID 2546 saves one long data column. And the more
images in a record, the more memory is saved (like 2546-A-thumb.jpg
etc.)

--
Bart

[Joost Diepenmaat]

Re: Preloading images stored in a database

Bart Van der Donck <bart@nijlen.co mwrites:

Thomas 'PointedEars' Lahn wrote:
>

>Keith Hughitt wrote:

>>I am having trouble preloading images in a javascript application, and
>>was wondering if anyone had any suggestions. Basically I have a bunch of
>>images stored in a database as BLOBs.

>>
>That might not be the best approach.  BLOBs increase the size of your
>database heavily, thereby reducing its performance and that of the server.
>You might want to use the filesystem for storing image files instead; it is
>optimized for this kind of data.  Then you can store only the path of the
>file in the database, which is optimized for that kind of data, and let the
>server-side application handle the rest.

>
That is a good step indeed, but still a waste of database memory. The
uploaded image needs to get a name on server anyhow - just calling it
2546.jpg for record ID 2546 saves one long data column. And the more
images in a record, the more memory is saved (like 2546-A-thumb.jpg
etc.)

Not to say that putting large blobs in the database is in general a
good idea, but it does have some advantages. For one, if you've got a
replicating database setup, you won't need any special tricks to also
replicate the "files", which makes data integrity less of an issue.

Also, some clever HTTP caching can reduce the server load for images
quite a bit, making the relative slowness of the database less of an
issue.

--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/

[Keith Hughitt]

Re: Preloading images stored in a database

Thank you both for your replies. Regarding the initial problem of
preloading the images, I figured out what was wrong.
The problem I was having wasn't actually a *preloading* problem (even
though that's how I thought of it at first), but
rather a sequence issue. The images were in fact being 'preloaded,'
but because the time from when they started to be
preloaded and the time when they were being displayed was 0, the
effect was the same as if they were preloaded. My case
is different from simple hover buttons and the such because the images
I wanted to load are dynamic, and only determined
after the user takes some action (at which point they need to be
loaded immediately).

The solution was to create an event listener for each images "load"
event. The event handler simply increments a counter
(numLoaded), and checks to see whether it is equal to the total number
of images. Once that condition is true, the old
image dom-nodes can be removed. This guaranties two things: 1) That
when the new images are loaded, they load at (about)
the same time, which is important for me since they are parts of a
larger single image, and 2) that there is no visible
gap between the old images and new ones (Before, there was a period
after the old images had been removed and before the
new ones were added when the screen was blank).

In regards to the issue of where to store the images I appreciate the
suggestions. In fact, originally, the images *were*
kept in the filesystem, but due to the very large number of images (>
1 million), and limitations from the number of i-nodes
we are allowed to occupy on our web-hosting, we shifted to the
database approach. This also made things slightly simpler
in that both the images and their associated meta information was kept
in a single place, and could be fetched in a single step.

If the performance hit from storing images in the database is very
significant, however, it may be worthwhile to try and
find a way to get back to using the filesystem. Would you happen to
know of any references comparing the two approaches?


Thanks again for the feedback and suggestions.


Take care,
Keith


On Jul 18, 2:02 pm, Bart Van der Donck <b...@nijlen.co mwrote:

Thomas 'PointedEars' Lahn wrote:

Keith Hughitt wrote:

I am having trouble preloading images in a javascript application, and
was wondering if anyone had any suggestions. Basically I have a bunch of
images stored in a database as BLOBs.

>

That might not be the best approach.  BLOBs increase the size of your
database heavily, thereby reducing its performance and that of the server.
You might want to use the filesystem for storing image files instead; it is
optimized for this kind of data.  Then you can store only the path ofthe
file in the database, which is optimized for that kind of data, and letthe
server-side application handle the rest.

>
That is a good step indeed, but still a waste of database memory. The
uploaded image needs to get a name on server anyhow - just calling it
2546.jpg for record ID 2546 saves one long data column. And the more
images in a record, the more memory is saved (like 2546-A-thumb.jpg
etc.)
>
--
 Bart

[Keith Hughitt]

Re: Preloading images stored in a database

Also, some clever HTTP caching can reduce the server load for images

quite a bit, making the relative slowness of the database less of an
issue.
>
--
Joost Diepenmaat | blog:http://joost.zeekat.nl/| work:http://zeekat.nl/

Hi Joost,

Could you recommend any good places to start learning about how this
might be done?

Thanks,
Keith

[Bart Van der Donck]

Re: Preloading images stored in a database

Joost Diepenmaat wrote:

Not to say that putting large blobs in the database is in general a
good idea, but it does have some advantages. For one, if you've got a
replicating database setup, you won't need any special tricks to also
replicate the "files", which makes data integrity less of an issue.

Yes, I suppose this could be a benefit too. Personally I consider it
as a rule of thumb that storing BLOBs is a bad idea. I might be a bit
pre-occupied since I need to do a lot of performance tuning and I have
seen enough what havoc they can cause when not used with care.

Also, some clever HTTP caching can reduce the server load for images
quite a bit, making the relative slowness of the database less of an
issue.

Once you enter the field of larger BLOB's, I think there isn't much
other choice :-)

--
Bart

[Joost Diepenmaat]

Re: Preloading images stored in a database

Keith Hughitt <keith.hughitt@ gmail.comwrites :

>Also, some clever HTTP caching can reduce the server load for images
>quite a bit, making the relative slowness of the database less of an
>issue.

[ ... ]

Could you recommend any good places to start learning about how this
might be done?

In short, you need to send out the right HTTP headers to make caching
possible, and if at all possible, you want URLs to images to *never*
change the content so you can set extemely long caching times, IOW,
new versions of the same image should get a new URL (probably by
adding a version nr or timestamp to the url).

Once that's done, any standard http caching proxy (like
http://www.squid-cache.org/ ) can be put "in front" of the webserver
and it will keep any duplicate requests from getting to the web server
doing this correctly can also help browsers to cache files
locally. You can even create distributed stacks of caches and/or CDNs
to spread the load over more machines.

http://www.mnot.net/cache_docs/ seems to be a reasonable introduction.

All this works not just for images, but for any HTTP resource, but
images are especially important because they tend to make up the bulk
of the traffic of most sites, and other resources tend to change more
often, making long-term caching more challenging.

--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/